How to use vue+springboot to upload large files-javaTutorial-php.cn

Home

Java

javaTutorial

How to use vue+springboot to upload large files

王林

May 16, 2023 am 09:43 AM

vuespringboot

Logic

If you need to upload a large file, you should consider the following logic:

To upload a large file, you generally need to upload the file in slices (chunks), and then upload all Slices are merged into a complete file. It can be implemented according to the following logic:
The front-end selects the file to be uploaded on the page and uses the Blob.slice method to slice the file. Generally, the size of each slice is a fixed value (such as 5MB) and record how many slices there are in total.
Upload the slices to the backend service separately, and you can use libraries such as XMLHttpRequest or Axios to send Ajax requests. For each slice, three parameters need to be included: current slice index (starting from 0), total number of slices, and slice file data.
After the backend service receives the slice, it saves it to a temporary file under the specified path, and records the uploaded slice index and upload status. If a slice fails to be uploaded, the front end is notified to retransmit the slice.
When all slices are uploaded successfully, the backend service reads the contents of all slices and merges them into a complete file. File merging can be achieved using java.io.SequenceInputStream and BufferedOutputStream.
Finally, return the response result of successful file upload to the front end.

Front-end

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>File Upload</title>
</head>
<body>
    <input type="file" id="fileInput">
    <button onclick="upload()">Upload</button>
    <script>
        function upload() {
            let file = document.getElementById("fileInput").files[0];
            let chunkSize = 5 * 1024 * 1024; // 切片大小为5MB
            let totalChunks = Math.ceil(file.size / chunkSize); // 计算切片总数
            let index = 0;
            while (index < totalChunks) {
                let chunk = file.slice(index * chunkSize, (index + 1) * chunkSize);
                let formData = new FormData();
                formData.append("file", chunk);
                formData.append("index", index);
                formData.append("totalChunks", totalChunks);
                // 发送Ajax请求上传切片
                $.ajax({
                    url: "/uploadChunk",
                    type: "POST",
                    data: formData,
                    processData: false,
                    contentType: false,
                    success: function () {
                        if (++index >= totalChunks) {
                            // 所有切片上传完成，通知服务端合并文件
                            $.post("/mergeFile", {fileName: file.name}, function () {
                                alert("Upload complete!");
                            })
                        }
                    }
                });
            }
        }
    </script>
</body>
</html>

Back-end

controller layer:

@RestController
public class FileController {

    @Value("${file.upload-path}")
    private String uploadPath;

    @PostMapping("/uploadChunk")
    public void uploadChunk(@RequestParam("file") MultipartFile file,
                            @RequestParam("index") int index,
                            @RequestParam("totalChunks") int totalChunks) throws IOException {
        // 以文件名+切片索引号为文件名保存切片文件
        String fileName = file.getOriginalFilename() + "." + index;
        Path tempFile = Paths.get(uploadPath, fileName);
        Files.write(tempFile, file.getBytes());
        // 记录上传状态
        String uploadFlag = UUID.randomUUID().toString();
        redisTemplate.opsForList().set("upload:" + fileName, index, uploadFlag);
        // 如果所有切片已上传，则通知合并文件
        if (isAllChunksUploaded(fileName, totalChunks)) {
            sendMergeRequest(fileName, totalChunks);
        }
    }

    @PostMapping("/mergeFile")
    public void mergeFile(String fileName) throws IOException {
        // 所有切片均已成功上传，进行文件合并
        List<File> chunkFiles = new ArrayList<>();
        for (int i = 0; i < getTotalChunks(fileName); i++) {
            String chunkFileName = fileName + "." + i;
            Path tempFile = Paths.get(uploadPath, chunkFileName);
            chunkFiles.add(tempFile.toFile());
        }
        Path destFile = Paths.get(uploadPath, fileName);
        try (OutputStream out = Files.newOutputStream(destFile);
             SequenceInputStream seqIn = new SequenceInputStream(Collections.enumeration(chunkFiles));
             BufferedInputStream bufIn = new BufferedInputStream(seqIn)) {
            byte[] buffer = new byte[1024];
            int len;
            while ((len = bufIn.read(buffer)) > 0) {
                out.write(buffer, 0, len);
            }
        }
        // 清理临时文件和上传状态记录
        for (int i = 0; i < getTotalChunks(fileName); i++) {
            String chunkFileName = fileName + "." + i;
            Path tempFile = Paths.get(uploadPath, chunkFileName);
            Files.deleteIfExists(tempFile);
            redisTemplate.delete("upload:" + chunkFileName);
        }
    }

    private int getTotalChunks(String fileName) {
        // 根据文件名获取总切片数
        return Objects.requireNonNull(Paths.get(uploadPath, fileName).toFile().listFiles()).length;
    }

    private boolean isAllChunksUploaded(String fileName, int totalChunks) {
        // 判断所有切片是否已都上传完成
        List<String> uploadFlags = redisTemplate.opsForList().range("upload:" + fileName, 0, -1);
        return uploadFlags != null && uploadFlags.size() == totalChunks;
    }

    private void sendMergeRequest(String fileName, int totalChunks) {
        // 发送合并文件请求
        new Thread(() -> {
            try {
                URL url = new URL("http://localhost:8080/mergeFile");
                HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                conn.setRequestMethod("POST");
                conn.setDoOutput(true);
                conn.setDoInput(true);
                conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=utf-8");
                OutputStream out = conn.getOutputStream();
                String query = "fileName=" + fileName;
                out.write(query.getBytes());
                out.flush();
                out.close();
                BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8));
                while (br.readLine() != null) ;
                br.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }).start();
    }

    @Autowired
    private RedisTemplate<String, Object> redisTemplate;
}

Among them, file.upload-path is the save file upload The path can be configured in application.properties or application.yml. At the same time, you need to add the Bean of RedisTemplate to record the upload status.

RedisTemplate configuration

If you need to use RedisTemplate, you need to introduce the following package

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

At the same time configure the redis information in yml

spring.redis.host=localhost
spring.redis.port=6379
spring.redis.database=0

Then in your own class When used like this

@Component
public class myClass {

    @Autowired
    private RedisTemplate<String, Object> redisTemplate;

    public void set(String key, Object value) {
        redisTemplate.opsForValue().set(key, value);
    }

    public Object get(String key) {
        return redisTemplate.opsForValue().get(key);
    }
}

Notes

It is necessary to control the size of the slices uploaded each time to take into account the upload speed and stability, and avoid occupying too many server resources or being affected by the network. The upload failed due to instability.
There is a sequence for uploading slices. You need to ensure that all slices are uploaded before merging. Otherwise, incomplete files or file merge errors may occur.
After the upload is completed, temporary files need to be cleaned up in time to avoid server crashes caused by taking up too much disk space. You can set up a periodic task to clean up expired temporary files.

The above is the detailed content of How to use vue+springboot to upload large files. For more information, please follow other related articles on the PHP Chinese website!

Statement

This article is reproduced at:亿速云. If there is any infringement, please contact admin@php.cn delete