How to use http.Transport in Go to implement breakpoint resume transfer of large files?

How to use http.Transport in Go to implement breakpoint resume transfer of large files?

During the network transmission process, the transmission of large files often takes a long time. In order to improve the efficiency and stability of transmission, breakpoint resume transmission technology has become a common solution. In the Go language, we can use http.Transport to implement breakpoint resume transfer of large files, making network transmission more stable and reliable. This article will introduce how to use http.Transport to implement breakpoint resumption and provide corresponding code examples.

1. Initialize http.Transport

First, we need to initialize an http.Transport object, which will be responsible for network transmission. We can control some details of the transmission by configuring the parameters of the http.Transport object.

transport := &http.Transport{
    MaxIdleConnsPerHost:   10,
    DisableCompression:    true,
    DisableKeepAlives:     true,
    ResponseHeaderTimeout: time.Second * 5,
}

In the above code, we set the maximum number of idle connections to 10, disabled the compression function and the option to keep connections, and set the response header timeout to 5 seconds. These parameters can be adjusted according to actual needs.

2. Initiate HTTP request

Next, we can use the http.Transport object to initiate an HTTP request. Before initiating a request, we need to construct an http.Request object and specify the requested URL, request method, request header and other information.

req, err := http.NewRequest("GET", "http://example.com/large_file.zip", nil)
if err != nil {
    log.Fatal(err)
}
// 添加断点续传的相关请求头信息
if fileInfo, err := os.Stat("local_file.zip"); err == nil {
    req.Header.Set("Range", "bytes="+strconv.FormatInt(fileInfo.Size(), 10)+"-")
}

In the above code, we create a GET request using the http.NewRequest method and specify the URL of the large file to be downloaded. At the same time, we implement breakpoint resumption by setting the "Range" request header according to the size of the downloaded local file.

3. Execute HTTP request

After the request construction is completed, we can execute the HTTP request through the RoundTrip method of the http.Transport object and obtain the response result.

resp, err := transport.RoundTrip(req)
if err != nil {
    log.Fatal(err)
}
defer resp.Body.Close()

if resp.StatusCode != http.StatusPartialContent {
    // 如果服务器未返回部分内容，无法进行断点续传
    log.Fatal("Server does not support partial content")
}

// 检查是否支持断点续传
contentRange := resp.Header.Get("Content-Range")
if contentRange == "" {
    // 如果服务器未返回Content-Range头部，无法进行断点续传
    log.Fatal("Server does not support content range")
}

// 获取已下载的文件大小
currentSize, err := getCurrentFileSize(contentRange)
if err != nil {
    log.Fatal(err)
}

In the above code, we first check whether the server has returned part of the content (status code is 206). If the server does not return part of the content, the breakpoint resumable transmission will not be possible. Then, we confirm whether the server supports resumable downloads by checking the "Content-Range" field in the response header. If the server does not return the "Content-Range" field, the upload will not be resumed.

4. Downloading files

If the previous steps are passed successfully, that is, the server supports breakpoint resumption, we can start downloading the file and save the file locally.

out, err := os.OpenFile("local_file.zip", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
    log.Fatal(err)
}
defer out.Close()

_, err = io.Copy(out, resp.Body)
if err != nil {
    log.Fatal(err)
}

// 下载完成后，检查文件完整性
fileSize, err := getFileSize(contentRange)
if err != nil {
    log.Fatal(err)
}

if currentSize != fileSize {
    log.Fatal("Downloaded file size does not match")
}

fmt.Println("Download completed")

In the above code, we first open a local file to save the downloaded content. Then, the response content is written to the local file through the io.Copy function. Finally, we check that the downloaded file size matches the file size returned by the server to ensure file integrity.

At this point, we have completed the process of using http.Transport to implement breakpoint resume transfer of large files. Through the above steps, we can improve the efficiency and stability of transmission during network transmission and ensure the safe transmission of large files.

Summary

This article introduces how to use http.Transport in Go language to implement breakpoint resume transfer of large files. By properly configuring the parameters of the http.Transport object, we can control the details of the transmission, such as the maximum number of idle connections, compression function, request timeout, etc. At the same time, we can implement breakpoint resumption by setting the "Range" request header according to the size of the downloaded file. Finally, we write the response content to the local file through the io.Copy function and check the integrity of the file. These steps can help us improve transmission efficiency and stability in large file transfers and achieve a better user experience.

Reference code: https://gist.github.com/anonymous/043cdf4cf58e183d08c5ad7d01c8db82

The above is the detailed content of How to use http.Transport in Go to implement breakpoint resume transfer of large files?. For more information, please follow other related articles on the PHP Chinese website!