http.Transport optimization skills and practice sharing in Go language

Sharing of http.Transport optimization skills and practices in Go language

Introduction:
In Go language, using http.Transport to make network requests is a very common operation. However, in large-scale concurrent requests, unreasonable use of http.Transport can easily lead to performance bottlenecks in network requests. This article will explore some optimization techniques and practices to help developers better improve the performance of network requests.

1. Use connection pool
By default, http.Transport will create a concurrent connection for each HTTP request. The advantage of this is that it can make full use of the concurrent processing capabilities of the server, but it will also cause some problems. First, frequently creating and closing connections can cause unnecessary performance overhead. Secondly, the server may have a limit on the number of connections, and too many concurrent requests may cause a denial of service on the server. Therefore, using connection pooling can effectively avoid these problems.

Code example:

package main

import (
    "fmt"
    "net/http"
    "time"
)

var client *http.Client

func init() {
    transport := &http.Transport{
        MaxIdleConns:        100,               // 最大空闲连接数
        IdleConnTimeout:     90 * time.Second,  // 空闲连接的超时时间
        TLSHandshakeTimeout: 10 * time.Second,  // TLS握手的超时时间
        ExpectContinueTimeout: 1 * time.Second, // 100-continue状态响应的超时时间
    }
    client = &http.Client{
        Transport: transport,
        Timeout:   10 * time.Second, // 完整请求的超时时间
    }
}

func main() {
    resp, err := client.Get("https://www.example.com")
    if err != nil {
        fmt.Println("请求错误：", err)
        return
    }
    defer resp.Body.Close()

    // 处理响应
    // ...
}

2. Enable Keep-Alive

Keep-Alive is an HTTP persistent connection mechanism that allows the client and server to connect on the same TCP connection Send multiple HTTP requests and responses. Enabling Keep-Alive can reduce the cost of connection establishment and improve the performance of network requests.

In http.Transport, Keep-Alive is enabled by default. However, if the server does not support Keep-Alive, connections will still be created and closed frequently. Therefore, we can control the maximum number of idle connections per host by setting the MaxIdleConnsPerHost property of http.Transport, thereby reducing the frequency of connection creation and closing.

Code example:

package main

import (
    "fmt"
    "net/http"
    "time"
)

var client *http.Client

func init() {
    transport := &http.Transport{
        MaxIdleConns:        100,               // 最大空闲连接数
        MaxIdleConnsPerHost: 10,                // 每个host的最大空闲连接数
        IdleConnTimeout:     90 * time.Second,  // 空闲连接的超时时间
        TLSHandshakeTimeout: 10 * time.Second,  // TLS握手的超时时间
        ExpectContinueTimeout: 1 * time.Second, // 100-continue状态响应的超时时间
    }
    client = &http.Client{
        Transport: transport,
        Timeout:   10 * time.Second, // 完整请求的超时时间
    }
}

func main() {
    resp, err := client.Get("https://www.example.com")
    if err != nil {
        fmt.Println("请求错误：", err)
        return
    }
    defer resp.Body.Close()

    // 处理响应
    // ...
}

3. Enable connection reuse

In some scenarios where requests need to be sent frequently, enabling connection reuse can further optimize performance. Connection reuse refers to keeping the TCP connection open after sending a request so that the connection can continue to be used the next time the request is made.

Code example:

package main

import (
    "fmt"
    "net/http"
    "time"
)

var client *http.Client

func init() {
    transport := &http.Transport{
        MaxIdleConns:        100,                // 最大空闲连接数
        MaxIdleConnsPerHost: 10,                 // 每个host的最大空闲连接数
        IdleConnTimeout:     90 * time.Second,   // 空闲连接的超时时间
        TLSHandshakeTimeout: 10 * time.Second,   // TLS握手的超时时间
        ExpectContinueTimeout: 1 * time.Second,  // 100-continue状态响应的超时时间
        DisableKeepAlives:   false,              // 启用连接复用
    }
    client = &http.Client{
        Transport: transport,
        Timeout:   10 * time.Second,  // 完整请求的超时时间
    }
}

func main() {
    resp, err := client.Get("https://www.example.com")
    if err != nil {
        fmt.Println("请求错误：", err)
        return
    }
    defer resp.Body.Close()

    // 处理响应
    // ...
}

Conclusion:
By using connection pooling, enabling Keep-Alive and connection reuse, we can effectively improve the use of http.Transport for network requests in Go language performance. Of course, further optimization and adjustment may be required for different business scenarios and needs. Hope this article can be helpful to developers.

The above is the detailed content of http.Transport optimization skills and practice sharing in Go language. For more information, please follow other related articles on the PHP Chinese website!