How to use context to implement request concurrency control in Go

How to use context to implement request concurrency control in Go

Overview:
Concurrency (concurrency) in the Go language is one of its most important features. Using concurrency can significantly improve the performance and responsiveness of your program, especially when making network requests. However, in high-concurrency scenarios, problems can easily arise. For example, too many concurrent requests may lead to resource exhaustion. In order to solve these problems, Go provides the context package to manage the life cycle of requests and implement control of concurrent requests. This article will introduce how to use context to implement request concurrency control in Go.

Use context to create the context of the request:
First, we need to use context to create the context of the request in order to track and control it throughout the life cycle of the request. You can use the WithCancel, WithDeadline, WithTimeout and WithValue functions in the context package to create a context. For example, using the WithCancel function you can create a context that can be canceled:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

In this example, we use the WithCancel function to create a context that can be canceled, and call the cancel function to release the resources after the request ends.

Control the number of concurrent requests:
In order to control concurrent requests, we can use the goroutine and channel mechanisms of the Go language. We can start multiple goroutines to handle requests and use channels to limit the number of concurrent requests.

func main() {
    urls := []string{"http://url1.com", "http://url2.com", "http://url3.com", ...}

    maxConcurrency := 5
    sem := make(chan struct{}, maxConcurrency)

    for _, url := range urls {
        sem <- struct{}{}

        go func(url string) {
            defer func() {
                <-sem
            }()

            makeRequest(url)
        }(url)
    }

    // 等待所有请求完成
    for i := 0; i < maxConcurrency; i++ {
        sem <- struct{}{}
    }
}

func makeRequest(url string) {
    // 创建请求的上下文
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
    if err != nil {
        log.Printf("Error creating request: %s", err.Error())
        return
    }

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        log.Printf("Error making request: %s", err.Error())
        return
    }

    defer resp.Body.Close()

    // 处理响应
    // ...
}

In the above example, we use goroutine to handle each request. To limit the number of concurrent requests, we create a channel with a buffer and send a value to the channel at the beginning of each goroutine. At the end of the goroutine, we receive a value from the channel to free a slot. By controlling the number of slots in the channel, we can limit the number of concurrent requests.

Use context to implement request timeout control:
Sometimes, we may need to set a timeout for the request to prevent the request from being unable to receive a response for too long. In Go, you can use the context's WithTimeout function to set the request timeout. Here is an example:

func main() {
    url := "http://url.com"

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
    defer cancel()

    req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
    if err != nil {
        log.Printf("Error creating request: %s", err.Error())
        return
    }

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        log.Printf("Error making request: %s", err.Error())
        return
    }

    defer resp.Body.Close()

    // 处理响应
    // ...
}

In this example, we create a context with a 5 second timeout using the WithTimeout function. Before calling the Do method to send the request, we pass this context to the NewRequestWithContext method. If the request does not receive a response for more than 5 seconds, the context will be automatically canceled.

Summary:
In the Go language, you can use context to easily manage the life cycle of requests and control the number of concurrent requests and timeout. By flexibly using mechanisms such as concurrency, channel, and context, we can achieve more efficient and reliable request processing in high-concurrency scenarios.

Note:
Although in this article, we mainly introduce how to use context to implement request concurrency control, for more complex scenarios, such as request retry, cancellation of multiple concurrent requests, etc., we also You can use other functions of context. It is recommended to read the detailed introduction about context in the official Go language documentation to learn more usage and best practices.

The above is the detailed content of How to use context to implement request concurrency control in Go. For more information, please follow other related articles on the PHP Chinese website!