How to Maintain a Fixed Number of Concurrent Goroutines in Go?

Maintaining a Fixed Number of Concurrent Goroutines

In Go, you may encounter scenarios where controlling the number of concurrently running goroutines is crucial. While tutorials often focus on waiting for goroutines to complete, achieving a specific number of active goroutines at any given time presents a different challenge.

Consider the following situation: you have hundreds of thousands of tasks to process. Processing each task requires its own goroutine, but your system resources can only handle a maximum of 20 concurrent goroutines. You need to ensure that there are always 20 goroutines running, launching a new one whenever an existing one completes.

Bounded Parallelism

To achieve this, the Go Concurrency Patterns article suggests using a pattern called "Bounded Parallelism." It involves using a channel of empty structs as a guard to limit the number of concurrent workers.

Implementation

Here's an example that demonstrates how to implement this pattern:

package main

import (
    "fmt"
    "sync"
)

func main() {
    const maxGoroutines = 20

    // Create a channel of empty structs to control worker count
    guard := make(chan struct{}, maxGoroutines)

    var wg sync.WaitGroup

    // Launch workers
    for i := 0; i < 30; i++ {
        wg.Add(1)
        guard <- struct{}{} // Blocks if guard channel is filled
        go func(n int) {
            defer wg.Done()
            worker(n)
            <-guard // Release slot in guard channel
        }(i)
    }
    wg.Wait()
}

func worker(i int) { fmt.Println("doing work on", i) }

In this example, the guard channel is used as a token bucket. The maximum number of goroutines that can run concurrently is limited by the capacity of the channel (20 in this case). Each goroutine acquires a "token" (an empty struct) from the channel before starting work. When a goroutine finishes, it releases its token back into the channel, making it available for another goroutine to acquire. By controlling the number of tokens in the channel, you effectively control the number of concurrent goroutines.

The above is the detailed content of How to Maintain a Fixed Number of Concurrent Goroutines in Go?. For more information, please follow other related articles on the PHP Chinese website!