Write chan in sync.WaitGroup goroutine

php editor Youzi will introduce a method of writing chan in sync.WaitGroup goroutine. In concurrent programming, sync.WaitGroup is a very useful synchronization mechanism that can wait for the execution of a group of goroutines to complete. However, sometimes we need to write the results to a chan after the goroutine completes execution for consumption by other goroutines. This article will introduce in detail how to implement this function in sync.WaitGroup goroutine, let's take a look!

Question content

I am getting a list of items from an API endpoint. Then, for each project, I make another API request to get data about the individual project.

I can't make a second API request to each project at the same time because my API token is rate limited and if I make too many requests at the same time I get throttled.

However, the initial API response data can be split into multiple pages, which allows me to process pages of data simultaneously.

After some research, the following code does exactly what I want:

func main() {
    // pretend paginated results from initial API request
    page1 := []int{1, 2, 3}
    page2 := []int{4, 5, 6}
    page3 := []int{7, 8, 9}
    pages := [][]int{page1, page2, page3}

    results := make(chan string)

    var wg sync.WaitGroup
    for i := range pages {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            for j := range pages[i] {
                // simulate making additional API request and building the report
                time.Sleep(500 * time.Millisecond)

                result := fmt.Sprintf("Finished creating report for %d", pages[i][j])
                results <- result
            }

        }(i)
    }

    go func() {
        wg.Wait()
        close(results)
    }()

    for result := range results {
        fmt.Println(result)
    }
}

I want to understand why it works:

go func() {
    wg.Wait()
    close(results)
}()

My first attempt was unsuccessful - I thought I could iterate over the channel after wg.Wait() and I would read the results as they are written to the results channel Get the result.

func main() {
    // pretend paginated results from initial API request
    page1 := []int{1, 2, 3}
    page2 := []int{4, 5, 6}
    page3 := []int{7, 8, 9}
    pages := [][]int{page1, page2, page3}

    results := make(chan string)

    var wg sync.WaitGroup
    for i := range pages {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            for j := range pages[i] {
                // simulate making additional API request and building the report
                time.Sleep(500 * time.Millisecond)

                result := fmt.Sprintf("Finished creating report for %d", pages[i][j])
                results <- result
            }

        }(i)
    }

    // does not work
    wg.Wait()
    close(results)

    for result := range results {
        fmt.Println(result)
    }
}

Workaround

On your first try:

1. The main goroutine causes 3 goroutines to put values ​​into the result channel.
2. The main coroutine waits for all coroutines to complete.
3. One of the goroutines puts a value into the result channel and fills the channel (channel size is 1 string).
4. Now all three goroutines can no longer put values ​​into the result channel and go to sleep until the result channel is released.
5. All goroutines are in sleep state. You're at an impasse.

On the second try:

1. The main goroutine contains 4 goroutines.
2. 3 goroutines put values ​​into the result channel.
3. The other goroutine (I'll call it the 4th one) waits for these 3 goroutines to finish.
4. At the same time, the main coroutine waits for the value in the result channel (for loop)
5. In this case, if one of the goroutines puts a value in the result channel, it will block the remaining three goroutines; the main Goroutine will take the value out of the result channel, thereby unblocking the other Goroutines.
6. So, all 3 goroutines put their respective values ​​and end
7. Then the fourth goroutine closes the channel
8. The main Goroutine ends its for loop.

The above is the detailed content of Write chan in sync.WaitGroup goroutine. For more information, please follow other related articles on the PHP Chinese website!