All goroutines are sleeping - deadlock, on buffered channels, don't understand why

Question content

I only want to create a certain number of go routines, say 5, but I can receive a variable number of jobs.

Here is the code I try to do this, the test is below it.

package main

import (
    "context"
    "fmt"
    "runtime"
    "time"
)

func dowork(size int, capacity int) int {
    start := time.now()
    jobs := make(chan *job, capacity)
    results := make(chan *job, capacity)
    sem := make(chan struct{}, capacity)
    go chanworker(jobs, results, sem)
    for i := 0; i < size; i++ {
        jobs <- &job{id: i}
    }
    close(jobs)
    successcount := 0
    for i := 0; i < size; i++ {
        item := <-results
        if item.result {
            successcount++
        }
        fmt.printf("job %d completed %v\n", item.id, item.result)
    }
    close(results)
    close(sem)
    fmt.printf("time taken to execute %d jobs with %d capacity = %v\n", size, capacity, time.since(start))
    return successcount
}

func chanworker(jobs <-chan *job, results chan<- *job, sem chan struct{}) {

    for item := range jobs {
        it := item
        sem <- struct{}{}
        fmt.printf("job %d started\n", it.id)
        go func() {
            timeoutctx, cancel := context.withtimeout(context.background(), 300*time.millisecond)
            defer cancel()
            time.sleep(time.duration(it.id) * 100 * time.millisecond)
            select {
            case <-timeoutctx.done():
                fmt.printf("job %d timed out\n", it.id)
                it.result = false
                results <- it
                <-sem
                return
            default:
                fmt.printf("total number of routines %d\n", runtime.numgoroutine())
                it.result = true
                results <- it
                <-sem
            }
        }()
    }
}

Testing this

package main

import (
    "testing"
)

func Test_doWork(t *testing.T) {
    type args struct {
        size     int
        capacity int
    }
    tests := []struct {
        name string
        args args
        want int
    }{
        {
            name: "jobs 10 capacity 5",
            args: args{
                size:     10,
                capacity: 5,
            },
            want: 3,
        },
        {
            name: "jobs 100 capacity 5",
            args: args{
                size:     100,
                capacity: 5,
            },
            want: 3,
        },
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            if got := doWork(tt.args.size, tt.args.capacity); got < tt.want {
                t.Errorf("doWork() = %v, want %v", got, tt.want)
            }
        })
    }
}

Test jobs 10 capacity 5 works, but jobs 100 capacity 5 fails.

If I set capacity 50 for 100 jobs it works but not for 30 jobs and can't understand the behavior.

The following is my understanding of the channel and my expectation that it will work.

The buffer channel, if full, will block until some free capacity is available. I would expect that once the jobs channel is full it will block until the chanworker frees some of them. The chanworker itself receives a capacity and uses an empty structure to ensure that no more than 5 worker threads are created.

Why do I get errors? Fatal error: All goroutines are sleeping - deadlock! ?

Correct answer

Since the main goroutine will not receive values ​​from results until all jobs have been sent to jobs, So the worker thread blocks while sending to results. The main goroutine blocks sending to jobs because the job is blocked. Deadlock!

Fixed by using goroutine to do the work.

go func() {
    for i := 0; i < size; i++ {
        jobs <- &Job{id: i}
    }
    close(jobs)
}()

https://www.php.cn/link/6e04df31f1bbb1c02666d0dfa3638f76

The above is the detailed content of All goroutines are sleeping - deadlock, on buffered channels, don't understand why. For more information, please follow other related articles on the PHP Chinese website!