chan chan construct causes deadlock

Question content

I am trying to understand the chan chan construct in Go by writing a small piece of code, as shown below, I You want 3 worker subroutines to handle 10 jobs. Each worker subroutine has its own channel where it receives "jobs" to be processed. The main Go routine distributes jobs to worker channels by getting channels from the channel pool (hence the chan chan construct).

But this code will cause a deadlock situation! I tried some variations of this code but got the same error.

Is it because the worker subroutine waits forever to read the job from its channel? Or is it due to other reasons (maybe the channel is closed prematurely, etc.)? I'm obviously missing something in my understanding of the whole structure.

Can someone help me understand this problem and how to solve it?

Code from the playground and copied below as requested.

package main

import (
    "fmt"
    "sync"
)

type Job struct {
    ID int
}

func worker(id int, jobs <-chan Job, wg *sync.WaitGroup) {
    defer wg.Done()
    fmt.Printf("Worker %d starting\n", id)
    for job := range jobs {
        fmt.Printf("Worker %d processing job %d\n", id, job.ID)
    }
    fmt.Printf("Worker %d done\n", id)
}

func main() {
    numWorkers := 3
    maxJobs := 10
    var wg sync.WaitGroup
    // Create the pool of worker channels
    pool := make(chan chan Job, numWorkers)

    for i := 0; i < numWorkers; i++ {
        workerChan := make(chan Job) // Create a new channel for each worker
        pool <- workerChan           // Add the worker channel to the pool
        go worker(i, workerChan, &wg)
    }
    defer close(pool)
    // Create jobs and distribute them to workers
    for i := 0; i < maxJobs; i++ {
        job := Job{ID: i}
        wg.Add(1)
        workerChan := <-pool
        workerChan <- job
    }

    // Wait for all workers to complete
    wg.Wait()
    fmt.Println("All jobs are processed")
}

Correct answer

First of all: there is no need for a channel of channels here. To distribute work to multiple workers, you simply have all workers read from a single shared channel. When you send a piece of work to a channel, if there are workers available, one of them will receive it, otherwise the channel send operation will block until there is a worker available.

If you want to use separate channels for each worker, you still don't need a channel of channels, you just need a subset of them. Each worker will listen from a dedicated channel and you will manage your own work assignments:

numWorkers := 3
maxJobs := 10
var wg sync.WaitGroup
pool := make([]chan Job, numWorkers)
for i := 0; i < numWorkers; i++ {
    pool[i] = make(chan Job)
    wg.Add(1)
    go worker(i, pool[i], &wg)
}
for i := 0; i < maxJobs; i++ {
   job := Job{ID: i}
   pool[i%len(pool)] <- job
}
for _,c:=range pool {
   close(c)
}
wg.Wait()

There are two problems with your code:

1. You should add to the wait group when creating the Goroutine, not when sending the job to the channel
2. When you read a channel from the channel, the channel is removed from the pool. So after you read 3 channels, workerChannel := <-pool will block because there are no other channels. If you insist on using the channel's channel, you have to put it back:

workerChan := <-pool
 workerChan <- job
 pool<-workerChan

This way you can use the channel's channel as a circular queue

1. Close pool will have no effect. You must close the channel in pool.

The above is the detailed content of chan chan construct causes deadlock. For more information, please follow other related articles on the PHP Chinese website!