Generator Concurrency Pattern in Go: A Comprehensive Guide

⚠️ How to go about this series?

1. Run Every Example: Don't just read the code. Type it out, run it, and observe the behavior.
2. Experiment and Break Things: Remove sleeps and see what happens, change channel buffer sizes, modify goroutine counts.
Breaking things teaches you how they work
3. Reason About Behavior: Before running modified code, try predicting the outcome. When you see unexpected behavior, pause and think why. Challenge the explanations.
4. Build Mental Models: Each visualization represents a concept. Try drawing your own diagrams for modified code.

In our previous post, we explored the basics of goroutines and channels, the building blocks of Go's concurrency. Read here:

Understanding and visualizing Goroutines and Channels in Golang

Souvik Kar Mahapatra ・ Dec 20

#go #programming #learning #tutorial

Now, let's look at how these primitives combine to form powerful patterns that solve real-world problems.

In this post we'll cover Generator Pattern and will try to visualize them. So let's gear up as we'll be hands on through out the process.

Generator Pattern

A generator is like a fountain that continuously produces values that we can consume whenever needed.

In Go, it's a function that produces a stream of values and sends them through a channel, allowing other parts of our program to receive these values on demand.

Let's look at an example:

// generateNumbers creates a generator that produces numbers from 1 to max
func generateNumbers(max int) chan int {
    // Create a channel to send numbers
    out := make(chan int)

    // Launch a goroutine to generate numbers
    go func() {
        // Important: Always close the channel when done
        defer close(out)

        for i := 1; i 



<p>In this example, our generator function does three key things:</p>

<ol>
<li>Creates a channel to send values</li>
<li>Launches a goroutine to generate values</li>
<li>Returns the channel immediately for consumers to use</li>
</ol>

<h3>
  
  
  Why Use Generators?
</h3>

<ol>
<li>Separate value production from consumption</li>
<li>Generate values on-demand (lazy evaluation)</li>
<li>Can represent infinite sequences without consuming infinite memory</li>
<li>Allow concurrent production and consumption of values</li>
</ol>

<h3>
  
  
  Real-world Use Case
</h3>

<p>Reading large files line by line:<br>
</p>

<pre class="brush:php;toolbar:false">func generateLines(filename string) chan string {
    out := make(chan string)
    go func() {
        defer close(out)
        file, err := os.Open(filename)
        if err != nil {
            return
        }
        defer file.Close()

        scanner := bufio.NewScanner(file)
        for scanner.Scan() {
            out 



<p>Now you might be thinking, <u>what's so special about it? we can do the same like generating sequence of data or read line by line without goroutines</u>. Isn't it an overkill? Let's try to visualize both cases:</p>

<p><strong>Without the goroutines</strong><br>
</p>

<pre class="brush:php;toolbar:false">// Traditional approach
func getNumbers(max int) []int {
    numbers := make([]int, max)
    for i := 1; i 



<p>Here you have to wait for everything to be ready before you can start processing.</p>

<p><strong>With goroutines</strong><br>
</p>

<pre class="brush:php;toolbar:false">// Generator approach
func generateNumbers(max int) chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for i := 1; i 



<p>You can start processing the data while the data is still being generated.</p>

<p><img src="/static/imghwm/default1.png" data-src="https://img.php.cn/upload/article/000/000/000/173594261452189.jpg?x-oss-process=image/resize,p_40" class="lazy" alt="Generator Concurrency Pattern in Go: A Comprehensive Guide"></p>

<h3>
  
  
  Key Benefits of Generator Pattern:
</h3>

<ol>
<li><p><strong>Non-Blocking Execution</strong>: Generation and processing happen simultaneously</p></li>
<li><p><strong>Memory Efficiency</strong>: Can generate and process one value at a time, no need to store in the memory right away</p></li>
<li><p><strong>Infinite Sequences</strong>: Can generate infinite sequences without memory issues</p></li>
<li><p><strong>Backpressure Handling</strong>: If your consumer is slow, the generator naturally slows down (due to channel blocking), preventing memory overload.<br>
</p></li>
</ol>

<pre class="brush:php;toolbar:false">// generateNumbers creates a generator that produces numbers from 1 to max
func generateNumbers(max int) chan int {
    // Create a channel to send numbers
    out := make(chan int)

    // Launch a goroutine to generate numbers
    go func() {
        // Important: Always close the channel when done
        defer close(out)

        for i := 1; i 



<h3>
  
  
  Common Pitfalls and Solutions
</h3>

<ol>
<li>Forgetting to Close Channels
</li>
</ol>

<pre class="brush:php;toolbar:false">func generateLines(filename string) chan string {
    out := make(chan string)
    go func() {
        defer close(out)
        file, err := os.Open(filename)
        if err != nil {
            return
        }
        defer file.Close()

        scanner := bufio.NewScanner(file)
        for scanner.Scan() {
            out 



<ol>
<li>Not Handling Errors
</li>
</ol>

<pre class="brush:php;toolbar:false">// Traditional approach
func getNumbers(max int) []int {
    numbers := make([]int, max)
    for i := 1; i 



<ol>
<li>Resource Leaks: When using generators with resources (like files), ensure proper cleanup:
</li>
</ol>

<pre class="brush:php;toolbar:false">// Generator approach
func generateNumbers(max int) chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for i := 1; i 



<p>That's all for the generator pattern. Up next is <strong>Pipeline concurrency pattern</strong>. Stay tuned to clear your concepts on Golang concurrency.</p>

<p>Did I miss something? Got questions? Got something interesting to share? All comments are welcomed.</p>


          

            
  

            
        

The above is the detailed content of Generator Concurrency Pattern in Go: A Comprehensive Guide. For more information, please follow other related articles on the PHP Chinese website!