How Can Goroutines and WaitGroups Be Used to Download Multiple Files Concurrently in Golang?

Downloading Multiple Files in Parallel with Goroutines in Golang

In this article, we'll explore how to utilize goroutines in Golang to download multiple files concurrently from a remote location.

Consider the following code snippet, which attempts to download files from Dropbox:

<code class="go">package main

import (
    "encoding/json"
    "fmt"
    "io"
    "io/ioutil"
    "net/http"
    "net/url"
    "os"
    "path/filepath"
    "sync"
)

...
func main() {

    ...
    for i, file := range flr.FileList {

        go download_file(file, tr)

        if i >= 2 {
            break
        }
    }
   ...
}</code>

However, upon attempting to execute this code using goroutines, it fails. This is because the main goroutine exits before the goroutines responsible for downloading files have completed their tasks.

To address this issue and ensure that the main goroutine waits for the goroutines to complete before exiting, we can employ a sync.WaitGroup, as follows:

<code class="go">package main

import (
    ...
    "sync"
)

...
func main() {

    ...
    var wg sync.WaitGroup
    for i, file := range flr.FileList {
        wg.Add(1)

        go download_file(file, tr, &wg)

        if i >= 2 {
            break
        }
    }
    wg.Wait()
    ...
}</code>

Here, we create a sync.WaitGroup and use it to keep track of the number of goroutines that have not yet completed their tasks. This allows the main goroutine to wait until all the goroutines have completed their tasks before exiting.

In the modified download_file function, we call wg.Done() to signal the completion of the goroutine, decrementing the wg counter.

By utilizing goroutines and a Waitgroup, we can effectively download multiple files concurrently, improving the overall efficiency of the download process.

The above is the detailed content of How Can Goroutines and WaitGroups Be Used to Download Multiple Files Concurrently in Golang?. For more information, please follow other related articles on the PHP Chinese website!