


#php editor Baicao discovered a common goroutine leak when solving programming problems. When we try to use `ReadAll(response.Body)`, the returned `response` object is returned by the `client.Do(...)` method of the `http` package. However, this operation can lead to goroutine leakage problems. In order to avoid this happening, we need to take some measures to properly handle http responses.
Question content
The question comes from a very common scenario where all bytes are read from the response body after an http method call.
With the help of https://github.com/uber-go/goleak, I discovered an interesting goroutine leak.
To demonstrate the issue, run the following test code using go test
, which results in found an unexpected goroutine
:
package main import ( "io" "net/http" "testing" "go.uber.org/goleak" "gotest.tools/v3/assert" ) func testleak(t *testing.t) { defer goleak.verifynone(t) request, err := http.newrequest(http.methodget, "https://google.com", nil) assert.nilerror(t, err) client := http.client{} response, err := client.do(request) assert.nilerror(t, err) defer response.body.close() _, err = io.readall(response.body) assert.nilerror(t, err) }
Full test output:
$ go test --- FAIL: TestLeak (1.26s) main_test.go:28: found unexpected goroutines: [Goroutine 14 in state IO wait, with internal/poll.runtime_pollWait on top of the stack: goroutine 14 [IO wait]: internal/poll.runtime_pollWait(0x7f945c1e0398, 0x72) /home/wxh/.local/go/src/runtime/netpoll.go:305 +0x89 internal/poll.(*pollDesc).wait(0xc000154280?, 0xc0000ca000?, 0x0) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000154280, {0xc0000ca000, 0x2000, 0x2000}) /home/wxh/.local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc000154280, {0xc0000ca000?, 0xc00017edd8?, 0xc0004d5808?}) /home/wxh/.local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc00009c000, {0xc0000ca000?, 0xc0000c68f0?, 0x2c?}) /home/wxh/.local/go/src/net/net.go:183 +0x45 crypto/tls.(*atLeastReader).Read(0xc0002d4138, {0xc0000ca000?, 0x0?, 0x29f6540f238f2947?}) /home/wxh/.local/go/src/crypto/tls/conn.go:787 +0x3d bytes.(*Buffer).ReadFrom(0xc0000a0278, {0x7c3860, 0xc0002d4138}) /home/wxh/.local/go/src/bytes/buffer.go:202 +0x98 crypto/tls.(*Conn).readFromUntil(0xc0000a0000, {0x7c3b80?, 0xc00009c000}, 0xc0000a41c0?) /home/wxh/.local/go/src/crypto/tls/conn.go:809 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc0000a0000, 0x0) /home/wxh/.local/go/src/crypto/tls/conn.go:616 +0x116 crypto/tls.(*Conn).readRecord(...) /home/wxh/.local/go/src/crypto/tls/conn.go:582 crypto/tls.(*Conn).Read(0xc0000a0000, {0xc0001cf000, 0x1000, 0x11?}) /home/wxh/.local/go/src/crypto/tls/conn.go:1287 +0x16f bufio.(*Reader).Read(0xc0001af1a0, {0xc0001aad60, 0x9, 0xc0004d5d18?}) /home/wxh/.local/go/src/bufio/bufio.go:237 +0x1bb io.ReadAtLeast({0x7c3760, 0xc0001af1a0}, {0xc0001aad60, 0x9, 0x9}, 0x9) /home/wxh/.local/go/src/io/io.go:332 +0x9a io.ReadFull(...) /home/wxh/.local/go/src/io/io.go:351 net/http.http2readFrameHeader({0xc0001aad60?, 0x9?, 0xc0004d5df0?}, {0x7c3760?, 0xc0001af1a0?}) /home/wxh/.local/go/src/net/http/h2_bundle.go:1565 +0x6e net/http.(*http2Framer).ReadFrame(0xc0001aad20) /home/wxh/.local/go/src/net/http/h2_bundle.go:1829 +0x95 net/http.(*http2clientConnReadLoop).run(0xc0004d5f98) /home/wxh/.local/go/src/net/http/h2_bundle.go:8875 +0x130 net/http.(*http2ClientConn).readLoop(0xc00009e180) /home/wxh/.local/go/src/net/http/h2_bundle.go:8771 +0x6f created by net/http.(*http2Transport).newClientConn /home/wxh/.local/go/src/net/http/h2_bundle.go:7478 +0xaaa Goroutine 37 in state IO wait, with internal/poll.runtime_pollWait on top of the stack: goroutine 37 [IO wait]: internal/poll.runtime_pollWait(0x7f945c1e02a8, 0x72) /home/wxh/.local/go/src/runtime/netpoll.go:305 +0x89 internal/poll.(*pollDesc).wait(0xc0003ca000?, 0xc000202000?, 0x0) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc0003ca000, {0xc000202000, 0x1300, 0x1300}) /home/wxh/.local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0003ca000, {0xc000202000?, 0xc0004e6798?, 0xc0004c0808?}) /home/wxh/.local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000014028, {0xc000202000?, 0xc000136630?, 0x2c?}) /home/wxh/.local/go/src/net/net.go:183 +0x45 crypto/tls.(*atLeastReader).Read(0xc000276288, {0xc000202000?, 0x0?, 0x150fbb8ccb566149?}) /home/wxh/.local/go/src/crypto/tls/conn.go:787 +0x3d bytes.(*Buffer).ReadFrom(0xc0000a0978, {0x7c3860, 0xc000276288}) /home/wxh/.local/go/src/bytes/buffer.go:202 +0x98 crypto/tls.(*Conn).readFromUntil(0xc0000a0700, {0x7c3b80?, 0xc000014028}, 0xc0004cc2c0?) /home/wxh/.local/go/src/crypto/tls/conn.go:809 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc0000a0700, 0x0) /home/wxh/.local/go/src/crypto/tls/conn.go:616 +0x116 crypto/tls.(*Conn).readRecord(...) /home/wxh/.local/go/src/crypto/tls/conn.go:582 crypto/tls.(*Conn).Read(0xc0000a0700, {0xc00023c000, 0x1000, 0x11?}) /home/wxh/.local/go/src/crypto/tls/conn.go:1287 +0x16f bufio.(*Reader).Read(0xc00022a2a0, {0xc00020e2e0, 0x9, 0xc0004c0d18?}) /home/wxh/.local/go/src/bufio/bufio.go:237 +0x1bb io.ReadAtLeast({0x7c3760, 0xc00022a2a0}, {0xc00020e2e0, 0x9, 0x9}, 0x9) /home/wxh/.local/go/src/io/io.go:332 +0x9a io.ReadFull(...) /home/wxh/.local/go/src/io/io.go:351 net/http.http2readFrameHeader({0xc00020e2e0?, 0x9?, 0xc0004c0df0?}, {0x7c3760?, 0xc00022a2a0?}) /home/wxh/.local/go/src/net/http/h2_bundle.go:1565 +0x6e net/http.(*http2Framer).ReadFrame(0xc00020e2a0) /home/wxh/.local/go/src/net/http/h2_bundle.go:1829 +0x95 net/http.(*http2clientConnReadLoop).run(0xc0004c0f98) /home/wxh/.local/go/src/net/http/h2_bundle.go:8875 +0x130 net/http.(*http2ClientConn).readLoop(0xc00019e300) /home/wxh/.local/go/src/net/http/h2_bundle.go:8771 +0x6f created by net/http.(*http2Transport).newClientConn /home/wxh/.local/go/src/net/http/h2_bundle.go:7478 +0xaaa ] FAIL exit status 1
Does anyone know how to solve this problem?
Thanks!
Solution
In Go, http.Client uses Transport (http.RoundTripper interface). http.Transport is its default implementation. The default http.Transfer maintains a connection pool that holds connections that remain active or idle for some reason.
Basically all you need to do is make sure there are no connections in the pool* when you call goleak.Verify
This can be done by:
-
Disable keep-alive for all http clients before performing any requests:
client := http.Client{Transport: &http.Transport{DisableKeepAlive: true}}
-
Close idle connections before calling goleak.Verify*:
client.CloseIdleConnections()
// Available since Go 1.12
The above is the detailed content of Goroutine leaks when trying `ReadAll(response.Body)` where `response` is returned by http `client.Do(...)`. For more information, please follow other related articles on the PHP Chinese website!

This article demonstrates creating mocks and stubs in Go for unit testing. It emphasizes using interfaces, provides examples of mock implementations, and discusses best practices like keeping mocks focused and using assertion libraries. The articl

This article explores Go's custom type constraints for generics. It details how interfaces define minimum type requirements for generic functions, improving type safety and code reusability. The article also discusses limitations and best practices

The article discusses writing unit tests in Go, covering best practices, mocking techniques, and tools for efficient test management.

This article explores using tracing tools to analyze Go application execution flow. It discusses manual and automatic instrumentation techniques, comparing tools like Jaeger, Zipkin, and OpenTelemetry, and highlighting effective data visualization

The article explains how to use the pprof tool for analyzing Go performance, including enabling profiling, collecting data, and identifying common bottlenecks like CPU and memory issues.Character count: 159

The article discusses Go's reflect package, used for runtime manipulation of code, beneficial for serialization, generic programming, and more. It warns of performance costs like slower execution and higher memory use, advising judicious use and best

The article discusses using table-driven tests in Go, a method that uses a table of test cases to test functions with multiple inputs and outcomes. It highlights benefits like improved readability, reduced duplication, scalability, consistency, and a

The article explains how to use sync.WaitGroup in Go to manage concurrent operations, detailing initialization, usage, common pitfalls, and best practices.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Atom editor mac version download
The most popular open source editor

Dreamweaver CS6
Visual web development tools

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

Zend Studio 13.0.1
Powerful PHP integrated development environment
