


#php editor Baicao discovered a common goroutine leak when solving programming problems. When we try to use `ReadAll(response.Body)`, the returned `response` object is returned by the `client.Do(...)` method of the `http` package. However, this operation can lead to goroutine leakage problems. In order to avoid this happening, we need to take some measures to properly handle http responses.
Question content
The question comes from a very common scenario where all bytes are read from the response body after an http method call.
With the help of https://github.com/uber-go/goleak, I discovered an interesting goroutine leak.
To demonstrate the issue, run the following test code using go test
, which results in found an unexpected goroutine
:
package main import ( "io" "net/http" "testing" "go.uber.org/goleak" "gotest.tools/v3/assert" ) func testleak(t *testing.t) { defer goleak.verifynone(t) request, err := http.newrequest(http.methodget, "https://google.com", nil) assert.nilerror(t, err) client := http.client{} response, err := client.do(request) assert.nilerror(t, err) defer response.body.close() _, err = io.readall(response.body) assert.nilerror(t, err) }
Full test output:
$ go test --- FAIL: TestLeak (1.26s) main_test.go:28: found unexpected goroutines: [Goroutine 14 in state IO wait, with internal/poll.runtime_pollWait on top of the stack: goroutine 14 [IO wait]: internal/poll.runtime_pollWait(0x7f945c1e0398, 0x72) /home/wxh/.local/go/src/runtime/netpoll.go:305 +0x89 internal/poll.(*pollDesc).wait(0xc000154280?, 0xc0000ca000?, 0x0) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000154280, {0xc0000ca000, 0x2000, 0x2000}) /home/wxh/.local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc000154280, {0xc0000ca000?, 0xc00017edd8?, 0xc0004d5808?}) /home/wxh/.local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc00009c000, {0xc0000ca000?, 0xc0000c68f0?, 0x2c?}) /home/wxh/.local/go/src/net/net.go:183 +0x45 crypto/tls.(*atLeastReader).Read(0xc0002d4138, {0xc0000ca000?, 0x0?, 0x29f6540f238f2947?}) /home/wxh/.local/go/src/crypto/tls/conn.go:787 +0x3d bytes.(*Buffer).ReadFrom(0xc0000a0278, {0x7c3860, 0xc0002d4138}) /home/wxh/.local/go/src/bytes/buffer.go:202 +0x98 crypto/tls.(*Conn).readFromUntil(0xc0000a0000, {0x7c3b80?, 0xc00009c000}, 0xc0000a41c0?) /home/wxh/.local/go/src/crypto/tls/conn.go:809 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc0000a0000, 0x0) /home/wxh/.local/go/src/crypto/tls/conn.go:616 +0x116 crypto/tls.(*Conn).readRecord(...) /home/wxh/.local/go/src/crypto/tls/conn.go:582 crypto/tls.(*Conn).Read(0xc0000a0000, {0xc0001cf000, 0x1000, 0x11?}) /home/wxh/.local/go/src/crypto/tls/conn.go:1287 +0x16f bufio.(*Reader).Read(0xc0001af1a0, {0xc0001aad60, 0x9, 0xc0004d5d18?}) /home/wxh/.local/go/src/bufio/bufio.go:237 +0x1bb io.ReadAtLeast({0x7c3760, 0xc0001af1a0}, {0xc0001aad60, 0x9, 0x9}, 0x9) /home/wxh/.local/go/src/io/io.go:332 +0x9a io.ReadFull(...) /home/wxh/.local/go/src/io/io.go:351 net/http.http2readFrameHeader({0xc0001aad60?, 0x9?, 0xc0004d5df0?}, {0x7c3760?, 0xc0001af1a0?}) /home/wxh/.local/go/src/net/http/h2_bundle.go:1565 +0x6e net/http.(*http2Framer).ReadFrame(0xc0001aad20) /home/wxh/.local/go/src/net/http/h2_bundle.go:1829 +0x95 net/http.(*http2clientConnReadLoop).run(0xc0004d5f98) /home/wxh/.local/go/src/net/http/h2_bundle.go:8875 +0x130 net/http.(*http2ClientConn).readLoop(0xc00009e180) /home/wxh/.local/go/src/net/http/h2_bundle.go:8771 +0x6f created by net/http.(*http2Transport).newClientConn /home/wxh/.local/go/src/net/http/h2_bundle.go:7478 +0xaaa Goroutine 37 in state IO wait, with internal/poll.runtime_pollWait on top of the stack: goroutine 37 [IO wait]: internal/poll.runtime_pollWait(0x7f945c1e02a8, 0x72) /home/wxh/.local/go/src/runtime/netpoll.go:305 +0x89 internal/poll.(*pollDesc).wait(0xc0003ca000?, 0xc000202000?, 0x0) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /home/wxh/.local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc0003ca000, {0xc000202000, 0x1300, 0x1300}) /home/wxh/.local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0003ca000, {0xc000202000?, 0xc0004e6798?, 0xc0004c0808?}) /home/wxh/.local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000014028, {0xc000202000?, 0xc000136630?, 0x2c?}) /home/wxh/.local/go/src/net/net.go:183 +0x45 crypto/tls.(*atLeastReader).Read(0xc000276288, {0xc000202000?, 0x0?, 0x150fbb8ccb566149?}) /home/wxh/.local/go/src/crypto/tls/conn.go:787 +0x3d bytes.(*Buffer).ReadFrom(0xc0000a0978, {0x7c3860, 0xc000276288}) /home/wxh/.local/go/src/bytes/buffer.go:202 +0x98 crypto/tls.(*Conn).readFromUntil(0xc0000a0700, {0x7c3b80?, 0xc000014028}, 0xc0004cc2c0?) /home/wxh/.local/go/src/crypto/tls/conn.go:809 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc0000a0700, 0x0) /home/wxh/.local/go/src/crypto/tls/conn.go:616 +0x116 crypto/tls.(*Conn).readRecord(...) /home/wxh/.local/go/src/crypto/tls/conn.go:582 crypto/tls.(*Conn).Read(0xc0000a0700, {0xc00023c000, 0x1000, 0x11?}) /home/wxh/.local/go/src/crypto/tls/conn.go:1287 +0x16f bufio.(*Reader).Read(0xc00022a2a0, {0xc00020e2e0, 0x9, 0xc0004c0d18?}) /home/wxh/.local/go/src/bufio/bufio.go:237 +0x1bb io.ReadAtLeast({0x7c3760, 0xc00022a2a0}, {0xc00020e2e0, 0x9, 0x9}, 0x9) /home/wxh/.local/go/src/io/io.go:332 +0x9a io.ReadFull(...) /home/wxh/.local/go/src/io/io.go:351 net/http.http2readFrameHeader({0xc00020e2e0?, 0x9?, 0xc0004c0df0?}, {0x7c3760?, 0xc00022a2a0?}) /home/wxh/.local/go/src/net/http/h2_bundle.go:1565 +0x6e net/http.(*http2Framer).ReadFrame(0xc00020e2a0) /home/wxh/.local/go/src/net/http/h2_bundle.go:1829 +0x95 net/http.(*http2clientConnReadLoop).run(0xc0004c0f98) /home/wxh/.local/go/src/net/http/h2_bundle.go:8875 +0x130 net/http.(*http2ClientConn).readLoop(0xc00019e300) /home/wxh/.local/go/src/net/http/h2_bundle.go:8771 +0x6f created by net/http.(*http2Transport).newClientConn /home/wxh/.local/go/src/net/http/h2_bundle.go:7478 +0xaaa ] FAIL exit status 1
Does anyone know how to solve this problem?
Thanks!
Solution
In Go, http.Client uses Transport (http.RoundTripper interface). http.Transport is its default implementation. The default http.Transfer maintains a connection pool that holds connections that remain active or idle for some reason.
Basically all you need to do is make sure there are no connections in the pool* when you call goleak.Verify
This can be done by:
-
Disable keep-alive for all http clients before performing any requests:
client := http.Client{Transport: &http.Transport{DisableKeepAlive: true}}
-
Close idle connections before calling goleak.Verify*:
client.CloseIdleConnections()
// Available since Go 1.12
The above is the detailed content of Goroutine leaks when trying `ReadAll(response.Body)` where `response` is returned by http `client.Do(...)`. For more information, please follow other related articles on the PHP Chinese website!

InterfacesandpolymorphisminGoenhancecodereusabilityandmaintainability.1)Defineinterfacesattherightabstractionlevel.2)Useinterfacesfordependencyinjection.3)Profilecodetomanageperformanceimpacts.

TheinitfunctioninGorunsautomaticallybeforethemainfunctiontoinitializepackagesandsetuptheenvironment.It'susefulforsettingupglobalvariables,resources,andperformingone-timesetuptasksacrossanypackage.Here'showitworks:1)Itcanbeusedinanypackage,notjusttheo

Interface combinations build complex abstractions in Go programming by breaking down functions into small, focused interfaces. 1) Define Reader, Writer and Closer interfaces. 2) Create complex types such as File and NetworkStream by combining these interfaces. 3) Use ProcessData function to show how to handle these combined interfaces. This approach enhances code flexibility, testability, and reusability, but care should be taken to avoid excessive fragmentation and combinatorial complexity.

InitfunctionsinGoareautomaticallycalledbeforethemainfunctionandareusefulforsetupbutcomewithchallenges.1)Executionorder:Multipleinitfunctionsrunindefinitionorder,whichcancauseissuesiftheydependoneachother.2)Testing:Initfunctionsmayinterferewithtests,b

Article discusses iterating through maps in Go, focusing on safe practices, modifying entries, and performance considerations for large maps.Main issue: Ensuring safe and efficient map iteration in Go, especially in concurrent environments and with l

The article discusses creating and manipulating maps in Go, including initialization methods and adding/updating elements.

The article discusses differences between arrays and slices in Go, focusing on size, memory allocation, function passing, and usage scenarios. Arrays are fixed-size, stack-allocated, while slices are dynamic, often heap-allocated, and more flexible.

The article discusses creating and initializing slices in Go, including using literals, the make function, and slicing existing arrays or slices. It also covers slice syntax and determining slice length and capacity.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Atom editor mac version download
The most popular open source editor

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

Dreamweaver Mac version
Visual web development tools

SublimeText3 Linux new version
SublimeText3 Linux latest version

Dreamweaver CS6
Visual web development tools
