Why do http post requests give me high memory usage in go?

#php editor Xinyi discovered the problem of high memory usage when using Go language to make http post requests. This question triggered his thinking, why does this happen? After research and analysis, he found some possible reasons and proposed some solutions. In this article, we’ll delve into this question and provide an answer.

Question content

I have a go application running inside a k8s container. It works as a rest api, receiving requests and writing them to elasticsearch.

My code is:

var r = gin.default()
r.post("/logs", func(c *gin.context) {
        fmt.println("receive log event")
        printmemusage()
        jsondata, err := ioutil.readall(c.request.body)
        d := strings.newreader(jsondata)
        http.post(fmt.sprintf("%s/_bulk", geteshost()), "application/json", d)
        ...
    })
}

In the above code, it listens to the path /logs and calls http to save the data into elasticsearch. When I print the memory usage using the function below, I can see that alloc keeps increasing until the memory is exhausted. If I remove the http.post call, the memory usage is always 1 to 3mb. What could be the reason for the increasing memory usage?

func bToMb(b uint64) uint64 {
    return b / 1024 / 1024
}
func PrintMemUsage() {
    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    // For info on each, see: https://golang.org/pkg/runtime/#MemStats
    fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc))
    fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc))
    fmt.Printf("\tSys = %v MiB", bToMb(m.Sys))
    fmt.Printf("\tNumGC = %v\n", m.NumGC)
}

Solution

http document mentioned many times:

The client must close the response body when finished:

Here is an example from the documentation:

resp, err := http.Get("http://example.com/")
if err != nil {
    // handle error
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
// ...

If you don't do this, a leak will occur because the body will remain in memory forever.

The above is the detailed content of Why do http post requests give me high memory usage in go?. For more information, please follow other related articles on the PHP Chinese website!