How to Read Variable-Length Data from a Network Connection in Go?

Reading Variable-Length Data in Go with net.Conn.Read

When working with network applications in Go, you often encounter situations where the data received over a connection is of variable length. The standard library's net.Conn provides a method called Read, which fills a byte array with the received data. However, this approach can be problematic if you don't know the exact length of the content beforehand and may result in reading either too much or insufficient data.

To address this challenge, one approach is to utilize the bufio package. However, a more efficient solution is to employ a growing buffer and read data until the end-of-file (EOF) is encountered:

package main

import (
    "fmt"
    "io"
    "net"
)

func main() {
    // Establish a connection with a remote host.
    conn, err := net.Dial("tcp", "google.com:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()

    // Write a request to the connection.
    fmt.Fprintf(conn, "GET / HTTP/1.0\r\n\r\n")

    // Create a buffer to store the received data.
    buf := make([]byte, 0, 4096)

    // Read from the connection in a loop until EOF.
    for {
        // Read into a temporary buffer to avoid overwriting existing data in `buf`.
        tmp := make([]byte, 256)
        n, err := conn.Read(tmp)
        if err != nil {
            if err != io.EOF {
                fmt.Println("read error:", err)
            }
            break
        }

        // Append the new data to the buffer.
        buf = append(buf, tmp[:n]...)
    }

    fmt.Println("total size:", len(buf))
}

This solution allocates a sizable initial buffer and gradually expands it as more data is received. It continues reading until the EOF is reached, ensuring that the buffer contains the complete response. Additionally, it keeps track of the total size of the received data.

Alternatively, you can utilize the bytes.Buffer type in conjunction with io.Copy to achieve a similar result:

package main

import (
    "bytes"
    "fmt"
    "io"
    "net"
)

func main() {
    // Establish a connection with a remote host.
    conn, err := net.Dial("tcp", "google.com:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()

    // Write a request to the connection.
    fmt.Fprintf(conn, "GET / HTTP/1.0\r\n\r\n")

    // Create a buffer to store the received data.
    var buf bytes.Buffer

    // Copy the data from the connection into the buffer.
    io.Copy(&buf, conn)

    fmt.Println("total size:", buf.Len())
}

This approach uses a buffer that automatically expands as needed and provides a cleaner, more concise solution.

The above is the detailed content of How to Read Variable-Length Data from a Network Connection in Go?. For more information, please follow other related articles on the PHP Chinese website!