How to do compression in Golang and set up some compression tricks

The compression library (compress) that comes with the Go language provides a variety of compression and decompression functions, which can be used to convert information from one form to another, making it more suitable for storage or transmission. In this article, we will delve into how to compress in Golang and set up some compression techniques to help you make better use of the Golang compression library.

1. gzip compression

Gzip is a compression format based on the DEFLATE algorithm, which can compress data by replacing repeated strings in the input data. The Gzip compression library is implemented in the Go language standard library and implements the Gzip compression algorithm. To use it, you need to import the compression package and instantiate it using the function GzipWriter, then you can use the Write() function to write the data into the gzip buffer, and finally use the Flush() function to write the data in the buffer Flush to memory or disk.

The following example can be used to illustrate the gzip compression technique:

package main

import (
    "compress/gzip"
    "fmt"
    "strings"
)

func main() {
    var b strings.Builder
    w := gzip.NewWriter(&b)
    defer w.Close()

    data := []byte("Hello, World!")
    _, err := w.Write(data)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Compressed data: %q\n", b.String())
}

Output:

Compressed data: "\x1f\x8b\x08\x00\x00\x09\x6e\x88\xff\x4b\xcc\x4d\x55\x70\x76\x.00\x04\x00\x00\xff\xff\x48\x65\x6c\x6c\x6f\x2c\x20\x57\x6f\x72\x6c\x64\x21\x2.00\x00\x00\x00"

In the above example, we combine the output stream with the gzip.NewWriter() function The compressor is associated and uses a defer statement to lazily close the writer to ensure that all data in the buffer is written. We also write a string to the compression buffer and print the compressed data at the end.

2. zlib compression

zlib compression is the process of converting input data into smaller output data with the same data content. It is based on the DEFLATE algorithm and is commonly used for compressing web content and data transmission as it ensures optimal transmission efficiency. zlib provides the zlib.Writer type for compressing data into zlib format. You can use the following example to understand how to compress zlib in Go:

package main

import (
    "bytes"
    "compress/zlib"
    "fmt"
)

func main() {
    var b bytes.Buffer

    w := zlib.NewWriter(&b)
    defer w.Close()

    data := []byte("Hello, World!")
    _, err := w.Write(data)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Compressed data: %q\n", b.Bytes())
}

Output:

Compressed data: "\x78\x9c\x4b\xcb\xcf\x4f\x2c\x4b\x2d\x01\x00\x12\x1c\x06\xc8"

In the above example, we create a bytes.Buffer buffer and use the zlib.NewWriter function to It is associated with a compressor. The data will be compressed into a buffer and at the end the compressed data will be printed to the terminal.

3. flate compression

flate compression package is one of Golang’s own compression packages, supporting single byte, 1-bit and 2-bit reading, 3-bit and 4 Encoding method for bit reading. Of course, this compression method is only suitable for simple data and text, etc., because it cannot handle complex data structures. You can use the following example to see how to use Golang flate compression:

package main

import (
    "compress/flate"
    "fmt"
    "strings"
)

func main() {
    var b strings.Builder
    w, err := flate.NewWriter(&b, flate.DefaultCompression)
    if err != nil {
        panic(err)
    }
    defer w.Close()

    data := []byte("Hello, World!")
    _, err = w.Write(data)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Compressed data: %q\n", b.String())
}

Output:

Compressed data: "\x01\x9d\x8c\x0f\x4c\x4f\x4e\xce\xcf\x49\xcd\x4b\xcd\xaf.00\x00\x00\xff\xff\x48\x65\x6c\x6c\x6f\x2,20\x57\x6f\x72\x6c\x64\x21\x2.00\x00"

In the above example, we created a string writer and used flate.NewWriter( ) function associates it with the compressor. When compressing data, we need to specify the compression level. DefaultCompression is the most commonly used compression level we specify, indicating optimal compression. We print the compressed data through code.

4. snappy compression

Snappy is a fast data compression and decompression algorithm from Google. It is usually used to process data that does not need to be stored at a high compression ratio. The snappy package of Go language implements this compression algorithm and provides effective compression and decompression functions. You can use the following example to understand how to use snappy in Go:

package main

import (
    "fmt"
    "github.com/golang/snappy"
)

func main() {
    data := []byte("Hello, World!")
    compressed := snappy.Encode(nil, data)

    fmt.Printf("Compressed data: %q\n", compressed)

    uncompressed, err := snappy.Decode(nil, compressed)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Uncompressed data: %q\n", uncompressed)
}

Output:

Compressed data: "\x0cHello, World!"
Uncompressed data: "Hello, World!"

In the above example, we use snappy.Encode() function to convert "Hello, World!" The string is compressed and then decompressed using the snappy.Decode() function.

Summary

This article provides examples of using the compress compression library to implement four compression algorithms in Golang. gzip and zlib are the most commonly used compression algorithms and are widely used in data transfer and network applications. Snappy is usually used in scenarios with very high performance requirements for data compression, while flate is less commonly used. In either case, we can choose the most suitable compression algorithm and its configuration based on the actual situation to improve the scalability and performance of the application.

The above is the detailed content of How to do compression in Golang and set up some compression tricks. For more information, please follow other related articles on the PHP Chinese website!