How to deal with Chinese garbled code in golang zip

With the popularity of Golang language, more and more people are starting to use it to develop new applications. One of the common applications is to pack files from the file system and compress them into a ZIP file. Especially when dealing with Chinese file names, it is easy to encounter the problem of garbled characters. This article will discuss how to solve the problem of Chinese garbled characters in Golang zip.

1. Problem description

When using Golang's zip package, if the file name processed contains Chinese characters, garbled characters will appear when output to the ZIP file. The figure below shows a directory structure containing Chinese file names:

example
├── file1.txt
└── 文件2.txt

We use the following code to package it into a ZIP file:

package main

import (
    "archive/zip"
    "os"
    "path/filepath"
)

func main() {
    zipFileName := "example.zip"
    files := []string{"example/file1.txt", "example/文件2.txt"}

    // Create a new ZIP file.
    zipFile, err := os.Create(zipFileName)
    if err != nil {
        panic(err)
    }
    defer zipFile.Close()

    // Create a new writer to write to the ZIP file.
    zipWriter := zip.NewWriter(zipFile)
    defer zipWriter.Close()

    // Iterate over the files and add them to the ZIP file.
    for _, file := range files {
        addFileToZip(file, zipWriter)
    }
}

func addFileToZip(file string, zipWriter *zip.Writer) error {
    // Open the file to be added to the ZIP file.
    fileToZip, err := os.Open(file)
    if err != nil {
        return err
    }
    defer fileToZip.Close()

    // Get the file information for the file being added.
    fileInfo, err := fileToZip.Stat()
    if err != nil {
        return err
    }

    // Create a new file header for the file being added.
    header, err := zip.FileInfoHeader(fileInfo)
    if err != nil {
        return err
    }

    // Set the name for the file being added (this is what appears in the ZIP archive).
    header.Name = filepath.Base(file)

    // Add the file header to the ZIP archive.
    writer, err := zipWriter.CreateHeader(header)
    if err != nil {
        return err
    }

    // Copy the contents of the file into the ZIP archive.
    _, err = io.Copy(writer, fileToZip)
    if err != nil {
        return err
    }

    return nil
}

Executing this program will generate the example.zip file, Open the compressed file and we can see that the file name is garbled. As shown in the figure below:

This is because when the program executes zipWriter.CreateHeader (header), UTF-8 encoding will be used by default to process the file name, but the file name uses is the system default encoding (GBK in my case). Therefore, it gets garbled when writing the ZIP file.

2. Solution

In order to solve the above problem, we need to ensure that the file name is converted to UTF-8 encoding before writing the ZIP file. However, the file name may be generated using the system default encoding, so we must ensure that the encoding format of the file name is correctly identified and converted to UTF-8 encoding.

The following is a simple example showing how to implement the above steps:

package main

import (
    "archive/zip"
    "bytes"
    "io"
    "os"
    "path/filepath"
    "golang.org/x/text/encoding/simplifiedchinese"
    "golang.org/x/text/transform"
)

func main() {
    zipFileName := "example.zip"
    files := []string{"example/file1.txt", "example/文件2.txt"}

    // Create a new ZIP file.
    zipFile, err := os.Create(zipFileName)
    if err != nil {
        panic(err)
    }
    defer zipFile.Close()

    // Create a new writer to write to the ZIP file.
    zipWriter := zip.NewWriter(zipFile)
    defer zipWriter.Close()

    // Iterate over the files and add them to the ZIP file.
    for _, file := range files {
        addFileToZip(file, zipWriter)
    }
}

func addFileToZip(file string, zipWriter *zip.Writer) error {
    // Open the file to be added to the ZIP file.
    fileToZip, err := os.Open(file)
    if err != nil {
        return err
    }
    defer fileToZip.Close()

    // Get the file information for the file being added.
    fileInfo, err := fileToZip.Stat()
    if err != nil {
        return err
    }

    // Create a new file header for the file being added.
    header, err := zip.FileInfoHeader(fileInfo)
    if err != nil {
        return err
    }

    // Convert the file name to UTF-8.
    header.Name, err = toUTF8(fileInfo.Name())
    if err != nil {
        return err
    }

    // Add the file header to the ZIP archive.
    writer, err := zipWriter.CreateHeader(header)
    if err != nil {
        return err
    }

    // Copy the contents of the file into the ZIP archive.
    _, err = io.Copy(writer, fileToZip)
    if err != nil {
        return err
    }

    return nil
}

func toUTF8(src string) (string, error) {
    var (
        buf bytes.Buffer
        w   = transform.NewWriter(&buf, simplifiedchinese.GBK.NewDecoder())
    )
    _, err := w.Write([]byte(src))
    if err != nil {
        return "", err
    }
    err = w.Close()
    if err != nil {
        return "", err
    }
    return buf.String(), nil
}

In the above code, we use the golang.org/x/text/transform package to convert the file name from GBK format In UTF-8 format. We first import the package and convert the filename from GBK to UTF-8 encoding through the toUTF8() function. Then in the addFileToZip() function we update Header.Name with the converted file name and add it to the ZIP file.

The ZIP file generated by executing this program will have the file name displayed in Chinese normally.

Summary

When using the Golang zip package, if there is a Chinese file name, you will encounter garbled characters when outputting to the ZIP file. To solve this problem, we need to convert the file name to UTF-8 encoding first to avoid garbled characters. In this article, we used the golang.org/x/text/transform package to convert file names from GBK format to UTF-8 format. In this way, we can ensure that the file name will not be garbled when adding it to the ZIP file.

The above is the detailed content of How to deal with Chinese garbled code in golang zip. For more information, please follow other related articles on the PHP Chinese website!