How to pack and compress files in Go

File packaging, compression and decompression are frequently used functions. We can complete these operations through tools such as tar and gzip. In Go, the standard library <span style="font-size: 15px;">archive</span> and <span style="font-size: 15px;">compress</span> provide us with these Ability, through the examples in this article, you will find that it is very simple to generate and process compressed package files using Go programming.

Packaging and Compression

Before starting the code, we need to clarify the concepts of packaging and compression.

• Packaging, also known as archive, refers to a collection of files or directories, and this collection is stored in a file.
• Compression refers to the use of algorithms to process files in order to retain the maximum file information and reduce the file size.

# Take the packaging tool tar as an example. The files produced by it are usually called tar packages, and their file names usually end with .tar. Then compress the tar package through other compression tools, such as gzip compression, and you will get a compressed file usually named with .tar.gz (gzip can be called with the -z parameter in tar).

tar package is a collection of files, and its structure is also composed of data segments. Each data segment contains the file header (metainformation describing the file) and file content.

+----------------------------------------+
| Header                                 |
| [name][mode][owner][group][size]  ...  |
+----------------------------------------+
| Content                                |
| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|
| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|
+----------------------------------------+
| Header                                 |
| [name][mode][owner][group][size]  ...  |
+----------------------------------------+
| Content                                |
| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|
| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|
+----------------------------------------+
| ...                                     |

archive library packaging and unpacking

archive The Chinese meaning of the archive library is archives, and its function is to archive (Packaging) and Unpacking (Unpacking). It provides two solutions: tar and zip. The calling paths are <span style="font-size: 15px;">archive/tar</span> and <span style="font-size: 15px;">archive/zip## respectively. </span>#.

We take tar as an example to show how to package and unpack files.

First, create a new target packaging file as out.tar, and then construct some file data readme.txt, gopher.txt and todo.txt for archiving.

import (
 "archive/tar"
  ...
）
func main() {
 // Create and add some files to the archive.
 tarPath := "out.tar"
 tarFile, err := os.Create(tarPath)
 if err != nil {
  log.Fatal(err)
 }
 defer tarFile.Close()
 tw := tar.NewWriter(tarFile)
 defer tw.Close()
 var files = []struct {
  Name, Body string
 }{
  {"readme.txt", "This archive contains some text files."},
  {"gopher.txt", "Gopher names:\nGeorge\nGeoffrey\nGonzo"},
  {"todo.txt", "Get animal handling license."},
 }
 ... 
}

接着依次构建文件头信息，分别指定了文件名、权限和大小（可定义更多文件头字段），再通过<span style="font-size: 15px;">*tar.Writer</span>类型的 tw 变量，按序调用<span style="font-size: 15px;">WriteHeader</span>和<span style="font-size: 15px;">Write</span>方法将需要打包的数据段（文件头+文件内容）写入到out.tar文件。

 ...
 for _, file := range files {
  hdr := &tar.Header{
   Name: file.Name,
   Mode: 0600,
   Size: int64(len(file.Body)),
  }
  if err := tw.WriteHeader(hdr); err != nil {
   log.Fatal(err)
  }
  if _, err := tw.Write([]byte(file.Body)); err != nil {
   log.Fatal(err)
  }
 }
}

执行以上代码，将得到打包后的 out.tar 文件，可通过 tar 工具指定 -tvf 参数查看归档信息。

$ tar -tvf out.tar
-rw-------  0 0      0          38 Jan  1  1970 readme.txt
-rw-------  0 0      0          35 Jan  1  1970 gopher.txt
-rw-------  0 0      0          28 Jan  1  1970 todo.txt

可以看到，指定的文件信息（文件名、权限和大小）符合预期，但其他未指定的元信息是有误的，例如日期（直接给的默认值）。

如果通过 tar 工具，我们可以执行以下命令来提取 out.tar 中的文件。

$ tar -xvf out.tar
x readme.txt
x gopher.txt
x todo.txt

但在程序中实现，应该怎么做呢？

func main() {
 tarPath := "out.tar"
 tarFile, err := os.Open(tarPath)
 if err != nil {
  log.Fatal(err)
 }
 defer tarFile.Close()
 tr := tar.NewReader(tarFile)
 for {
  hdr, err := tr.Next()
  // End of archive
  if err == io.EOF {
   break
  }
  if err != nil {
   log.Fatal(err)
  }
  fmt.Printf("Contents of %s: ", hdr.Name)
  if _, err := io.Copy(os.Stdout, tr); err != nil {
   log.Fatal(err)
  }
  fmt.Println()
 }
}

// Output:
Contents of readme.txt: This archive contains some text files.
Contents of gopher.txt: Gopher names:
George
Geoffrey
Gonzo
Contents of todo.txt: Get animal handling license.

首先需要打开 out.tar，并构造<span style="font-size: 15px;">*tar.Reader</span>类型的 tr 变量。之后，利用<span style="font-size: 15px;">tr.Next</span>依次提取每个数据段内容，并通过 io.Copy(os.Stdout, tr)，将文件内容拷贝至标准输出。直到<span style="font-size: 15px;">tr.Next</span>遇到<span style="font-size: 15px;">io.EOF</span>，它代表读取到了归档文件末尾，则退出提取。

compress 库压缩与解压缩

compress 库中支持了多种压缩方案，包括 bzip2、flate、gzip、lzw 和 zlib，调用路径为<span style="font-size: 15px;">compress/xxx</span>。

我们以常用的 gzip 为例，来展示压缩与解压缩代码。

如果同样是上文中的文件数据 readme.txt、gopher.txt 和 todo.txt，我们想得到 tar 归档且被压缩了的 out.tar.gz 文件，应该如何做呢？

package main

import (
 "archive/tar"
 "compress/gzip"
 ...
)

func main() {
 tarPath := "out.tar.gz"
 tarFile, err := os.Create(tarPath)
 if err != nil {
  log.Fatal(err)
 }
 defer tarFile.Close()
 gz := gzip.NewWriter(tarFile)
 defer gz.Close()
 tw := tar.NewWriter(gz)
 defer tw.Close()
 ...
}

非常简单！只需要将<span style="font-size: 15px;">tar.NewWriter(tarFile)</span>改为<span style="font-size: 15px;">tar.NewWriter(gz)</span>即可，其中<span style="font-size: 15px;">gz</span>是由<span style="font-size: 15px;">gzip.NewWriter(tarFile)</span> 而来。

我们比较有压缩与无压缩的归档 tar 包大小，可以看到文件体积从4.0K压缩为了224B。

$ ls -alh out.tar out.tar.gz
-rw-r--r--  1 slp  staff   4.0K Jul  3 21:52 out.tar
-rw-r--r--  1 slp  staff   224B Jul  3 21:53 out.tar.gz

同理，如果要解压并解包 out.tar.gz 文件，应该如何做呢？

package main

import (
 "archive/tar"
 "compress/gzip"
  ...
)

func main() {
 tarPath := "out.tar.gz"
 tarFile, err := os.Open(tarPath)
 if err != nil {
  log.Fatal(err)
 }
 defer tarFile.Close()
 gz, err := gzip.NewReader(tarFile)
 if err != nil {
  log.Fatal(err)
 }
  defer gz.Close()
 tr := tar.NewReader(gz)
  ...
}

依然很简单！只需要将<span style="font-size: 15px;">tar.NewReader(tarFile)</span>改为<span style="font-size: 15px;">tar.NewReader(gz)</span>即可，其中<span style="font-size: 15px;">gz</span>是由<span style="font-size: 15px;">gzip.NewReader(tarFile)</span> 而来。

总结

本文展示了如何通过archive/tar包实现文件的打包与解包操作，如何通过<span style="font-size: 15px;">compress/gzip</span>包对tar包开展进一步的压缩与解压缩。

在展示<span style="font-size: 15px;">compress/gzip</span>使用时，多封装一层Writer/Reader，即可为tar归档文件增加压缩与解压缩功能。更棒的是，如果你想切换打包/解包、压缩/解压缩策略，仅仅替换掉对应的 Writer/Reader 即可。而这种便利，源于 Go 优秀的流式 IO 设计。

Of course, I only know that this matter needs to be carried out in detail. Readers who have not used the <span style="font-size: 15px;">archive</span> and <span style="font-size: 15px;">compress</span> libraries can try this article An unused solution to try to process packaged and compressed files.

The above is the detailed content of How to pack and compress files in Go. For more information, please follow other related articles on the PHP Chinese website!