Detailed explanation of Huffman encoding/decoding steps in PHP

This time I will bring you a detailed explanation of the steps to implement Huffman encoding/decoding in PHP. What are the precautions for implementing Huffman encoding/decoding in PHP? Here are practical cases, let’s take a look.

This article will use PHP to practice Huffman encoding and decoding.

1. Encoding

Word count

The first step of Huffman encoding is to count the occurrence of each character in the document times, PHP's built-in function count_chars() can do:

$input = file_get_contents('input.txt');
$stat = count_chars($input, 1);

Construct Huffman tree

Connect Next, construct the Huffman tree based on the statistical results. Construction method is described in detail in Wikipedia. Here is a simple version written in PHP:

$huffmanTree = [];
foreach ($stat as $char => $count) {
  $huffmanTree[] = [
    'k' => chr($char),
    'v' => $count,
    'left' => null,
    'right' => null,
  ];
}
// 构造树的层级关系，思想见wiki：https://zh.wikipedia.org/wiki/%E9%9C%8D%E5%A4%AB%E6%9B%BC%E7%BC%96%E7%A0%81
$size = count($huffmanTree);
for ($i = 0; $i !== $size - 1; $i++) {
  uasort($huffmanTree, function ($a, $b) {
    if ($a['v'] === $b['v']) {
      return 0;
    }
    return $a[&#39;v&#39;] < $b[&#39;v&#39;] ? -1 : 1;
  });
  $a = array_shift($huffmanTree);
  $b = array_shift($huffmanTree);
  $huffmanTree[] = [
    &#39;v&#39; => $a[&#39;v&#39;] + $b[&#39;v&#39;],
    &#39;left&#39; => $b,
    &#39;right&#39; => $a,
  ];
}
$root = current($huffmanTree);

After calculation, $root will point to the root node of the Huffman tree

Generate a coding dictionary based on the Huffman tree

With the Huffman tree, you can generate a dictionary for encoding:

function buildDict($elem, $code = '', &$dict) {
  if (isset($elem['k'])) {
    $dict[$elem['k']] = $code;
  } else {
    buildDict($elem['left'], $code.'0', $dict);
    buildDict($elem['right'], $code.'1', $dict);
  }
}
$dict = [];
buildDict($root, '', $dict);

Write the file

Use the dictionary to encode the file content and write Import the file. There are several things to pay attention to when writing Huffman encoding to a file:

After writing the encoding dictionary and encoding content to the file together, it is impossible to distinguish their boundaries, so they need to be written at the beginning of the file. Number of bytes occupied

The fwrite() function provided by PHP can write 8-bit (one byte) or an integer multiple of 8 bits at a time. However, in Huffman encoding, a character may be represented by only 1-bit, and PHP does not support the operation of writing only 1-bit to the file. Therefore, we need to splice the encoding ourselves, and only write the file after every 8-bit is obtained.

Write every time 8-bit is obtained

Similar to the second item, the final file size must be an integer multiple of 8-bit. So if the size of the entire encoding is 8001-bit, 7 0s must be added at the end

$dictString = serialize($dict);
// 写入字典和编码各自占用的字节数
$header = pack('VV', strlen($dictString), strlen($input));
fwrite($outFile, $header);
// 写入字典本身
fwrite($outFile, $dictString);
// 写入编码的内容
$buffer = '';
$i = 0;
while (isset($input[$i])) {
  $buffer .= $dict[$input[$i]];
  while (isset($buffer[7])) {
    $char = bindec(substr($buffer, 0, 8));
    fwrite($outFile, chr($char));
    $buffer = substr($buffer, 8);
  }
  $i++;
}
// 末尾的内容如果没有凑齐 8-bit，需要自行补齐
if (!empty($buffer)) {
  $char = bindec(str_pad($buffer, 8, '0'));
  fwrite($outFile, chr($char));
}
fclose($outFile);

Decoding

The decoding of Huffman encoding is relatively simple: read first Get the encoding dictionary, and then decode the original characters according to the dictionary.

There is a problem that needs to be noted during the decoding process: since we have added several 0-bits at the end of the file during the encoding process, if these 0-bits happen to be the encoding of a certain character in the dictionary, This will cause incorrect decoding.

So during the decoding process, when the number of decoded characters reaches the document length, decoding will stop.

<?php
$content = file_get_contents(&#39;a.out&#39;);
// 读出字典长度和编码内容长度
$header = unpack(&#39;VdictLen/VcontentLen&#39;, $content);
$dict = unserialize(substr($content, 8, $header[&#39;dictLen&#39;]));
$dict = array_flip($dict);
$bin = substr($content, 8 + $header[&#39;dictLen&#39;]);
$output = &#39;&#39;;
$key = &#39;&#39;;
$decodedLen = 0;
$i = 0;
while (isset($bin[$i]) && $decodedLen !== $header[&#39;contentLen&#39;]) {
  $bits = decbin(ord($bin[$i]));
  $bits = str_pad($bits, 8, &#39;0&#39;, STR_PAD_LEFT);
  for ($j = 0; $j !== 8; $j++) {
    // 每拼接上 1-bit，就去与字典比对是否能解码出字符
    $key .= $bits[$j];
    if (isset($dict[$key])) {
      $output .= $dict[$key];
      $key = &#39;&#39;;
      $decodedLen++;
      if ($decodedLen === $header[&#39;contentLen&#39;]) {
        break;
      }
    }
  }
  $i++;
}
echo $output;

Test

We save the HTML code of the Huffman coding Wiki page locally and conduct the Huffman coding test. The test results are:

Before encoding: 418,504 bytes

After encoding: 280,127 bytes

The space is saved by 33%. If the original text has a lot of repeated content, the space saved by Huffman encoding can reach 50% Above.

In addition to the text content, we try to Huffman encode a binary file, such as the f.lux installation program. The test results are as follows:

Before encoding: 770,384 bytes

After encoding: 773,076 bytes

After encoding, it takes up more space. On the one hand, it is because we do not do additional processing when storing the dictionary, which takes up a lot of space. Less space. On the other hand, in binary files, the probability of each character appearing is relatively even, and the advantages of Huffman coding cannot be used.

I believe you have mastered the method after reading the case in this article. For more exciting information, please pay attention to other related articles on the php Chinese website!

Recommended reading:

Detailed explanation of the steps for using the PHP quick sort algorithm

Detailed explanation of the iterator steps implemented by PHP based on SPL

The above is the detailed content of Detailed explanation of Huffman encoding/decoding steps in PHP. For more information, please follow other related articles on the PHP Chinese website!