Best Practices for PHP and UTF-8

This article is the second part of "PHP, strings, encoding, UTF-8" related knowledge. Let me start with the conclusion - Use UTF-8 encoding in all aspects of PHP.

PHP language level does not support Unicode character set, but most problems can be solved through UTF-8 encoding.

The best practice is to clearly know the input encoding (detect it if you don’t know), uniformly convert it internally to UTF-8 encoding, and the output encoding is also uniformly UTF-8 encoding.

How to handle UTF-8 at the PHP level

When operating the Unicode character set, be sure to install the mbstring extension and use the corresponding functions instead of the native string functions. For example, if a file is encoded as UTF-8 PHP code and it is wrong to use the strlen() function, please use the mb_strlen() function instead.

Most functions of the mbstring extension need to be processed based on an encoding (internal encoding). Please be sure to use UTF-8 encoding uniformly. Most of this can be configured in PHP.INI.

Starting from PHP 5.6, the default_charset configuration can replace mbstring.http_input, mbstring.http_output.

Another important configuration is mbstring.language. The default value is Neutral (UTF-8).

Note that file encoding and the internal encoding of mbstring extension are not the same concept.

In summary:

• Try to use UTF-8 in the parts of PHP.INI that involve mbstring extension.

• Please use mbstring extension function instead of native string operation function.

• When using related functions, please be sure to understand the encoding of the characters you operate. When using the corresponding functions, display the UTF-8 encoding parameters, such as htmlentities () The third parameter of the function is written in UTF-8.

How to handle UTF-8 in file IO operations

Here is an example, if you want to open a file but don’t know what encoding the file content is, what should you do? What about processing?

The best practice is to uniformly convert to UTF-8 when opening, and then convert back to the original encoding after modifying the content and save it to the file. Look at the code:

if ( mb_internal_encoding()!="UTF-8") {
        mb_internal_encoding("UTF-8");
}
 
$file = "file.txt"; //一个编码为gbk的中文文件
$str= file_get_contents($file);
//不管来源是什么编码，统一显示的时候转换为 UTF-8
 if (mb_check_encoding($str,"GBK"))
    $str =  mb_convert_encoding($str,"UTF-8",“GBK”);
 
$str ="修改内容";
$str =  mb_convert_encoding($str,$srcbm,"UTF-8"); //原样转回去
file_put_contents($file,$str);

The best practices for Mysql and UTF-8

This is relatively simple. First, make sure that your Mysql is UTF-8. Then the Mysql client also maintains UTF-8 when connecting. Specifically in PHP, when imysql or PDO extension connects to Mysql, UTF-8 is set as the connection encoding. If both sides are consistent, you will generally not encounter problems.

Best practices for browsers and UTF-8

This is also relatively simple, that is, if your output content is a web page, then your string processing output should always be kept as UTF- 8; at the same time, the default_charset is also clearly set to UTF-8 in PHP.INI; the Meta Tag of HTML is also clearly marked as UTF-8.

Is everything okay now? No, although the server and browser allow users to use UTF-8 encoding, the user's behavior is not binding. He may enter characters in other encodings or upload files. The name is a character in another encoding, so what should I do? The user's encoding can be detected via the mb_http_input() and mb_check_encoding() functions and then converted to UTF-8 internally. Ensure that at any level, the final processing is UTF-8 encoding. In other words, you need a way to know what encoding your input is, and the encoding of the control output after processing is UTF-8.

It is not recommended to use the mbstring.encoding_translation directive and mb_detect_encoding() function. Tortured me for half a day.

Operating system and UTF-8 best practices

Due to operating system reasons, PHP will have different processing mechanisms when processing Unicode file names.

In Linux, the file name is always UTF-8 encoded, while in the Chinese Windows environment, the file name is always GBK encoded. Just remember this.

Let’s illustrate with an example:

//命令行程序函数，运行在中文版 Windows 10 操作系统 ，文件编码为 UTF-8
 function filenameexample() {
    $filename = "测试.txt" ;
    $gbk_filename = iconv("UTF-8","GBK",$filename);
    file_put_contents($gbk_filename, "测试");
    echo file_get_contents($gbk_filename);
}
 function scandirexample() {
    $arr = scandir("./tmp");
    foreach ($arr as $v) {
        if ($v == "." || $v =="..")
            continue ;
        $filename = iconv( "GBK","UTF-8",$v ) ;
        $content = file_get_contents("./tmp/" . $v );
    }
}

If you don’t want to write a program that is compatible with Windows and Linux, you can urlencode the file name, such as:

function urlencodeexample() {
   $filename = "测试2.txt" ;
   $urlencodefilename = urlencode($filename) ;
   file_put_contents($urlencodefilename, "测试");
   echo file_get_contents($urlencodefilename);
}

Using PHP When the header() function downloads files, you must also consider the browser and operating system (most people use Windows). For Chrome, the output file name encoding can be UTF-8, and Chrome will automatically convert the file name. Encoded for GBK.

For the lower version of IE, it inherits the operating system environment, so if the downloaded file name is Chinese, it must be transcoded into UTF-8 encoding, otherwise the user will see a garbled file when downloading. name. Explain through code:

$agent=$_SERVER["HTTP_USER_AGENT"];
if(strpos($agent,'MSIE')!==false  ｛
    $filename = iconv("UTF-8","GBK","附件.txt");
    header("Content-Disposition: attachment; filename=\"$filename\"");
｝