Home >Backend Development >PHP Tutorial >Please read the PHP Chinese character substitution and pattern matching issues_PHP Tutorial
Please read the PHP Chinese character substitution and pattern matching issues_PHP Tutorial
- WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOriginal
- 2016-07-13 17:26:08782browse
Author: bluedoor
Original post address: http://www.anbbs.com/anbbs/index.php?f_id=3&page=1
I have been working on a keyword highlighting program for the past two days. I have written it. The program ran fine when tested locally, but as soon as I went up to the page, piles of garbled characters appeared. Let alone highlighting them, it was impossible to read them!
I looked for errors and searched again and again. I found that there was no problem with English, but it was easy to have problems when encountering Chinese characters. Sometimes there would be problems when encountering Chinese characters.
To summarize:
When using pattern matching, such as: preg_match_all($pat,...) and preg_replace($pat,...)...
The situations that are prone to problems are as follows:
preg_match_all("/(Chinese character)+/ism","I am a Chinese character, let's see what you do to me!",$m_a);
This pattern is very simple, it just matches "Chinese characters". In this case, the pattern containing Chinese characters can be successfully matched, but don't be too happy too early. The result is uncertain. Why are you not sure? Read on.
The problems that must occur are as follows:
preg_match_all("/[Chinese characters]+/ism","I am a Chinese character, let’s see what you do to me!",$m_a);
I wanted to match "Chinese characters" ", "zi" or "kanji". This is bound to cause problems. The matching results will be a bunch of garbled characters, and there may be an infinite loop. Why does this happen? This is because PHP does not use UNICODE internally and does not support multi-byte text, so a "Chinese character" is treated as a 4-byte ASCII for pattern matching. It would be strange if there is no error!
Later, I tried to rewrite the pattern matching, and found a seemingly (why do I say seemingly? Look later) method that can solve the problem:
preg_match_all("/(汉|字)+/ism","我It's a Chinese character, let's see what you do to me! ",$m_a);
Writing like this can match "汉", "字" or "汉字", the result in $m_a
Array
(
[0] => Array
(
[0] => Chinese character
)
[1] => Array
(
[0] => Chinese character
)
)
How about a fully matching string appears! But it’s too early to be happy, and problems will often occur when using it in practice! I looked for the problem again and finally found the root of the problem! PHP does not support multi-byte text, so pattern matching and character operations are performed after internal code conversion (I don’t know if this is correct). Here is an example:
eregi_replace("性", "No", "Responsible"); This operation is to replace the word "gender" in the string "Responsible" with "No". What is the final result? Because there is no "sex" in "sense of responsibility", the result should be that "sense of responsibility" was returned without executing the replacement operation, but the result turned out to be "sense of responsibility"!
I didn’t expect it! Why? You will understand if you look at the ASCII code. Two ASCII codes code a Chinese character "sense of responsibility". The ASCII codes are: 211,208 (have), 212,240 (responsibility), 200,206 (any), 184,208 (sense)
And " The encoding of "sex" is: 208,212 (sex), which happens to be consistent with the combination of the 2nd byte and the 1st byte of some! So PHP knew to find the same pattern to match, split the Chinese characters into half and then combined them with the replaced string, so something went wrong!
At that time, I thought that the most commonly used str_replace() should not be a problem, but in fact str_replace() will also make errors when performing the same operation! Now I think I was really lucky to have done kanji substitution before! It may be that the Chinese character substitutions performed at that time were relatively long Chinese character strings, so the above situation is less likely to occur. Even if nothing goes wrong, know that it's not safe!
There are problems, and we have to continue to work. The only difficulty we have to overcome is our current self.
Fortunately, I remembered a set of PHP extension modules, Multibyte String Functions, which add many functions that support multi-byte text operations, such as: ereg_replace() corresponds to mb_ereg_replace() and so on. For specific function descriptions, please consult relevant articles.
Summary: For safe operations on Chinese characters, it is best to use Multibyte String Functions.