Home  >  Article  >  Backend Development  >  Sensitive word replacement algorithm, 4 times more efficient than str_replace (with 6,000 sensitive words attached)

Sensitive word replacement algorithm, 4 times more efficient than str_replace (with 6,000 sensitive words attached)

WBOY
WBOYOriginal
2016-07-25 09:08:001520browse

Efficiency comparison (12688 characters, 1 replacement):
  • str_replace: 0.109937906265 seconds
  • strtr: 0.0306839942932 seconds

Comparison of replacement results
  • For example: "Zhang San", "Zhang Sanfeng", "Zhang San Toyota" are all prohibited words (Why is there such a distinction? Please see "法X", "法Xgong")
  • Original text: "I drove a Honda Toyota to work today"
  • strtr: "I drove **** to work today" (replace all four words with *)
  • str_replace: "I drove **Toyota to work today" (only the first match was replaced)
So using str_replace to replace cannot essentially solve the problem.


Time comparison:
Number of keywords: 6712 (no duplication) self init:0.00789093971252 (load xcache) self:0.0354378223419 strtr:0.0311169624329 strtr_array:0.432713985443 str_replace:0.109627008438
  1. require('badword.src.php');
  2. $badword1 = array_combine($badword,array_fill(0,count($badword),'*'));
  3. $bb = 'I am open today San Toyota goes to work';
  4. $str = strtr($bb, $badword1);
Copy code
  1. //Interested friends can study it
  2. function strtr_array(&$str,&$replace_arr) {
  3. $maxlen = 0;$minlen = 1024*128;
  4. if (empty($replace_arr)) return $ str;
  5. foreach($replace_arr as $k => $v) {
  6. $len = strlen($k);
  7. if ($len < 1) continue;
  8. if ($len > $maxlen) $maxlen = $len;
  9. if ($len < $minlen) $minlen = $len;
  10. }
  11. $len = strlen($str);
  12. $pos = 0;$result = '';
  13. while ($pos < ; $len) {
  14. if ($pos + $maxlen > $len) $maxlen = $len - $pos;
  15. $found = false;$key = '';
  16. for($i = 0;$i< $maxlen;++$i) $key .= $str[$i+$pos]; //Original text: memcpy(key,str+$pos,$maxlen)
  17. for($i = $maxlen;$i >= $minlen;--$i) {
  18. $key1 = substr($key, 0, $i); //Original text: key[$i] = '
Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn