Home  >  Article  >  Backend Development  >  How to quickly remove duplicate data in php

How to quickly remove duplicate data in php

藏色散人
藏色散人Original
2020-08-13 10:38:176339browse

php method to remove duplicate data: 1. Use the "array_unique" method to deduplicate array elements, and use the "array_values" function to reorder the key values; 2. Use the "array_flip" method to deduplicate.

How to quickly remove duplicate data in php

Recommended: "PHP Video Tutorial"

php Quickly remove duplicates from array elements

1. Use the array_unique method to deduplicate

To deduplicate array elements, we generally use the array_unique method. This method can deduplicate the elements in the array.

<?php
$arr = array(1,1,2,3,3,3,4,4,5,6,6,7,8,8,9,9,9);
$arr = array_unique($arr);
$arr = array_values($arr);
print_r($arr);
?>

Output:

Array
(
    [0] => 1
    [1] => 2
    [2] => 3
    [3] => 4
    [4] => 5
    [5] => 6
    [6] => 7
    [7] => 8
    [8] => 9
)

After deduplication, the key values ​​will be out of order. You can use array_values ​​to reorder the key values.

2. Efficiency of using array_unique method to remove duplicates

<?php
$arr = array();
// 创建100000个随机元素的数组
for($i=0; $i<100000; $i++){
    $arr[] = mt_rand(1,99);
}
// 记录开始时间
$starttime = getMicrotime();
// 去重
$arr = array_unique($arr);
// 记录结束时间
$endtime = getMicrotime();
$arr = array_values($arr);
echo &#39;unique count:&#39;.count($arr).&#39;<br>&#39;;
echo &#39;run time:&#39;.(float)(($endtime-$starttime)*1000).&#39;ms<br>&#39;;
echo &#39;use memory:&#39;.getUseMemory();
/**
 * 获取使用内存
 * @return float
 */
function getUseMemory(){
    $use_memory = round(memory_get_usage(true)/1024,2).&#39;kb&#39;;
    return $use_memory;
}
/**
 * 获取microtime
 * @return float
 */
function getMicrotime(){
    list($usec, $sec) = explode(&#39; &#39;, microtime());
    return (float)$usec + (float)$sec;
}
?>
unique count:99
run time:653.39303016663ms
use memory:5120kb

Using array_unique method to remove duplicates, the running time takes about 650ms and the memory usage is about 5m

3. Faster array removal Duplication method

php has a key-value exchange method array_flip. We can use this method to remove duplicates. Because of key-value exchange, the original duplicate values ​​will become the same key.

Then perform a key-value exchange again, and exchange the keys and values ​​back to complete deduplication.

<?php
$arr = array();
// 创建100000个随机元素的数组
for($i=0; $i<100000; $i++){
    $arr[] = mt_rand(1,99);
}
// 记录开始时间
$starttime = getMicrotime();
// 使用键值互换去重
$arr = array_flip($arr);
$arr = array_flip($arr);
// 记录结束时间
$endtime = getMicrotime();
$arr = array_values($arr);
echo &#39;unique count:&#39;.count($arr).&#39;<br>&#39;;
echo &#39;run time:&#39;.(float)(($endtime-$starttime)*1000).&#39;ms<br>&#39;;
echo &#39;use memory:&#39;.getUseMemory();
/**
 * 获取使用内存
 * @return float
 */
function getUseMemory(){
    $use_memory = round(memory_get_usage(true)/1024,2).&#39;kb&#39;;
    return $use_memory;
}
/**
 * 获取microtime
 * @return float
 */
function getMicrotime(){
    list($usec, $sec) = explode(&#39; &#39;, microtime());
    return (float)$usec + (float)$sec;
}
?>
unique count:99
run time:12.840032577515ms
use memory:768kb

Using the array_flip method to remove duplicates takes about 18ms to run and takes up about 2m of memory.

Therefore, using the array_flip method to delete duplicates takes 98% less running time than using the array_unique method, and the memory usage is reduced by 4/ 5;

The above is the detailed content of How to quickly remove duplicate data in php. For more information, please follow other related articles on the PHP Chinese website!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn