Implementation code for excluding duplicate data from millions of data in PHP array_PHP tutorial

If you get a uid list with more than one million rows, the format is as follows:

Copy the code The code is as follows:

10001000
10001001
10001002
......
10001000
......
10001111　To sort out the duplication, let’s first take a look at the definition of a PHP array: an array in PHP is actually an ordered map. A map is a type that associates values ​​to keys. This type is optimized in many ways, so it can be treated as a real array, or a list (vector), a hash table (an implementation of a map), a dictionary, a set, a stack, a queue, and many more possibilities. The value of an array element can also be another array. Tree structures and multidimensional arrays are also allowed.

In PHP arrays, keys are also called indexes and are unique. We can use this feature to perform deduplication. The sample code is as follows:



Copy the code

The code is as follows: //Define an array to store the results after deduplication

$result = array ();
//Read uid list file
$fp = fopen('test.txt', 'r');

while(!feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "r");
$uid = trim($ uid, "n");

if($uid == '')
{
continue;
}
//Us uid as the key to see if the value exists
if(empty($result[$uid]))
{
$result[$uid] = 1;
}
}

fclose($fp) ;

//Save the result to file
$content = '';
foreach($result as $k => $v)
{
$content .= $k."n";
}
$fp = fopen('result.txt', 'w');
fwrite($fp, $content);
fclose($fp) ;
?>　


With more than 20 lines of code, more than one million data can be deduplicated. The efficiency is also good and very practical. Mobile phone numbers and emails can also be deduplicated in this way.

Also, this method can also be used to deduplicate two files. If you have two uid list files, the format is the same as the uid list above. The sample program is as follows:



Copy the code

The code is as follows: //Define an array to store the results after deduplication

$result = array();
//Read the first uid list file and put $result_1
$fp = fopen('test_1.txt', 'r');
while( !feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "r") ;
$uid = trim($uid, "n");
if($uid == '')
{
continue;
}
//Uid is The key is written to $result. If there is a duplicate, it will be overwritten.
$result[$uid] = 1;
}
fclose($fp);
//Read the second uid list file , and perform deduplication operation
$fp = fopen('test_2.txt', 'r');
while(!feof($fp))
{
$uid = fgets($ fp);
$uid = trim($uid);
$uid = trim($uid, "r");
$uid = trim($uid, "n");
if($uid == '')
{
continue;
}
//Us uid as the key to see if the value exists
if(empty($result[$uid] ))
{
$result[$uid] = 1;
}
}
fclose($fp);
//What is saved in $result is after deduplication The result can be output to a file, and the code is omitted
?>


If you think about it carefully, it is not difficult to find that using this feature of arrays can solve more problems in our work.

http://www.bkjia.com/PHPjc/321981.html

www.bkjia.comtruehttp: //www.bkjia.com/PHPjc/321981.htmlTechArticleIf you get a uid list with more than one million rows, the format is as follows: Copy the code as follows: 10001000 10001001 10001002 ...... 10001000 ...... 10001111 In fact, using php arrays...

Statement：

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Previous article：PHP XML error parsing SOAP payload on line 1_PHP教程Next article：PHP XML error parsing SOAP payload on line 1_PHP教程

Related articles

See more

• How to use cURL to implement Get and Post requests in PHP
• How to use cURL to implement Get and Post requests in PHP
• How to use cURL to implement Get and Post requests in PHP
• How to use cURL to implement Get and Post requests in PHP
• All expression symbols in regular expressions (summary)