Home > Article > Backend Development > PHP sort 100 million numbers
Disassemble into thousands of parts, sort and merge them again.
First create a txt with 100 million QQ numbers.
PHP code?
//Create txt of 100 million QQ numbers (about 85~100 seconds)
set_time_limit (0);
$fn = 'qq.txt';
$fp = fopen($fn, 'w');
$st = microtime(true);
$l = range(0,10000 );
shuffle($l);
foreach ($l as $k=>$v)
{
$arr = range($v*10000+10000,10000*($v+1) +9999);
shuffle($arr);
fputs($fp,implode("n", $arr)."n");
unset($arr);
}
echo microtime(true)-$st;
?>
Wait a minute or two for 100 million random QQs to be created.
QQ number range is >10000. The file size is approximately 840MB.
The following will be classified into thousands of documents.
The length of the QQ number is used as the folder, and the first 3 digits of the QQ number are used as the file name.
PHP code?
//Length number classification (about 360~400 seconds)
set_time_limit(0);
$st = microtime (true);
if(!is_dir('qq_no')) mkdir('qq_no');
$file = fopen('qq.txt', 'r');
$ i=0;
$end_s = '';
while(!feof($file))
{
$g = 1042*1024;
fseek($file,$g*$i);
$s = fread($file, $g);
$end = strrpos($s, "n");
$arr_s = $end_s.substr($s, 0, $end);
$end_s = substr($s, $end);
$arr = explode("n", $arr_s);
foreach ($arr as $k=>$v)
{
if ($ v! = '') {
$ tag = "$ v [0] $ v [1] $ v [2]";
$ text_arr [strlen ($ v)] [$ tag] [] = $v;
}
}
foreach ($text_arr as $k=>$v)
{
$n_dir = 'qq_no /'.$k;
if (! is_dir($n_dir)) mkdir($n_dir); txt ', 'a+');
+ +$i;
}
echo microtime(true)-$st;
?>
Finally, each file is sorted and merged.
PHP code?
// Sorting completed (800~920 seconds)
set_time _limit(0);
$st = microtime(true);
$qq_done = fopen('qq_done.txt', 'a+');
$root = 'qq_no';
$dir_array = scandir($root);
foreach ($dir_array as $key=>$val)
{
if ($val != '.' && $val != '..')
$dirs[$val] = scandir($ root.'/'.$val);
}
foreach ($dirs as $key=>$val)
{
foreach ($val as $v)
{
if ($v != '.' && $v != '..')
{
$file = $root. '/' . $key . '/'. $v;
$c = file_get_contents($file);
$arr = explode("n", $c);
sort($arr);
fputs($qq_done, implode("n",$arr));
unlink($file);
}
}
rmdir($root. '/' . $key);
}
rmdir($root);
echo microtime(true)-$st;
?>