Home  >  Article  >  Backend Development  >  PHP sort 100 million numbers

PHP sort 100 million numbers

巴扎黑
巴扎黑Original
2016-11-22 10:05:381220browse

Disassemble into thousands of parts, sort and merge them again.


First create a txt with 100 million QQ numbers.

PHP code?



//Create txt of 100 million QQ numbers (about 85~100 seconds)

set_time_limit (0);

$fn = 'qq.txt';

$fp = fopen($fn, 'w');

$st = microtime(true);

$l = range(0,10000 );

shuffle($l);

foreach ($l as $k=>$v)

{

$arr = range($v*10000+10000,10000*($v+1) +9999);

shuffle($arr);

fputs($fp,implode("n", $arr)."n");

unset($arr);

}

echo microtime(true)-$st;

?>






Wait a minute or two for 100 million random QQs to be created.

QQ number range is >10000. The file size is approximately 840MB.



The following will be classified into thousands of documents.

The length of the QQ number is used as the folder, and the first 3 digits of the QQ number are used as the file name.

PHP code?


//Length number classification (about 360~400 seconds)

set_time_limit(0);

$st = microtime (true);

if(!is_dir('qq_no')) mkdir('qq_no');

$file = fopen('qq.txt', 'r');

$ i=0;

$end_s = '';

while(!feof($file))

{

$g = 1042*1024;

fseek($file,$g*$i);

$s = fread($file, $g);

$end = strrpos($s, "n");

$arr_s = $end_s.substr($s, 0, $end);

$end_s = substr($s, $end);

$arr = explode("n", $arr_s);

foreach ($arr as $k=>$v)

{

if ($ v! = '') {

$ tag = "$ v [0] $ v [1] $ v [2]";

$ text_arr [strlen ($ v)] [$ tag] [] = $v;

}

}

foreach ($text_arr as $k=>$v)

{

 $n_dir = 'qq_no /'.$k;

    if (! is_dir($n_dir)) mkdir($n_dir); txt ', 'a+');

+ +$i;

}

echo microtime(true)-$st;

?>

Finally, each file is sorted and merged.

PHP code?

// Sorting completed (800~920 seconds)

set_time _limit(0);

$st = microtime(true);


$qq_done = fopen('qq_done.txt', 'a+');

$root = 'qq_no';

$dir_array = scandir($root);

foreach ($dir_array as $key=>$val)

{

if ($val != '.' && $val != '..')

$dirs[$val] = scandir($ root.'/'.$val);

}

foreach ($dirs as $key=>$val)

{

foreach ($val as $v)

    {

        if ($v != '.' && $v != '..')

        {

            $file = $root. '/' . $key . '/'. $v;

            $c = file_get_contents($file);

            $arr = explode("n", $c);

            sort($arr);

            fputs($qq_done, implode("n",$arr));

            unlink($file);

        }

    }

    rmdir($root. '/' . $key);

}

rmdir($root);

 

echo  microtime(true)-$st;

 

?>

   



Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn