PHP implementation of reading a 1G file size_PHP tutorial

The requirements are as follows: There is a log file of about 1G with about 5 million lines. Use PHP to return the contents of the last few lines.

1. Directly use the file function to operate or file_get_content() will definitely report a memory overflow
Note: Since the file function reads all the content into the memory at one time, PHP in order to prevent some writes Worse programs occupy too much memory, causing insufficient system memory and causing the server to crash. Therefore, by default, the maximum memory usage is limited to 16M. This is set through memory_limit = 16M in php.ini. This value If set to -1, memory usage is unlimited.

The following is a piece of code that uses file to extract the last line of this file.

Copy the code The code is as follows:

ini_set('memory_limit','-1');
$file = 'access.log';
$data = file($file);
$line = $data[count($data) -1];2. Directly call the linux tail command to display the last few lines

Under the linux command line, you can directly use tail -n 10 access.log to easily display the last few lines of the log file. OK, you can directly use php to call the tail command, and execute the php code as follows.

Copy the code The code is as follows:

file = 'access.log';
$file = escapeshellarg($file); // Safely escape command line parameters
$line = `tail -n 1 $file`;
echo $line; 3. Directly use php’s fseek to perform file operations

This method is the most common method. It does not need to read all the contents of the file into memory, but operates directly through pointers, so The efficiency is quite efficient. When using fseek to operate files, there are many different methods, and the efficiency may be slightly different. The following are two commonly used methods.

Method 1:
First find the last EOF of the file through fseek, then find the starting position of the last line, take the data of this line, and search again The starting position of a row, then the position of this row, and so on, until the $num row is found.

Copy code The code is as follows:

function tail($fp,$n,$base=5)
{
assert($n>0);
$pos = $n+1;
$lines = array();
while(count($lines)< =$n){
try{
fseek($fp,-$pos,SEEK_END);
} catch (Exception $e){
fseek(0);
break;
}
$pos *= $base;
while(!feof($fp)){
array_unshift($lines,fgets($fp));
}
}
return array_slice($ lines,0,$n);
}
var_dump(tail(fopen("access.log","r+"),10));

Method 2:
Still use fseek to read from the end of the file, but this time it is not read one by one, but one by one. Each time a piece of data is read, The read data is placed in a buf, and then the number of newline characters (n) is used to determine whether the last $num rows of data have been read.

The implementation code is as follows

Copy code The code is as follows:

$fp = fopen($file, "r") ;
$line = 10;
$pos = -2;
$t = " ";
$data = "";
while ($line > 0) {
while ($t != "n") {
fseek($fp, $pos, SEEK_END);
$t = fgetc($fp);
$pos --;
}
$t = " ";
$data .= fgets($fp);
$line --;
}
fclose ($fp);

echo $data method three:

Copy code The code is as follows:

$fp = fopen($file, "r");
$num = 10;
$chunk = 4096;
$fs = sprintf("%u", filesize( $file));
$max = (intval($fs) == PHP_INT_MAX) ? PHP_INT_MAX : filesize($file);
for ($len = 0; $len < $max; $len + = $chunk) {
$seekSize = ($max - $len > $chunk) ? $chunk : $max - $len;
fseek($fp, ($len + $seekSize) * -1 , SEEK_END);
$readData = fread($fp, $seekSize) . $readData;

if (substr_count($readData, "n") >= $num + 1) {
preg_match("!(.*?n){".($num)."}$!", $readData, $match);
$data = $match[0];
break;
}
}
fclose($fp);
echo $data;

http://www.bkjia.com/PHPjc/328160.htmlwww.bkjia.comtruehttp: //www.bkjia.com/PHPjc/328160.htmlTechArticleThe requirements are as follows: There is a log file of about 1G with about 5 million lines. Use php to return the last few line content. 1. Directly use the file function to operate or file_get_content() will definitely report...