PHP uses file function and fseek function to read large files efficiency analysis

PHP can use the file function and fseek function to read large files, but there may be differences in efficiency between the two. This article introduces to you a comparative analysis of the efficiency of reading large files between the php file function and the fseek function. Friends who need it can refer to it. .

1. Directly use the file function to operate

Since the file function reads all the contents into the memory at one time, PHP prevents some poorly written programs from occupying too much memory and causing insufficient system memory and causing the server to crash. Downtime, so by default the maximum memory usage is limited to 16M. This is set through memory_limit = 16M in php.ini. If this value is set to -1, the memory usage is not limited.

The following is a piece of code that uses file to extract the last line of this file:

<?php
   ini_set(&#39;memory_limit&#39;, &#39;-1&#39;);
   $file = &#39;access.log&#39;;
   $data = file($file);
   $line = $data[count($data) - 1];
   echo $line;
   ?>

The entire code execution takes 116.9613 (s).

My machine has 2G of memory. When I press F5 to run, the system turns gray and only recovers after almost 20 minutes. It can be seen that the consequences of reading such a large file directly into the memory are serious, so I won’t discuss it here. As a last resort, memory_limit cannot be set too high, otherwise you will have to call the computer room to reset the machine.

2. Directly use PHP’s fseek to perform file operations

This method is the most common method. It does not need to read all the contents of the file into the content, but operates directly through pointers, so the efficiency is quite high. Efficient. When using fseek to operate files, there are many different methods, and the efficiency may be slightly different. The following are two commonly used methods:

Method 1

First find the last EOF of the file through fseek. Then find the starting position of the last row, take the data of this row, then find the starting position of the next row, then take the position of this row, and so on, until the $num row is found.

#The implementation code is as follows

<?php
   $fp = fopen($file, "r");
   $line = 10;
   $pos = -2;
   $t = " ";
   $data = "";
   while ($line > 0)
   {
       while ($t != "\n")
       {
           fseek($fp, $pos, SEEK_END);
           $t = fgetc($fp);
           $pos--;
       }//  http://www.manongjc.com
       $t = " ";
       $data .= fgets($fp);
       $line--;
   }
   fclose($fp);
   echo $data
   ?>

The entire code execution takes 0.0095 (s)

Method 2

still uses fseek to read from the end of the file, but this time it is not reading one by one, but It is read piece by piece. Each time a piece of data is read, the read data is placed in a buf, and then the number of newline characters (n) is used to determine whether the last $num rows of data have been read.

#The implementation code is as follows

<?php
   $fp = fopen($file, "r");
   $num = 10;
   $chunk = 4096;
   $fs = sprintf("%u", filesize($file));
   $max = (intval($fs) == PHP_INT_MAX) ? PHP_INT_MAX : filesize($file);
   for ($len = 0; $len < $max; $len += $chunk)
   {
       $seekSize = ($max - $len > $chunk) ? $chunk : $max - $len;
       fseek($fp, ($len + $seekSize) * -1, SEEK_END);
       $readData = fread($fp, $seekSize) . $readData;
       if (substr_count($readData, "\n") >= $num + 1)
       {
           // 作者：码农教程   http://www.manongjc.com
           preg_match("!(.*?\n){" . ($num) . "}$!", $readData, $match);
           $data = $match[0];
           break;
       }
   }
   fclose($fp);
   echo $data;
   ?>

The entire code execution takes 0.0009(s).

Method 3

<?php
   function tail($fp, $n, $base = 5)
   {
       assert($n > 0);
       $pos = $n + 1;
       $lines = array();
       while (count($lines) <= $n)
       {
           try
           {
               fseek($fp, -$pos, SEEK_END);
           }
           catch (Exception $e)
           {
               fseek(0);
               break;
           }
           $pos *= $base;
           while (!feof($fp))
           {
               array_unshift($lines, fgets($fp));
           }
       }
    
       return array_slice($lines, 0, $n);
   }
    
   var_dump(tail(fopen("access.log", "r+"), 10));
   ?>

The entire code execution takes 0.0003(s)