There are some differences between reading large files in PHP and reading ordinary files. If your file reaches hundreds of MB or GB, ordinary PHP may be slow to read the file or get stuck. Let me introduce it below. PHP reading large file tips.
Generally, we use fopen or file_get_contents to read files. The former can be read in a loop, and the latter can be read in one go, but they both load the file contents at once. If the loaded file is particularly large, such as several hundred M, and the performance drops when it reaches G size, is there any function or class for processing large files in PHP? The answer is: yes.
PHP is really becoming more and more "object-oriented", and some of the original basic SPL methods have begun to implement classes one after another.
Starting from PHP 5.1.0, the SPL library has added two standard file operation classes: SplFileObject and SplFileInfo. SplFileInfo is implemented starting with PHP 5.1.2.
From the literal meaning, it can be seen that SplFileObject is more powerful than SplFileInfo.
Yes, SplFileInfo is only used to obtain some attribute information of the file, such as file size, file access time, file modification time, suffix name and other values, and SplFileObject inherits these functions of SplFileInfo.
| The code is as follows |
Copy code |
| 代码如下 |
复制代码 |
|
/** 返回文件从X行到Y行的内容(支持php5、php4)
* @param string $filename 文件名
* @param int $startLine 开始的行数
* @param int $endLine 结束的行数
* @return string
*/
function getFileLines($filename, $startLine = 1, $endLine=50, $method='rb') {
$content = array();
$count = $endLine - $startLine;
// 判断php版本（因为要用到SplFileObject，PHP>=5.1.0）
if(version_compare(PHP_VERSION, '5.1.0', '>=')){
$fp = new SplFileObject($filename, $method);
$fp->seek($startLine-1);// 转到第N行, seek方法参数从0开始计数
for($i = 0; $i <= $count; ++$i) {
$content[]=$fp->current();// current()获取当前行内容
$fp->next();// 下一行
}
}else{//PHP<5.1
$fp = fopen($filename, $method);
if(!$fp) return 'error:can not read file';
for ($i=1;$i<$startLine;++$i) {// 跳过前$startLine行
fgets($fp);
}
for($i;$i<=$endLine;++$i){
$content[]=fgets($fp);// 读取文件行内容
}
fclose($fp);
}
return array_filter($content); // array_filter过滤：false,null,''
}
|
/**Return the content of the file from line X to line Y (supports php5, php4)
* @param string $filename file name
* @param int $startLine The starting line number
* @param int $endLine Ending line number
* @return string
*/
function getFileLines($filename, $startLine = 1, $endLine=50, $method='rb') {
$content = array();
$count = $endLine - $startLine;
// Determine the php version (because SplFileObject is used, PHP>=5.1.0)
If(version_compare(PHP_VERSION, '5.1.0', '>=')){
$fp = new SplFileObject($filename, $method);
$fp->seek($startLine-1);//Go to line N, the seek method parameters start counting from 0
for($i = 0; $i <= $count; ++$i) {
$content[]=$fp->current();// current() gets the current row content
$fp->next();//Next line
}
}else{//PHP<5.1
$fp = fopen($filename, $method);
If(!$fp) return 'error:can not read file';
for ($i=1;$i<$startLine;++$i) {// Skip the previous $startLine line
fgets($fp);
}
for($i;$i<=$endLine;++$i){
$content[]=fgets($fp);//Read file line content
}
fclose($fp);
}
Return array_filter($content); //array_filter filter: false,null,''
} <🎜>
|
Ps: There is no "read to the end judgment" added above: !$fp->eof() or !feof($fp). In addition, this judgment affects efficiency. I have to add many, many, many lines of tests myself. The running time is already known, and it is completely unnecessary to add it here.
It can be seen from the above function that using SplFileObject is much faster than fgets below, especially when the number of file lines is very large and the following content needs to be fetched. fgets requires two loops, and it needs to loop $endLine times.
This method took a lot of effort and tested many Chinese writing methods, just to find the most efficient method. If anyone thinks there is anything worth improving, please let me know.
Used, return the content of lines 35270-35280:
The code is as follows
| 代码如下 |
复制代码 |
|
echo ' ';<br>
var_dump(getFileLines('test.php',35270,35280));<br>
echo '';
|
|
Copy code
|
| 代码如下 |
复制代码 |
|
function readBigFile($filename, $count = 20, $tag = "rn") {
$content = "";//最终内容
$current = "";//当前读取内容寄存
$step= 1;//每次走多少字符
$tagLen = strlen($tag);
$start = 0;//起始位置
$i = 0;//计数器
$handle = fopen($filename,'r+');//读写模式打开文件，指针指向文件起始位置
while($i < $count && !feof($handle)) {
fseek($handle, $start, SEEK_SET);//指针设置在文件开头
$current = fread($handle,$step);//读取文件
$content .= $current;//组合字符串
$start += $step;//依据步长向前移动
//依据分隔符的长度截取字符串最后免得几个字符
$substrTag = substr($content, -$tagLen);
if ($substrTag == $tag) { //判断是否为判断是否是换行或其他分隔符
$i++;
$content .= "
";
}
}
//关闭文件
fclose($handle);
//返回结果
return $content;
}
$filename = "csdn.sql";//需要读取的文件
$tag = "n";//行分隔符 注意这里必须用双引号
$count = 100;//读取行数
$data = readBigFile($filename,$count,$tag);
echo $data;
|
echo '';<br>
var_dump(getFileLines('test.php',35270,35280));<br>
echo '';
|
Look at another example
| The code is as follows
|
Copy code
function readBigFile($filename, $count = 20, $tag = "rn") {
$content = "";//Final content
$current = "";//Storage of currently read content
$step= 1;//How many characters to step each time
$tagLen = strlen($tag);
$start = 0; //Start position
$i = 0;//Counter
$handle = fopen($filename,'r+');//Open the file in read-write mode, and the pointer points to the beginning of the file
while($i < $count && !feof($handle)) {
fseek($handle, $start, SEEK_SET);//The pointer is set at the beginning of the file
$current = fread($handle,$step);//Read file
$content .= $current;//Combined string
$start += $step;//Move forward according to the step size
//Truncate the last few characters of the string according to the length of the separator
$substrTag = substr($content, -$tagLen);
if ($substrTag == $tag) { //Determine whether it is a newline or other delimiter
$i++;
$content .= "
";
}
}
//Close the file
fclose($handle);
//Return results
return $content;
}
$filename = "csdn.sql";//The file to be read
$tag = "n";//Line separator Note that double quotes must be used here
$count = 100;//Read the number of rows
$data = readBigFile($filename,$count,$tag);
echo $data;
Note: By using the combination of PHP's fseek and fread, you can read a certain part of the data in the file at will. Regarding the value of the variable $tag passed in by the function, the value passed in varies according to the system. The values are also different: "rn" is used for Windows, "n" is used for linux/unix, and "r" is used for Mac OS.
http://www.bkjia.com/PHPjc/632771.htmlwww.bkjia.comtruehttp: //www.bkjia.com/PHPjc/632771.htmlTechArticlePHP There are some differences between reading large files and reading ordinary files. If your file reaches several hundred MB Or GB, ordinary PHP may be slow to read files or stuck. Let me introduce them below...
|
|
