PHP encapsulation class sharing for quickly reading large CSV files line by line (also applicable to other very large text files)_PHP Tutorial

The reading of large CSV files has been described before (code example of PHP reading and processing large CSV files line by line), but there are still some problems in how to quickly and completely operate large files.

1. How to quickly get the total number of lines of a large CSV file?

Method 1: Get the file content directly, use newline characters to split it to get the total number of lines. This method is feasible for small files, but not feasible when processing large files;
Method 2: Use fgets to traverse line by line, and get Get the total number of lines. This method is better than method one, but there is still the possibility of timeout for large files;
Method three: Use the SplFileObject class to directly position the pointer to the end of the file, and get the total number of lines through the SplFileObject::key method. This method is feasible and efficient.

Specific implementation method:

Copy code The code is as follows:

$csv_file = 'path/bigfile.csv';
$spl_object = new SplFileObject($csv_file, 'rb');
$spl_object->seek(filesize($csv_file));
echo $spl_object->key();

2. How to quickly obtain data from large CSV files?

Still using PHP's SplFileObject class to achieve quick positioning through the seek method.

Copy code The code is as follows:

$csv_file = 'path/bigfile.csv';
$start = 100000 ; // Read from line 100000
$num = 100; // Read 100 lines
$data = array();
$spl_object = new SplFileObject($csv_file, 'rb') ;
$spl_object->seek($start);
while ($num-- && !$spl_object->eof()) {
$data[] = $spl_object->fgetcsv ();
$spl_object->next();
}
print_r($data);

3. Based on the above two points, organize it into a class for reading csv files:

Copy the code The code is as follows:

class CsvReader {
private $csv_file;
private $spl_object = null;
private $error;

public function __construct($csv_file = '') {
if($csv_file && file_exists($csv_file)) {
$this->csv_file = $csv_file;
}
}

public function set_csv_file($csv_file) {
if(!$csv_file || !file_exists($csv_file)) {
$this->error = 'File invalid';
return false;
}
$this- >csv_file = $csv_file;
$this->spl_object = null;
}

public function get_csv_file() {
return $this->csv_file;
}

private function _file_valid($file = '') {
$file = $file ? $file : $this->csv_file;
if(!$file || !file_exists($ file)) {
return false;
}
if(!is_readable($file)) {
return false;
}
return true;
}

private function _open_file() {
if(!$this->_file_valid()) {
$this->error = 'File invalid';
return false;
}
if($this->spl_object == null) {
$this->spl_object = new SplFileObject($this->csv_file, 'rb');
}
return true ;
}

public function get_data($length = 0, $start = 0) {
if(!$this->_open_file()) {
return false;
}
$length = $length ? $length : $this->get_lines();
$start = $start - 1;
$start = ($start < 0) ? 0 : $start;
$data = array();
$this->spl_object->seek($start);
while ($length-- && !$this->spl_object->eof()) {
$data[] = $this->spl_object->fgetcsv();
$this->spl_object->next();
}
return $data;
}

public function get_lines() {
if(!$this->_open_file()) {
return false;
}
$this->spl_object-> seek(filesize($this->csv_file));
return $this->spl_object->key();
}

public function get_error() {
return $this->error;
}
}

The calling method is as follows:

Copy codeThe code is as follows:

include('CsvReader.class.php');

$csv_file = 'path/bigfile.csv';

$csvreader = new CsvReader($csv_file);

$line_number = $csvreader->get_lines();

$data = $csvreader->get_data(10);

echo $line_number, chr(10);

print_r($data);

In fact, the above-mentioned CsvReader class is not only for CSV large files, it can also be used for large or very large files of other text types, provided that the fgetcsv method in the class is slightly changed to current.

http://www.bkjia.com/PHPjc/751935.htmlwww.bkjia.comtruehttp: //www.bkjia.com/PHPjc/751935.htmlTechArticleThe reading of large CSV files has been described before (PHP reads and processes large CSV files line by line Code examples), but there are still some problems in how to quickly and completely operate large files. ...