How to Avoid Memory Exhaustion Errors when Using file_get_contents() with Large Files?

File_get_contents Memory Exhaustion: A Comprehensive Solution

When dealing with large file processing, the infamous PHP Fatal error: Allowed memory exhausted error can be a recurring issue. This problem arises when file_get_contents() attempts to read the entire contents of a sizeable file into memory, often exceeding the allocated memory limit.

Alternative to file_get_contents()

Instead of loading the entire file into memory, a more efficient approach is to open the file as a pointer and read it in smaller chunks using fread(). This allows for memory management, which is critical for handling large files.

Below is a custom function that mimics the functionality of Node.js's file processing API:

<code class="php">function file_get_contents_chunked($file, $chunk_size, $callback)
{
    try {
        $handle = fopen($file, "r");
        $i = 0;
        while (!feof($handle)) {
            call_user_func_array($callback, array(fread($handle, $chunk_size), &$handle, $i));
            $i++;
        }
    } catch (Exception $e) {
        trigger_error("file_get_contents_chunked::" . $e->getMessage(), E_USER_NOTICE);
        return false;
    }
    fclose($handle);

    return true;
}</code>

This function accepts three parameters: the file path, the desired chunk size, and a callback function that will be called for each chunk read.

Usage of Custom Function

The file_get_contents_chunked() function can be used as follows:

<code class="php">$success = file_get_contents_chunked("my/large/file", 4096, function ($chunk, &$handle, $iteration) {
    /* Process the chunk here... */
});</code>

Regex Considerations

Performing multiple regex operations on a large chunk of data is inefficient. Consider using native string manipulation functions like strpos(), substr(), trim(), and explode().

Example Cleanup

Instead of:

<code class="php">$newData = str_replace("^M", "", $myData);</code>

Use:

<code class="php">$pattern = '/\r\n/';
$replacement = '';
$newData = preg_replace($pattern, $replacement, $myData);</code>

By utilizing the aforementioned techniques, it is possible to effectively process large files without encountering memory exhaustion errors.

The above is the detailed content of How to Avoid Memory Exhaustion Errors when Using file_get_contents() with Large Files?. For more information, please follow other related articles on the PHP Chinese website!