PHP to read & parse big sized file? [closed]

Question

I am about to read a text file sized 200Mb and then edit something inside and then to save it back. But i'm having errors. So:

• Which exact settings should be modified in php?

Also what file reading method is the best for opening & parsing the Big Sized files? I mean:

• fread ?
• file_get_contents ?

Answer 1

I had to do something similar, reading 1GB file. I wanted to stay whithin PHP, so finally I used fread to read parts of the file, bit by bit:

while (!feof($source_file)) {
    $buffer = fread($source_file, 1024);  // use a buffer of 1024 bytes
    $buffer = str_replace($old,$new,$buffer);
    fwrite($target_file, $buffer);
}

This way only a small part of the file is kept in memory at any given time. I've checked the efficiency and it's good, about half minute for the whole file.

A small note- if the replaced string is in at the end of the buffer it might not be replaced. to make sure you've change all of the occurrences run the script again with a small offset:

$buffer = fread($source_file, 512);
fwrite($target_file, $buffer);  
while (!feof($source_file)) {
    $buffer = fread($source_file, 1024);  // use a buffer of 1024 bytes
    $buffer = str_replace($old,$new,$buffer);
    fwrite($target_file, $buffer);
}

Answer 2

PHP isn't designed or intended to do this. You may want to consider using Perl, or changing the text into XML, or putting it into a database.

Doing this the way you're intending means the entire file will be loaded into memory. If you have multiple users doing the same thing, you'll run out of memory real fast.

For XML parsing, look here XMLReader

Answer 3

Mostly the same as an already existing answer, but with file pointers.

$original = fopen("/tmp/inputfile.txt", "r");
$new = fopen("/tmp/outputfile.txt", "w");
if ($original && $new) {
    while (($buffer = fgets($handle)) !== false) {
        //do modification on $buffer (which is a single line)

        fwrite($new, $buffer);
    }

    fclose($original);
    fclose($new);
}

Answer 4

I use the following to complete a similar task:

$file = file_get_contents("/path/to/file");
$lines = explode("\n", $file);

$arr = preg_grep("/search_string/", $lines);

// $arr is now a smaller array of things to match
// do whatever here

// write back to file
file_put_contents("/path/to/file", implode("\n", array_merge($arr, $lines)));