Downloading a large file using curl

问题:

I need to download remote file using curl.

Here\'s the sample code I have:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

$st = curl_exec($ch);
$fd = fopen($tmp_name, \'w\');
fwrite($fd, $st);
fclose($fd);

curl_close($ch);

But it can\'t handle big files, because it reads to memory first.

Is it possible to stream the file directly to disk?

回答1:

<?php
set_time_limit(0);
//This is the file where we save the    information
$fp = fopen (dirname(__FILE__) . \'/localfile.tmp\', \'w+\');
//Here is the file we are downloading, replace spaces with %20
$ch = curl_init(str_replace(\" \",\"%20\",$url));
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
// write curl response to file
curl_setopt($ch, CURLOPT_FILE, $fp); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
// get curl response
curl_exec($ch); 
curl_close($ch);
fclose($fp);
?>

回答2:

I use this handy function:

By downloading it with a 4094 byte step it will not full your memory

function download($file_source, $file_target) {
    $rh = fopen($file_source, \'rb\');
    $wh = fopen($file_target, \'w+b\');
    if (!$rh || !$wh) {
        return false;
    }

    while (!feof($rh)) {
        if (fwrite($wh, fread($rh, 4096)) === FALSE) {
            return false;
        }
        echo \' \';
        flush();
    }

    fclose($rh);
    fclose($wh);

    return true;
}

Usage:

     $result = download(\'http://url\',\'path/local/file\');

You can then check if everything is ok with:

     if (!$result)
         throw new Exception(\'Download error...\');

回答3:

Find below code if you want to download the contents of the specified URL also want to saves it to a file.

<?php
$ch = curl_init();
/**
* Set the URL of the page or file to download.
*/
curl_setopt($ch, CURLOPT_URL,\'http://news.google.com/news?hl=en&topic=t&output=rss\');

$fp = fopen(\'rss.xml\', \'w+\');
/**
* Ask cURL to write the contents to a file
*/
curl_setopt($ch, CURLOPT_FILE, $fp);

curl_exec ($ch);

curl_close ($ch);
fclose($fp);
?>

If you want to downloads file from the FTP server you can use php FTP extension. Please find below code:

<?php
$SERVER_ADDRESS=\"\";
$SERVER_USERNAME=\"\";
$SERVER_PASSWORD=\"\";
$conn_id = ftp_connect($SERVER_ADDRESS);

// login with username and password
$login_result = ftp_login($conn_id, $SERVER_USERNAME, $SERVER_PASSWORD);

$server_file=\"test.pdf\" //FTP server file path 
$local_file = \"new.pdf\"; //Local server file path 

##----- DOWNLOAD $SERVER_FILE AND SAVE TO $LOCAL_FILE--------##
if (ftp_get($conn_id, $local_file, $server_file, FTP_BINARY)) {
    echo \"Successfully written to $local_file\\n\";
} else {
    echo \"There was a problem\\n\";
}

ftp_close($conn_id);
?>

回答4:

when curl is used to download a large file then CURLOPT_TIMEOUT is the main option you have to set for.

CURLOPT_RETURNTRANSFER has to be true in case you are getting file like pdf/csv/image etc.

You may find the further detail over here(correct url) Curl Doc

From that page:

curl_setopt($request, CURLOPT_TIMEOUT, 300); //set timeout to 5 mins

curl_setopt($request, CURLOPT_RETURNTRANSFER, true); // true to get the output as string otherwise false

回答5:

You can use this function, which creates a tempfile in the filesystem and returns the path to the downloaded file if everything worked fine:

function getFileContents($url)
{
    // Workaround: Save temp file
    $img = tempnam(sys_get_temp_dir(), \'pdf-\');
    $img .= \'.\' . pathinfo($url, PATHINFO_EXTENSION);

    $fp = fopen($img, \'w+\');

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FILE, $fp);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

    $result = curl_exec($ch);
    curl_close($ch);

    fclose($fp);

    return $result ? $img : false;
}