Having trouble limiting download size of PHP's

问题:

I'm using PHP's cURL function to read profiles from steampowered.com. The data retrieved is XML, and only the first roughly 1000 bytes are needed.

The method I'm using is to add a Range header, which I read on a Stack Overflow answer (curl: How to limit size of GET?). Another method I tried was using the curlopt_range but that didn't work either.

<?
$curl_url = 'http://steamcommunity.com/id/edgen?xml=1';
$curl_handle = curl_init($curl_url);

curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt ($curl_handle, CURLOPT_HTTPHEADER, array("Range: bytes=0-1000"));

$data_string = curl_exec($curl_handle);

echo $data_string;

curl_close($curl_handle);
?>

When this code is executed, it returns the whole thing.

I'm using PHP Version 5.2.14.

回答1:

The server does not honor the Range header. The best you can do is to cancel the connection as soon as you receive more data than you want. Example:

<?php
$curl_url = 'http://steamcommunity.com/id/edgen?xml=1';
$curl_handle = curl_init($curl_url);

$data_string = "";
function write_function($handle, $data) {
    global $data_string;
    $data_string .= $data;
    if (strlen($data_string) > 1000) {
        return 0;
    }
    else
        return strlen($data);
}

curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt ($curl_handle, CURLOPT_WRITEFUNCTION, 'write_function');

curl_exec($curl_handle);

echo $data_string;

Perhaps more cleanly, you could use the http wrapper (this would also use curl if it was compiled with --with-curlwrappers). Basically you would call fread in a loop and then fclose on the stream when you got more data than you wanted. You could also use a transport stream (open the stream with fsockopen, instead of fopen and send the headers manually) if allow_url_fopen is disabled.