Writing file from HttpWebRequest periodically vs.

2019-07-20 18:37发布

问题:

Right now I am using this code to download files (with a Range header). Most of the files are large, and it is running 99% of CPU currently as the file downloads. Is there any way that the file can be written periodically so that it does not remain in RAM constantly?

private byte[] GetWebPageContent(string url, long start, long finish)
{
    byte[] result = new byte[finish];
    HttpWebRequest request;

    request = WebRequest.Create(url) as HttpWebRequest;

    //request.Headers.Add("Range", "bytes=" + start + "-" + finish);

    request.AddRange((int)start, (int)finish);

    using (WebResponse response = request.GetResponse())
    {
        return ReadFully(response.GetResponseStream());
    }    
}

public static byte[] ReadFully(Stream stream)
{
    byte[] buffer = new byte[32768];
    using (MemoryStream ms = new MemoryStream())
    {
        while (true)
        {
            int read = stream.Read(buffer, 0, buffer.Length);
            if (read <= 0)
                return ms.ToArray();
            ms.Write(buffer, 0, read);
        }
    }
}

回答1:

Instead of writing the data to a MemoryStream (which stores the data in memory), write the data to a FileStream (which stores the data in a file on disk).

byte[] buffer = new byte[32768];
using (FileStream fileStream = File.Create(path))
{
    while (true)
    {
        int read = stream.Read(buffer, 0, buffer.Length);
        if (read <= 0)
            break;
        fileStream.Write(buffer, 0, read);
    }
}

Using .NET 4.0:

using (FileStream fileStream = File.Create(path))
{
    stream.CopyTo(fileStream);
}