我需要一个C＃控制台应用程序下载一个大文件（2 GB）通过HTTP。 问题是，经过约1.2 GB，应用程序运行的内存。
下面是我使用的代码：
WebClient request = new WebClient();
request.Credentials = new NetworkCredential(username, password);
byte[] fileData = request.DownloadData(baseURL + fName);
正如你所看到的......我直接把文件读入内存中。 我敢肯定，我可以解决这个问题，如果我是读数据从HTTP回块，并将其写入到磁盘上的文件。
我怎么能这样做？
如果您使用WebClient.DownloadFile ，你可以直接将其保存到一个文件中。
The WebClient class is the one for simplified scenarios. Once you get past simple scenarios (and you have), you'll have to fall back a bit and use WebRequest.
With WebRequest, you'll have access to the response stream, and you'll be able to loop over it, reading a bit and writing a bit, until you're done.
Example:
public void MyDownloadFile(Uri url, string outputFilePath)
{
const int BUFFER_SIZE = 16 * 1024;
using (var outputFileStream = File.Create(outputFilePath, BUFFER_SIZE))
{
var req = WebRequest.Create(url);
using (var response = req.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
var buffer = new byte[BUFFER_SIZE];
int bytesRead;
do
{
bytesRead = responseStream.Read(buffer, 0, BUFFER_SIZE);
outputFileStream.Write(buffer, 0, bytesRead);
} while (bytesRead > 0);
}
}
}
}
Note that if WebClient.DownloadFile works, then I'd call it the best solution. I wrote the above before the "DownloadFile" answer was posted. I also wrote it way too early in the morning, so a grain of salt (and testing) may be required.
你需要得到响应流，然后在读取的块，每个块写入文件，以允许内存被重用。
当你写它，整个响应，所有2GB，需要在内存中。 即使是在64位系统，将达到2GB限制单个.NET对象。
更新：更容易的选择。 获取WebClient为你做的工作：其DownloadFile方法将直接把数据放到一个文件中。
WebClient.OpenRead返回一个Stream，只需使用浏览遍历所有的内容，所以数据不会在内存中缓冲，但可以写成块到文件中。
我会用像这样
该连接可以被打断，所以最好是下载文件小块。
阿卡流可以使用多线程帮助小块下载文件从System.IO.Stream。 https://getakka.net/articles/intro/what-is-akka.html
下载方法将字节附加到文件开始长fileStart。 如果该文件不存在，fileStart值必须是0。
using Akka.Actor;
using Akka.IO;
using Akka.Streams;
using Akka.Streams.Dsl;
using Akka.Streams.IO;
private static Sink<ByteString, Task<IOResult>> FileSink(string filename)
{
return Flow.Create<ByteString>()
.ToMaterialized(FileIO.ToFile(new FileInfo(filename), FileMode.Append), Keep.Right);
}
private async Task Download(string path, Uri uri, long fileStart)
{
using (var system = ActorSystem.Create("system"))
using (var materializer = system.Materializer())
{
HttpWebRequest request = WebRequest.Create(uri) as HttpWebRequest;
request.AddRange(fileStart);
using (WebResponse response = request.GetResponse())
{
Stream stream = response.GetResponseStream();
await StreamConverters.FromInputStream(() => stream, chunkSize: 1024)
.RunWith(FileSink(path), materializer);
}
}
}
