Asynchronous download not reporting via IProgress.

Question

I'm facing a strange problem, which I must admit - do not understand. I've a Task, which is downloading a file from Web asynchronously:

public async Task DownloadFile(Uri requestUri, string fileName, IProgress<int> progress)
{
    HttpWebRequest request = HttpWebRequest.CreateHttp(requestUri);
    request.Method = "GET";
    request.AllowReadStreamBuffering = false;
    using (WebResponse response = await request.GetResponseAsync())
    using (Stream mystr = response.GetResponseStream())
    {
        StorageFolder local = ApplicationData.Current.LocalFolder;
        StorageFile file = await local.CreateFileAsync(fileName);
        using (Stream fileStream = await file.OpenStreamForWriteAsync())
        {
            const int BUFFER_SIZE = 100 * 1024;
            byte[] buf = new byte[BUFFER_SIZE];
            int bytesread = 0;
            // the problem is here below
            while ((bytesread = await mystr.ReadAsync(buf, 0, BUFFER_SIZE)) > 0)
            {
                await fileStream.WriteAsync(buf, 0, bytesread);
                progress.Report(bytesread);
            }
        }
    }
}

The Task is working, but as you can see it should Report its progress.

It turns out that the problem is when the program hits the line bytesread = await mystr.ReadAsync(buf, 0, BUFFER_SIZE) - it doesn't perform any other action on that thread until the whole file has been downloaded (not only BUFFER_SIZE). After the complete download the while loop is fired that many times it is needed and copies stream from memory - which goes quite fast.

The strange thing is that the same code was working without issue on WP8.0, now I try to run it on WP8.1 and I'm facing this problem. Does anybody has an idea what I'm doing wrong?

Answer 1

One thing to keep in mind is that IProgress<T>.Report is itself "asynchronous". Not in the sense that it returns a Task, but in the sense that the progress reporting implementation may not have done its actual progress reporting by the time Report returns. In particular, Progress<T>.Report will marshal a progress report to a captured SynchronizationContext before invoking its delegate or event handler.

Normally, this is exactly what you want: if you create a Progress<T> on the UI thread, then its delegate/event handler is executed on the UI thread, and you don't have to worry about cross-thread marshaling when updating your UI with the progress report.

In this case, I believe what you're seeing is some aggressive caching by WP8.1. Microsoft has really been pushing the caching on WP. If my guess is correct, then the behavior you're seeing is due to ReadAsync completing synchronously (response is already in HTTP cache) as well as WriteAsync completing synchronously (buffered file writes). In this case, none of the awaits will ever actually yield, so the IProgress<T>.Report calls just stack up waiting for their chance to run on the UI thread.

Caching is usually just a "problem" during development. If you do expect your end users to encounter this situation, you can add an await Task.Yield(); into your while loop.