WebAPI StreamContent vs PushStreamContent

Question

I'm implementing a MVC4 + WebAPI version of the BluImp jQuery File Upload all works well with my initial attempt but Im trying to ensure the best use of memory whilst downloading very large files (~2GB).

I've read Filip Woj's article on PushStreamContent and implemented it as best I can (removing the async parts - perhaps this is the problem?). When Im running tests and watching TaskManager Im not seeing much difference memory usage wise and Im trying to understand the difference between how the responses are handled.

Here's my StreamContent version:

private HttpResponseMessage DownloadContentNonChunked()
{
    var filename = HttpContext.Current.Request["f"];
    var filePath = _storageRoot + filename;
    if (File.Exists(filePath))
    {
        HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
        response.Content = new StreamContent(new FileStream(filePath, FileMode.Open, FileAccess.Read));
        response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
        response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
        {
            FileName = filename
        };
        return response;
    }
    return ControllerContext.Request.CreateErrorResponse(HttpStatusCode.NotFound, "");
}

And here's my PushStreamContent version:

public class FileDownloadStream
{
    private readonly string _filename;

    public FileDownloadStream(string filePath)
    {
        _filename = filePath;
    }

    public void WriteToStream(Stream outputStream, HttpContent content, TransportContext context)
    {
        try
        {
            var buffer = new byte[4096];

            using (var video = File.Open(_filename, FileMode.Open, FileAccess.Read))
            {
                var length = (int)video.Length;
                var bytesRead = 1;

                while (length > 0 && bytesRead > 0)
                {
                    bytesRead = video.Read(buffer, 0, Math.Min(length, buffer.Length));
                    outputStream.Write(buffer, 0, bytesRead);
                    length -= bytesRead;
                }
            }
        }
        catch (HttpException ex)
        {
            return;
        }
        finally
        {
            outputStream.Close();
        }
    }
}

private HttpResponseMessage DownloadContentChunked()
{
    var filename = HttpContext.Current.Request["f"];
    var filePath = _storageRoot + filename;
    if (File.Exists(filePath))
    {
        var fileDownload = new FileDownloadStream(filePath);
        var response = Request.CreateResponse();
        response.Content = new PushStreamContent(fileDownload.WriteToStream, new MediaTypeHeaderValue("application/octet-stream"));
        response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
        {
            FileName = filename
        };
        return response;
    }
    return ControllerContext.Request.CreateErrorResponse(HttpStatusCode.NotFound, "");
}

My question is why am I not seeing much difference in memory usage between the two approaches? Additionally Ive downloaded the PDB for the StreamContent type and can see references to buffer sizes and such forth (see below) so I'd like to know exactly what PushStreamContent is doing above and beyond StreamContent. Ive check the Type info on MSDN but the article were a little light on explanation!

namespace System.Net.Http
{
  /// <summary>
  /// Provides HTTP content based on a stream.
  /// </summary>
  [__DynamicallyInvokable]
  public class StreamContent : HttpContent
  {
    private Stream content;
    private int bufferSize;
    private bool contentConsumed;
    private long start;
    private const int defaultBufferSize = 4096;

    /// <summary>
    /// Creates a new instance of the <see cref="T:System.Net.Http.StreamContent"/> class.
    /// </summary>
    /// <param name="content">The content used to initialize the <see cref="T:System.Net.Http.StreamContent"/>.</param>
    [__DynamicallyInvokable]
    [TargetedPatchingOptOut("Performance critical to inline this type of method across NGen image boundaries")]
    public StreamContent(Stream content)
      : this(content, 4096)
    {
    }

Answer 1

Regarding the memory usage of these both approaches, for StreamContent and PushStreamContent, Web API doesn't buffer the responses. Following snapshot of code is from WebHostBufferPolicySelector. Source code here.

    /// <summary>
    /// Determines whether the host should buffer the <see cref="HttpResponseMessage"/> entity body.
    /// </summary>
    /// <param name="response">The <see cref="HttpResponseMessage"/>response for which to determine
    /// whether host output buffering should be used for the response entity body.</param>
    /// <returns><c>true</c> if buffering should be used; otherwise a streamed response should be used.</returns>
    public virtual bool UseBufferedOutputStream(HttpResponseMessage response)
    {
        if (response == null)
        {
            throw Error.ArgumentNull("response");
        }

        // Any HttpContent that knows its length is presumably already buffered internally.
        HttpContent content = response.Content;
        if (content != null)
        {
            long? contentLength = content.Headers.ContentLength;
            if (contentLength.HasValue && contentLength.Value >= 0)
            {
                return false;
            }

            // Content length is null or -1 (meaning not known).  
            // Buffer any HttpContent except StreamContent and PushStreamContent
            return !(content is StreamContent || content is PushStreamContent);
        }

        return false;
    }

Also PushStreamContent is for scenarios where you need to 'push' data to the stream, where as StreamContent 'pulls' data from the stream. So, for your current scenario of downloading files, using StreamContent should be fine.

Examples below:

// Here when the response is being written out the data is pulled from the file to the destination(network) stream
response.Content = new StreamContent(File.OpenRead(filePath));

// Here we create a push stream content so that we can use XDocument.Save to push data to the destination(network) stream
XDocument xDoc = XDocument.Load("Sample.xml", LoadOptions.None);
PushStreamContent xDocContent = new PushStreamContent(
(stream, content, context) =>
{
     // After save we close the stream to signal that we are done writing.
     xDoc.Save(stream);
     stream.Close();
},
"application/xml");