Streaming video from an external service

Question

I am working on a project (server side) where i need to stream data (videos, large files) to clients.

This worked perfect using ByteRangeStreamContent, as i was serving files from disk and could create a seekable stream (FileStream).

    if (Request.Headers.Range != null)
    {
                try
                {
                        HttpResponseMessage partialResponse = Request.CreateResponse(HttpStatusCode.PartialContent);
                        partialResponse.Content = new ByteRangeStreamContent(fs, Request.Headers.Range, mediaType);
                        return partialResponse;
                }
                catch (InvalidByteRangeException invalidByteRangeException)
                {
                        return Request.CreateErrorResponse(invalidByteRangeException);
                }
     }
     else
     {
                    response.Content = new StreamContent(fs);
                    response.Content.Headers.ContentType = mediaType;
                    return response;
     }

But, i moved the file provider from disk to an external service. The service allows me to get chunks of data (Range{0}-{1}).

Of course, it's not possible to download whole file in memory and then use a MemoryStream for ByteRangeStreamContent because of the obvious reasons (too many concurrent downloads will consume all the available memory at some point).

I found this article https://vikingerik.wordpress.com/2014/09/28/progressive-download-support-in-asp-net-web-api/ where the author says:

A change request I got for my library was to support reading only the necessary data and sending that out rather than opening a stream for the full data. I wasn’t sure what this would buy until the user pointed out they are reading their resource data from a WCF stream which does not support seeking and would need to read the whole stream into a MemoryStream in order to allow the library to generate the output.

That limitation still exists in this specific object but there is a workaround. Instead of using a ByteRangeStreamContent, you could instead use a ByteArrayContent object instead. Since the majority of RANGE requests will be for a single start and end byte, you could pull the range from the HttpRequestMessage, retrieve only the bytes you need and send it back out as a byte stream. You’ll also need to add the CONTENT-RANGE header and set the response code to 206 (PartialContent) but this could be a viable alternative (though I haven’t tested it) for users who do not want or can’t easily get a compliant stream object.

So, my question basically is: how can i do that ?

Answer 1

I finally managed to do it.

Here's how:

Custom implementation of a stream:

public class BufferedHTTPStream : Stream
    {
        private readonly Int64 cacheLength = 4000000;
        private const Int32 noDataAvaiable = 0;
        private MemoryStream stream = null;
        private Int64 currentChunkNumber = -1;
        private Int64? length;
        private Boolean isDisposed = false;
        private Func<long, long, Stream> _getStream;
        private Func<long> _getContentLength;

        public BufferedHTTPStream(Func<long, long, Stream> streamFunc, Func<long> lengthFunc)
        {
            _getStream = streamFunc;
            _getContentLength = lengthFunc;
        }

        public override Boolean CanRead
        {
            get
            {
                EnsureNotDisposed();
                return true;
            }
        }

        public override Boolean CanWrite
        {
            get
            {
                EnsureNotDisposed();
                return false;
            }
        }

        public override Boolean CanSeek
        {
            get
            {
                EnsureNotDisposed();
                return true;
            }
        }

        public override Int64 Length
        {
            get
            {
                EnsureNotDisposed();
                if (length == null)
                {
                    length = _getContentLength();
                }
                return length.Value;
            }
        }

        public override Int64 Position
        {
            get
            {
                EnsureNotDisposed();
                Int64 streamPosition = (stream != null) ? stream.Position : 0;
                Int64 position = (currentChunkNumber != -1) ? currentChunkNumber * cacheLength : 0;

                return position + streamPosition;
            }
            set
            {
                EnsureNotDisposed();
                EnsurePositiv(value, "Position");
                Seek(value);
            }
        }

        public override Int64 Seek(Int64 offset, SeekOrigin origin)
        {
            EnsureNotDisposed();
            switch (origin)
            {
                case SeekOrigin.Begin:
                    break;
                case SeekOrigin.Current:
                    offset = Position + offset;
                    break;
                default:
                    offset = Length + offset;
                    break;
            }

            return Seek(offset);
        }

        private Int64 Seek(Int64 offset)
        {
            Int64 chunkNumber = offset / cacheLength;

            if (currentChunkNumber != chunkNumber)
            {
                ReadChunk(chunkNumber);
                currentChunkNumber = chunkNumber;
            }

            offset = offset - currentChunkNumber * cacheLength;

            stream.Seek(offset, SeekOrigin.Begin);

            return Position;
        }

        private void ReadNextChunk()
        {
            currentChunkNumber += 1;
            ReadChunk(currentChunkNumber);
        }

        private void ReadChunk(Int64 chunkNumberToRead)
        {
            Int64 rangeStart = chunkNumberToRead * cacheLength;

            if (rangeStart >= Length) { return; }

            Int64 rangeEnd = rangeStart + cacheLength - 1;
            if (rangeStart + cacheLength > Length)
            {
                rangeEnd = Length - 1;
            }

            if (stream != null) { stream.Close(); }
            stream = new MemoryStream((int)cacheLength);

            var responseStream = _getStream(rangeStart, rangeEnd);

            responseStream.Position = 0;
            responseStream.CopyTo(stream);
            responseStream.Close();

            stream.Position = 0;
        }

        public override void Close()
        {
            EnsureNotDisposed();

            base.Close();
            if (stream != null) { stream.Close(); }
            isDisposed = true;
        }

        public override Int32 Read(Byte[] buffer, Int32 offset, Int32 count)
        {
            EnsureNotDisposed();

            EnsureNotNull(buffer, "buffer");
            EnsurePositiv(offset, "offset");
            EnsurePositiv(count, "count");

            if (buffer.Length - offset < count) { throw new ArgumentException("count"); }

            if (stream == null) { ReadNextChunk(); }

            if (Position >= Length) { return noDataAvaiable; }

            if (Position + count > Length)
            {
                count = (Int32)(Length - Position);
            }

            Int32 bytesRead = stream.Read(buffer, offset, count);
            Int32 totalBytesRead = bytesRead;
            count -= bytesRead;

            while (count > noDataAvaiable)
            {
                ReadNextChunk();
                offset = offset + bytesRead;
                bytesRead = stream.Read(buffer, offset, count);
                count -= bytesRead;
                totalBytesRead = totalBytesRead + bytesRead;
            }

            return totalBytesRead;

        }

        public override void SetLength(Int64 value)
        {
            EnsureNotDisposed();
            throw new NotImplementedException();
        }

        public override void Write(Byte[] buffer, Int32 offset, Int32 count)
        {
            EnsureNotDisposed();
            throw new NotImplementedException();
        }

        public override void Flush()
        {
            EnsureNotDisposed();
        }

        private void EnsureNotNull(Object obj, String name)
        {
            if (obj != null) { return; }
            throw new ArgumentNullException(name);
        }
        private void EnsureNotDisposed()
        {
            if (!isDisposed) { return; }
            throw new ObjectDisposedException("BufferedHTTPStream");
        }
        private void EnsurePositiv(Int32 value, String name)
        {
            if (value > -1) { return; }
            throw new ArgumentOutOfRangeException(name);
        }
        private void EnsurePositiv(Int64 value, String name)
        {
            if (value > -1) { return; }
            throw new ArgumentOutOfRangeException(name);
        }
        private void EnsureNegativ(Int64 value, String name)
        {
            if (value < 0) { return; }
            throw new ArgumentOutOfRangeException(name);
        }
    }

Usage:

    var fs = new BufferedHTTPStream((start, end) => 
    {
        // return stream from external service
    }, () => 
    {
       // return stream length from external service
    });

    HttpResponseMessage partialResponse = Request.CreateResponse(HttpStatusCode.PartialContent);
partialResponse.Content = new ByteRangeStreamContent(fs, Request.Headers.Range, mediaType);
    partialResponse.Content.Headers.ContentDisposition = new     ContentDispositionHeaderValue("attachment")
                            {
                                FileName = fileName
                            };
    return partialResponse;