Is Stream.Copy piped?

Question

Suppose I am writing a tcp proxy code. I am reading from the incoming stream and writing to the output stream. I know that Stream.Copy uses a buffer, but my question is: Does the Stream.Copy method writes to the output stream while fetching the next chunk from the input stream or it a loop like "read chunk from input, write chunk to ouput, read chunk from input, etc" ?

Answer 1

According to Reflector it does not. Such behavior better be documented because it would introduce concurrency. This is never safe to do in general. So the API design to not "pipe" is sound.

So this is not just a question of Stream.Copy being more or less smart. Copying in a concurrent way is not an implementation detail.

Answer 2

Here's the implementation of CopyTo in .NET 4.5:

private void InternalCopyTo(Stream destination, int bufferSize)
{
    int num;
    byte[] buffer = new byte[bufferSize];
    while ((num = this.Read(buffer, 0, buffer.Length)) != 0)
    {
        destination.Write(buffer, 0, num);
    }
}

So as you can see, it reads from the source, then writes to the destination. This could probably be improved ;)

---

EDIT: here's a possible implementation of a piped version:

public static void CopyToPiped(this Stream source, Stream destination, int bufferSize = 0x14000)
{
    byte[] readBuffer = new byte[bufferSize];
    byte[] writeBuffer = new byte[bufferSize];

    int bytesRead = source.Read(readBuffer, 0, bufferSize);
    while (bytesRead > 0)
    {
        Swap(ref readBuffer, ref writeBuffer);
        var iar = destination.BeginWrite(writeBuffer, 0, bytesRead, null, null);
        bytesRead = source.Read(readBuffer, 0, bufferSize);
        destination.EndWrite(iar);
    }
}

static void Swap<T>(ref T x, ref T y)
{
    T tmp = x;
    x = y;
    y = tmp;
}

Basically, it reads a chunk synchronously, starts to copy it to the destination asynchronously, then read the next chunk and waits for the write to complete.

I ran a few performance tests:

• using MemoryStreams, I didn't expect a significant improvement, since it doesn't use IO completion ports (AFAIK); and indeed, the performance is almost identical
• using files on different drives, I expected the piped version to perform better, but it doesn't... it's actually slightly slower (by 5 to 10%)

So it apparently doesn't bring any benefit, which is probably the reason why it isn't implemented this way...

Answer 3

Writing to the output stream is impossible (when using one buffer) while fetching next chunk because fetching the next chunk can overwrite the buffer while its being used for output.
You can say use double buffering but its pretty much the same as using a double sized buffer.

Answer 4

Stream.Copy is synchronous operation. I don't think it is reasonable to expect it to use asynchronous read/write to make simultaneous read and write.

I would expect asynchrounous version (like RandomAccessStream.CopyAsync) to use simultaneous read and write.

Note: using multiple threads during copy would be unwelcome behavior, but using asynchronous read and write to run them at the same time is ok.