C# NetworkStream.DataAvailable seems to be unrelia

2019-05-21 12:50发布

问题:

I have an app that uses a TCP socket to exchange byte arrays which in most cases contain JSON string data. What I'm experiencing is that, for larger messages and less than ideal network conditions, use of NetworkStream.DataAvailable does NOT seem to be a reliable way to detect an end of message. It seems that in some cases DataAvailable is set to false even when only part of the message has been transmitted by peer (which is using TcpClient.GetStream().Write(data, 0, data.Length). This results in incomplete data being passed back to the app, which in the case of a JSON message, means deserialization fails.

I've tried two implementations which exhibit the same issue:

Implementation 1:

byte[] Data;
byte[] buffer = new byte[2048];
using (MemoryStream ms = new MemoryStream())
{
    int read;

    while ((read = ClientStream.Read(buffer, 0, buffer.Length)) > 0)
    {
        ms.Write(buffer, 0, read);
        BytesRead += read;

        if (!ClientStream.DataAvailable) break;
    }

    Data = ms.ToArray();
}

Implementation 2:

byte[] Data;
using (MemoryStream ms = new MemoryStream())
{
    byte[] buffer = new byte[2048];

    do
    {
        int read = ClientStream.Read(buffer, 0, buffer.Length);
        if (read > 0)
        {
            ms.Write(buffer, 0, read);
            BytesRead += read;
        }
    }
    while (ClientStream.DataAvailable);

    Data = ms.ToArray();
}

It seems one solution that works really well but is completely sub-optimal is to add a Thread.Sleep in case NetworkStream.DataAvailable is false (while inside the loop) to allow data to be delivered. However this severely limits overall IOPS which I would like to avoid, i.e.

Implementation 3 (works, but suboptimal)

byte[] Data;
using (MemoryStream ms = new MemoryStream())
{
    byte[] buffer = new byte[2048];

    do
    {
        int read = ClientStream.Read(buffer, 0, buffer.Length);
        if (read > 0)
        {
            ms.Write(buffer, 0, read);
            BytesRead += read;
        }

        if (!ClientStream.DataAvailable) System.Threading.Thread.Sleep(250);
    }
    while (ClientStream.DataAvailable);

    Data = ms.ToArray();
}

I'd really like to find a way to remain in the loop until all of the data is delivered. As I mentioned, I'm doing a simple write operation on the client from zero to data length, so I'm not thinking there is an issue there.

Has anyone had any experience like this before and a recommendation?

回答1:

It seems .DataAvailable is indeed reliable and that, since the data arrives over the network potentially at a rate slower than data is read from the stream, .DataAvailable can flip-flop between the start and end of what my application thinks is a message.

I'm answering and closing this as I believe the only solutions to this are:

1) add an over-arching receive timeout value, and perform a thread.sleep in the read loop, and expiring the operation once the receive timeout is reached

2) implement some mechanism of indicating the data payload size - either explicitly or by creating a system of metadata headers - to indicate how much data should be read, and exiting after that much data has been read or the operation has timed out

These two are the best I could come up with and seem to be validated by the likes of other TCP-based protocols like HTTP and generally any other RPC out there.

Hopefully this saves someone some time.