What is a good buffer size for socket programming?

Question

We are using .Net and sockets. The server is using the Socket.Sender(bytes[]) method so it just sends the entire payload. On the other side we are clients consuming the data. Socket.Receive(buffer[]). In all the examples from Microsoft (and others) they seem to stick with a buffer size of 8192. We have used this size but every now and then we are sending data down to the clients that exceeds this buffer size.

Is there a way of determining how much data the server's sent method sent us? What is the best buffer size?

Answer 1

It depends upon your protocol. If you are expecting messages in excess of 8192 bytes, then you should increase your buffer size accordingly. But keep in mind this buffer size is only for one call to Receive. If you really want/need to, you can loop over Receive multiple times and copy the received data into an arbitrarily large data structure or buffer.

Also keep in mind it is good practice to call Receive repeatedly until you have verified that you have read all of the data for a given message; even if a single message is less than your buffer size, it still might not all be retrieved by a single Receive call.

Answer 2

Jon Skeet's answer unfortunately leaves a big part of the picture out - the send buffer size, and the bandwidth-delay product of the pipe you're writing to.

If you are trying to send data over a large pipe using a single socket, and you want TCP to fill that pipe, you need to use a send buffer size that is equivalent to the bandwidth-delay product of the pipe. Otherwise, TCP will not fill the pipe because it will not leave enough 'bytes in flight' at all times.

Consider a connection that has a speed of 1 gigabit, and has a one-way latency of 10 milliseconds, on average. The round-trip-time (aka, the amount of time that elapses between your socket sending a packet and the time it receives the ack for that packet and thus knows to send more data) is usually twice the latency.

So if you have a 1 gigabit connection, and a RTT of 20 milliseconds, then that pipe has 1 gigabit/sec * 20 milliseconds == 2.5 megabytes of data in flight at all time if it's being utilized completely.

If your TCP send buffer is anything less than 2.5 megabytes, then that one socket will never fully utilize the pipe - you'll never get a gigabit/sec of performance out of your socket.

If your application uses many sockets, then the aggregate size of all TCP send buffers must be 2.5 MB in order to fully utilize this hypothetical 1 gigabit/20 ms RTT pipe. For instance, if you use 8192-byte buffers, you need 306 simultaneous TCP sockets to fill that pipe.

Answer 3

Even if you're sending more data than that, it may well not be available in one call to Receive.

You can't determine how much data the server has sent - it's a stream of data, and you're just reading chunks at a time. You may read part of what the server sent in one Send call, or you may read the data from two Send calls in one Receive call. 8K is a reasonable buffer size - not so big that you'll waste a lot of memory, and not so small that you'll have to use loads of wasted Receive calls. 4K or 16K would quite possibly be fine too... I personally wouldn't start going above 16K for network buffers - I suspect you'd rarely fill them.

You could experiment by trying to use a very large buffer and log how many bytes were received in each call - that would give you some idea of how much is generally available - but it wouldn't really show the effect of using a smaller buffer. What concerns do you have over using an 8K buffer? If it's performance, do you have any evidence that this aspect of your code is a performance bottleneck?

Answer 4

8192 would be ideal. If you have data which exceed this size it wouldbe better of you to send the data in constant length packets.

The size of data that is sent by server can be checked using the recv function in WINSOCK which has a parameter that gives length of buffer.

Answer 5

Not really Microsoft related, but I am just experimenting with a C++ threaded echo server using a TCP port (not Unix domain socket) to see the throughput. Timing a 4M input with various buffer sizes gave the following results:

1024 - real 0m0,102s; user  0m0,018s; sys   0m0,009s
2048 - real 0m0,112s; user  0m0,017s; sys   0m0,009s
8192 - real 0m0,163s; user  0m0,017s; sys   0m0,007s
 256 - real 0m0,101s; user  0m0,019s; sys   0m0,008s
  16 - real 0m0,144s; user  0m0,016s; sys   0m0,010s

Seems reading in 1024 byte chunks reduces the TCP overhead while the processing time (just echoing the input back) was not affected by the buffer size. 8192 bytes seem high and really low values (like 16) are not good either.