Nagle-Like Problem

Question

so I have this real-time game, with a C++ sever with disabled nagle using SFML library , and client using asyncsocket, also disables nagle. I'm sending 30 packets every 1 second. There is no problem sending from the client to the server, but when sending from the server to the clients, some of the packets are migrating. For example, if I'm sending "a" and "b" in completly different packets, the client reads it as "ab". It's happens just once a time, but it makes a real problem in the game.

So what should I do? How can I solve that? Maybe it's something in the server? Maybe OS settings?

To be clear: I AM NOT using nagle but I still have this problem. I disabled in both client and server.

Answer 1

You have to disable Nagle in both peers. You might want to find a different protocol that's record-based such as SCTP.

EDIT2

Since you are asking for a protocol here's how I would do it:

• Define a header for the message. Let's say I would pick a 32 bits header.

  Header:
      MSG Length: 16b
      Version: 8b
      Type: 8b
  

• Then the real message comes in, having MSG Length bytes.

So now that I have a format, how would I handle things ?

Server

When I write a message, I prepend the control information (the length is the most important, really) and send the whole thing. Having NODELAY enabled or not makes no difference.

Client

I continuously receive stuff from the server, right ? So I have to do some sort of read.

• Read bytes from the server. Any amount can arrive. Keep reading until you've got at least 4 bytes.
• Once you have these 4 bytes, interpret them as the header and extract the MSG Length
• Keep reading until you've got at least MSG Length bytes. Now you've got your message and can process it

This works regardless of TCP options (such as NODELAY), MTU restrictions, etc.

Answer 2

For example, if I'm sending "a" and "b" in completly different packets, the client reads it as "ab". It's happens just once a time, but it makes a real problem in the game.

I think you have lost sight of the fundamental nature of TCP: it is a stream protocol, not a packet protocol. TCP neither respects nor preserves the sender's data boundaries. To put it another way, TCP is free to combine (or split!) the "packets" you send, and present them on the receiver any way its wants. The only restriction that TCP honors is this: if a byte is delivered, it will be delivered in the same order in which it was sent. (And nothing about Nagle changes this.)

So, if you invoke send (or write) on the server twice, sending these six bytes:

"packet" 1: A B C
"packet" 2: D E F

Your client side might recv (or read) any of these sequences of bytes:

ABC / DEF
ABCDEF
AB / CD / EF

If your application requires knowledge of the boundaries between the sender's writes, then it is your responsibility to preserve and transmit that information.

As others have said, there are many ways to go about that. You could, for example, send a newline after each quantum of information. This is (in part) how HTTP, FTP, and SMTP work.

You could send the packet length along with the data. The generalized form for this is called TLV, for "Type, Length, Value". Send a fixed-length type field, a fixed-length length field, and then an arbitrary-length value. This way you know when you have read the entire value and are ready for the next TLV.

You could arrange that every packet you send is identical in length.

I suppose there are other solutions, and I suppose that you can think of them on your own. But first you have to realize this: TCP can and will merge or break your application packets. You can rely upon the order of the bytes' delivery, but nothing else.