I have created a client/server program, the client starts an instance of Writer class and the server starts an instance of Reader class. Writer will then write a DATA_SIZE bytes of data asynchronously to the Reader every USLEEP mili seconds.
Every successive async_write request by the Writer is done only if the "on write" handler from the previous request had been called.
The problem is, If the Writer (client) is writing more data into the socket than the Reader (server) is capable of receiving this seems to be the behaviour:
Writer will start writing into (I think) system buffer and even though the data had not yet been received by the Reader it will be calling the "on write" handler without an error.
When the buffer is full, boost::asio won't fire the "on write" handler anymore, untill the buffer gets smaller.
In the meanwhile, the Reader is still receiving small chunks of data.
The fact that the Reader keeps receiving bytes after I close the Writer program seems to prove this theory correct.
What I need to achieve is to prevent this buffering because the data need to be "real time" (as much as possible).
I'm guessing I need to use some combination of the socket options that asio offers, like the no_delay or send_buffer_size, but I'm just guessing here as I haven't had success experimenting with these.
I think that the first solution that one can think of is to use UDP instead of TCP. This will be the case as I'll need to switch to UDP for other reasons as well in the near future, but I would first like to find out how to do it with TCP just for the sake of having it straight in my head in case I'll have a similar problem some other day in the future.
NOTE1: Before I started experimenting with asynchronous operations in asio library I had implemented this same scenario using threads, locks and asio::sockets and did not experience such buffering at that time. I had to switch to the asynchronous API because asio does not seem to allow timed interruptions of synchronous calls.
NOTE2: Here is a working example that demonstrates the problem: http://pastie.org/3122025
EDIT: I've done one more test, in my NOTE1 I mentioned that when I was using asio::iosockets I did not experience this buffering. So I wanted to be sure and created this test: http://pastie.org/3125452 It turns out that the buffering is there event with asio::iosockets, so there must have been something else that caused it to go smoothly, possibly lower FPS.