Why can't my code read the number of bytes ava

Question

I have the following snippet of code in the body of a loop responsible for reading data from a QTcpSocket (nntp is a pointer to a QTcpSocket).

std::vector<char> buffer;
int bytesAvailable = nntp->bytesAvailable();
qDebug() << "bytesAvailable: "<<bytesAvailable;
if(bytesAvailable <= 0) break;
buffer.resize(bytesAvailable);
bytesRead = nntp->read(&buffer[0], bytesAvailable);
qDebug() << (nntp->state() == QAbstractSocket::ConnectedState);
qDebug() << "bytesRead: "<<bytesRead;

Intermittently, this outputs something akin to the following:

bytesAvailable:  24 
true 
bytesRead:  0 

And my code from there mis-behaves. This seems very strange to me and suggests that I completely misunderstand how QTcpSockets work. Surely if bytesAvailable > 0 then a subsequent read will mean that bytesAvailable bytes can be read into a buffer in which case bytesRead should == bytesAvailable. Or am I missing something? My current suspicion is that it could be a memory corruption of some sort..

EDIT: Throwing in some nntp.errorString() messages reports that during this failure, the "Network operation timed out". I need to investigate what this means...(ideas?)

EDIT 2: It seems that "Network operation timed out" just means that the read timed out. When the code works as it should (i.e. intermittently), I still get this 'error'.

EDIT 3: The full algorithmic context for the above snippet of code can be found at this pastebin link.

EDIT 4: A slightly different version of the function in EDIT 3 but still with the same problems nonetheless is at this newer pastebin link

Answer 1

What probably happens if that bytesAvailable() reports the size of the data waiting in the QIODevice internal buffer plus the size reported by the OS (e.g. in Linux that would be obtained by ioctl(fd,FIONREAD,&bytesCount)).

That makes sense by itself, but to be able to read those bytes from a QTcpSocket without the event loop, waitForReadyRead() must be called in your own loop. Otherwise the data from the kernel buffer doesn't make it into the QIODevice's buffer.

If your code is not allowed to block when there's nothing to read on the socket, and you don't want to reshape it into an event-driven structure, use a small value for waitForReadyRead's timeout so that in practice it will behave as if not blocking.

Also don't forget to call waitForBytesWritten() as well if you're also writing to the socket with no event loop.

Answer 2

I discovered the issue: the QTcpSocket I had been trying to read from belonged to a different thread. Even though bytes were available according to the QIODevice's buffer, they could not be read because of this.

Therefore:

* Always ensure that the socket you want to read from belongs to the thread on which you want to do the reading *

To help with this, and as Taylor suggested above, one can take advantage of QTcpSocket's signals and slots infrastructure (preferred). This has the appropriate threading infrastructure in place and in theory should make things a lot simpler.

OR

Be super-cautious and

(a) utilize ones own QThread, moving the object that contains the QTcpSocket to this QThread

then,

(b) use the waitForReadyRead of the QTcpSocket in the blocking read loop of this other thread.

This latter approach is more difficult since if one wants to retain the QTcpSocket for other threads in future reads and writes, then after it has been moved to another thread and processed by that other thread, it must be moved back to the main thread before it can ever be moved a further thread. -- gives me a headache just trying to word that! Of course one might chose to keep the same QTcpSocket worker thread throughout so that it only has to be moved once, or simple creating a new QTcpSocket each time a QTcpSocket is required, moving it to its own QThread and then immediately deleting it when its finished with.

Basically, go for the first signals and slots method if you can.

Answer 3

I haven't encountered this precise problem (possibly because I'm using a different version of Qt), but something I would recommend trying is to switch from a loop to an event-driven approach. If your loop is running in the main thread, then there is no way for objects to deliver queued signals (as the QTcpSocket class may be doing internally), which may be why you're seeing "Network operation timed out"?

So connect some of the fundamental QTcpSocket signals:

connect(nntp, SIGNAL(disconnected()),
        this, SLOT(onDisconnected()));
connect(nntp, SIGNAL(readyRead()),
        this, SLOT(onReadyRead()));
connect(nntp, SIGNAL(error(QAbstractSocket::SocketError)),
        this, SLOT(onSocketError(QAbstractSocket::SocketError)));

And then put your existing "read" code in onReadyRead.

Maybe your problem is unrelated, but I recently was seeing similar problems in code a colleague wrote, and this is how I fixed it.