Realtime streaming with QAudioOutput (qt)

Question

I want to play real-time sounds responding with no appreciable lag to user interaction.

To have low latency I have to send small chunks of pcm data. What I am doing:

    QAudioFormat format;
    format.setSampleRate(22050);
    format.setChannelCount(1);
    format.setSampleSize(16);
    format.setCodec("audio/pcm");
    format.setByteOrder(QAudioFormat::LittleEndian);
    format.setSampleType(QAudioFormat::SignedInt);

    QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
    if (!info.isFormatSupported(format)) {
        qWarning()<<"raw audio format not supported by backend, cannot play audio.";
        return;
    }

    qAudioOutput = new QAudioOutput(format, NULL);

    qAudioDevice=qAudioOutput->start();

and later

void Enqueue(TYPESAMPLEPCM *data,int countBytes){
    while(qAudioOutput->bytesFree()<countBytes){
          Sleep(1);
    }
    qAudioDevice->write((char *)data,countBytes);
}

The chunks of data are 256 bytes (128 samples that would give "granularity" of around 6 milliseconds.

Enqueue is called from a loop in a thread with high priority that provides the data chunks. There's no latency there as the speed it calls Enqueue is by far faster than rendering the audio data.

But it looks to me there's a buffer underrun situation because the sound plays but with kind of a "crackling" regular noise.

If I raise the chunk size to 256 samples the problem almost disappears. Only some crackling at the beginning (?)

The platform is Windows and Qt 5.3.

Is that the right procedure or I am missing something?

Answer 1

The issue is about

void Enqueue(TYPESAMPLEPCM *data,int countBytes){
    while(qAudioOutput->bytesFree()<countBytes){
          Sleep(1);
    }
    qAudioDevice->write((char *)data,countBytes);
}

being a little naive.

First of all, Sleep(1);. You are on windows. The problem is that windows is not a realtime os, and is expected to have a time resolution around 10 - 15 ms. Which means when there is no place for incoming audio you are going to sleep lot more than you expect.

Second. Do you really need to sleep when audio output cannot consume the amount of data which was provided? What you really want to is to provide some audio after the audio output has consumed some. In concrete terms, it means :

1. Setting the QAudioOutput notify interval, ie the period at which the system will consume the audio data and tell you about it.
2. Getting notified about QAudioOutput consuming some data. Which is in a slot connected to QAudioOutput::notify()
3. Buffering data chunks which come from your high priority thread when audio output is full.

This give :

QByteArray samplebuffer;

//init code
{
     qAudioOutput = new QAudioOutput(format, NULL);
     ...
     qAudioOutput->setNotifyInterval(128); //play with this number
     connect(qAudioOutput, SIGNAL(notify), someobject, SLOT(OnAudioNotify));
     ...
     qAudioDevice=qAudioOutput->start();
}

void EnqueueLock(TYPESAMPLEPCM *data,int countBytes)
{
    //lock mutex
    samplebuffer.append((char *)data,countBytes);
    tryWritingSomeSampleData();
    //unlock mutex
}

//slot
void SomeClass::OnAudioNotify()
{
   //lock mutex
   tryWritingSomeSampleData()
   //unlock mutex
}

void SomeClass::tryWritingSomeSampleData()
{
    int towritedevice = min(qAudioOutput->bytesFree(), samplebuffer.size());
    if(towritedevice > 0)
    {
        qAudioDevice->write(samplebuffer.data(),towritedevice);
        samplebuffer.remove(0,towritedevice); //pop front what is written
    }
}

As you see you need to protect samplebuffer from concurrent access. Provide adequate mutex.