Displaying a stream of images with Qt

Question

The current solution looks like this:

//paintlabel.h
class PaintLabel : public QWidget
{
    Q_OBJECT

public:
    explicit PaintLabel(QWidget *parent = 0);

public slots:
    void setImage(char *img_ptr, QSize img_size, int pitch);

protected:
    void paintEvent(QPaintEvent *event) override;

private:
    QImage image;

}

//paintlabel.cpp

PaintLabel::PaintLabel(QWidget *parent) : QWidget(parent)
{
    setAttribute(Qt::WA_NoSystemBackground, true);
}

void PaintLabel::setImageLive(char *img_ptr, QSize img_size, int pitch)
{
    image = QImage((uchar *)img_ptr, img_size.width(), img_size.height(), pitch, QImage::Format_RGB32).scaled(this->size());
    update();  
}

void PaintLabel::paintEvent(QPaintEvent *event)
{
    Q_UNUSED(event);
    QPainter painter;
    painter.begin(this);
    if (!image.isNull()) {
        const QPoint p = QPoint(0,0);
        painter.drawImage(p, image);
    }
    painter.end();
}

I expect 20-40 frames per second. The problem is, that the performance scales really bad with size. With the size around fullHD the painting takes 1-2 ms. But if I resize it to 4k, it gets awfully laggy (16 ms painting). Is there any way to implement the same functionality but with less resource consumption?

Theoretically, if I change the parent class to QOpenGLWidget, the QPainter runs with hardware acceleration. But doing it, it runs even slower.

Answer 1

Firstly one of the limiting factors in your code seems to be the scaled() function. Try taking it out and you should see some speed-up.

Other problem is simply the limitation of performance of QImage. Refer this question for more information. How can QPainter performance be improved?

QPainter will use software rasterizer to draw QImage instances.

refer http://doc.qt.io/qt-5/topics-graphics.html for more information.

You cant go for QPixmap since your entire buffer changes each frame as well.

You can try using the QOpenGLPaintDevice to make the QPainter use hardware acceleration. Another approach you tried is QOpenGLWidget.

QOpenGLWidget - A painter can also be opened on a QOpenGLWidget. This is provided as a convenience, since technically this is no different than using QOpenGLPaintDevice.

refer the same qt link above.

So yes this should technically be faster, BUT since you are resizing the Widget thereby resizing the OpenGL Viewport, the underneath buffers get re-made each time. Hence the slowdown is much more. Do note that QOpenGLWidget handles some functions such as glViewport() by itself. This is not useful when you are trying to optimize performance.

Here are a couple of suggestions

1. I believe the major issue is the handling of QImage. I suggest you setup your own OpenGL Scene, simply with one quad to render a texture, and try writing to the texture each frame. The scaling will be handled on the GPU so it will surely be faster .

2. Alternatively you can use glDrawPixels() to render the image from host memory (RAM) to the frame buffer. Although this will not handle scaling for you. refer to https://www.opengl.org/sdk/docs/man2/xhtml/glDrawPixels.xml for more information.

I am currently using QGLWidget and custom OpenGL texture and draw calls to achieve a fast update (>60 FPS). I have observed this to be about 8x to 10x faster than the QImage / Qpixmap method. Please note that my texture size never surpasses 700x700. Also my Image is always stored on GPU memory and updated by mapped CUDA kernel.

You should definitely see a speed-up but I can't guarantee it will meet your FPS requirement.