Using arbitrary QML items as cached image source

Question

EDIT: The original question is still contained below, but I decided to re-title to a form that will be more useful to developers in a variety of cases, some of which described in my answer below, as the solution to the original problem turned out to provide a much wider area of application.

---

I have a set of greyscale icons for an application, and a requirement that the icon color can be changed by the user.

So, the obvious solution is to use the stock Colorize element from QtGraphicalEffects.

The effect itself has a cached property - which caches the result of that particular effect so that it is not continuously calculated. However, this only applies to that particular instance of the effect, meaning if there are multiple icon instances, and every one has a colorize effect, this cache will not be shared between the different instances.

Obviously, one single cache would be enough, considering that all icons are the same size and color, that data from VRAM can be reused, saving on VRAM and GPU time.

So the big question is how to reuse that single cache of that single effect and display it multiple times without any overheads.

Also, the previous question is regarding the current course I've taken with colorizing icons. However, there might be another approach I am missing.

Naturally, efficiency is key, but simplicity is also desired, I mean I can think of several low level ways to do that very efficiently, but they all require more complex low level implementations, they are not possible to do in QML.

Answer 1

What I did to optimize QtGraphicalEffects was using Item.grabToImage() :)

This function returns QQuickItemGrabResult item which has url() function. This function returns an QUrl which can be set as a source for an Image object.

So what you need to do is create one Image with Colorize applied on it. When it is ready use grabToImage() and after successful grab save the QUrl somewhere safe and destroy the source object.

---

I suppose that you will need to change the color of the icons from time to time while the application is running. If so keep in mind that just changing the source of the Image objects so that no one uses grabbed image url will make it be released from memory. Not instantaneously but when it is needed.

Because of some incompability my applications only manage the memory correctly if I use QGuiApplication::setAttribute(Qt::AA_UseOpenGLES); in main.cpp file.

---

Also here is an important fact:

Images are cached and shared internally, so if several Image items have the same source, only one copy of the image will be loaded. (source)

---

Here is working example. The source object is Rectangle but it can easily be changed to an Image.

import QtQuick 2.3
import QtQuick.Window 2.2
import QtGraphicalEffects 1.0

Window {
    visible: true    
    width: 500
    height: 500

    property string urlOfMyIcon: ""


    Grid {
        columns: 1
        spacing: 10
        Image {
            width: 100
            height: 100
            source: urlOfMyIcon
        }
        Image {
            width: 100
            height: 100
            source: urlOfMyIcon
        }
        Image {
            width: 100
            height: 100
            source: urlOfMyIcon
        }
    }

    Component.onCompleted: {
        component.createObject(this)
    }

    Component {
        id: component
        Item {
            id: yourImageWithLoadedIconContainer
            Rectangle {
                id: yourImageWithLoadedIcon
                width: 80
                height: 80
                color: "white"
                visible: false

                // needed because I used Rectangle instead of Image
                Component.onCompleted: {
                    colorizeEffect.grabToImage(function(result) {
                        urlOfMyIcon = result.url;
                        yourImageWithLoadedIconContainer.destroy()
                    }, Qt.size(width, height));
                }

                // use this when using Image instead of Rectangle
//                onStatusChanged: {
//                    if (status === Image.Ready)
//                        colorizeEffect.grabToImage(function(result) {
//                            urlOfMyIcon = result.url;
//                            yourImageWithLoadedIconContainer.destroy()
//                        }, yourImageWithLoadedIcon.sourceSize);
//                }
            }
            Colorize {
                id: colorizeEffect
                anchors.fill: yourImageWithLoadedIcon
                source: yourImageWithLoadedIcon
                hue: 0.8
                saturation: 0.5
                lightness: -0.2
                visible: false
            }
        }
    }
}

Answer 2

The solution turned out to be unexpectedly simple.

In the case, specific to the OP - that is colorize icons, the most efficient way is to simply use a custom ShaderEffect with a trivial fragment shader - set gl_FragColor to the desired color, passed as a vec4 and the alpha value from the source image. There is really no need to cache anything, as the shader is really simple and fast, as fast as it gets.

There is only one thing to consider - the possibility that the QML scenegraph might allocate the original image in a texture atlas, the default implementation will then copy the texture from the atlas to another texture. We do not want this as it defeats the purpose - VRAM usage will rise, as this will be done for every "instance" and there is also the possibility that the newly allocate textures will be larger than they need to be, since on some platforms, there is a limitation to how small a texture can be, and in this case we are talking icons, so they won't be all that big.

The solution is to explicitly set supportsAtlasTextures to true. This means you must also pass the offset of the texture in the atlas and calculate the offset - still very little overhead. This will ensure efficiency, textures from atlases will not be duplicated in memory, and furthermore, the render engine will actually allow different shader effects using different texture from the same atlas to be batched together in one call.

---

A similar approach can be used to cache pretty much anything, and use that cache to display an "image" - use a ShaderEffectSource to "capture" the desired image, and then use a ShaderEffect with an even more trivial fragment shader - simply output the data from the source sampler. Several extremely useful use-cases immediately come to mind:

• it can be used to "instantiate" images, the result of computationally intenssive shaders, keep in mind that ShaderEffectSources and ShaderEffects can be chained in arbitrary order
• it can be used to instantiate procedurally generated images, once again using shaders, such can be used as tiling textures, and even animated very efficiently
• it can be used together with a QML Canvas to use a complex canvas drawing as a cache and source for multiple "images"

• it can be used as an image, produced by the composition of complex QML Items - those are actually quite heavy on RAM, imagine a scenario where you have a 1000 objects, and each of them is made out of 20 different QML items - rectangles, texts, images, god forbid animations, that's 20000 object in memory - that's like 500 MBs of RAM usage based on my tests, but if they are identical, a single object can be used to provide cache, and all the other objects can only use a single shader effect to display that cache. It has implications on CPU time as well - say your design is bound to changing values - a very usual scenario, if you have 20000 objects in memory, that's 20000 evaluated bindings - even for trivial expressions this may take several seconds on a mobile device, freezing the screen for that duration. Caching that will reduce the freeze time 1000 times, practically to non-existent.

• it can can also be used to cache and instantiate animations, significantly reducing the needed CPU time, and also it can work with video as well