How to serialize a QAbstractItemModel into QDataSt

Question

I've set up a QAbstractItemModel and filled that with data. My QTreeView widget displays every data in that model properly.

Now, I would like to store that model serialized in a binary file (and later of cource load that binary file back into a model). Is that possible?

Answer 1

The particulars of model serialization depend somewhat on the model's implementation. Some gotchas include:

1. Perfectly usable models might not implement insertRows/insertColumns, preferring to use custom methods instead.

2. Models like QStandardItemModel may have underlying items of varying types. Upon deserialization, the prototype item factory will repopulate the model with clones of one prototype type. To prevent that, the items't type identifier must be exposed for serialization, and a way provided to rebuild the item of a correct type upon deserialization.

   Let's see one way of implementing it for the standard item model. The prototype polymorphic item class can expose its type via a data role. Upon setting this role, it should re-create itself with a correct type.

Given this, a universal serializer isn't feasible.

Let's look at a complete example, then. The behaviors necessary for a given model type must be represented by a traits class that parametrizes the serializer. The methods reading data from the model take a constant model pointer. The methods modifying the model take a non-constant model pointer, and return false upon failure.

// https://github.com/KubaO/stackoverflown/tree/master/questions/model-serialization-32176887
#include <QtGui>

struct BasicTraits  {
    BasicTraits() {}
    /// The base model that the serializer operates on
    typedef QAbstractItemModel Model;
    /// The streamable representation of model's configuration
    typedef bool ModelConfig;
    /// The streamable representation of an item's data
    typedef QMap<int, QVariant> Roles;
    /// The streamable representation of a section of model's header data
    typedef Roles HeaderRoles;
    /// Returns a streamable representation of an item's data.
    Roles itemData(const Model * model, const QModelIndex & index) {
        return model->itemData(index);
    }
    /// Sets the item's data from the streamable representation.
    bool setItemData(Model * model, const QModelIndex & index, const Roles & data) {
        return model->setItemData(index, data);
    }
    /// Returns a streamable representation of a model's header data.
    HeaderRoles headerData(const Model * model, int section, Qt::Orientation ori) {
        Roles data;
        data.insert(Qt::DisplayRole, model->headerData(section, ori));
        return data;
    }
    /// Sets the model's header data from the streamable representation.
    bool setHeaderData(Model * model, int section, Qt::Orientation ori, const HeaderRoles & data) {
        return model->setHeaderData(section, ori, data.value(Qt::DisplayRole));
    }
    /// Should horizontal header data be serialized?
    bool doHorizontalHeaderData() const { return true; }
    /// Should vertical header data be serialized?
    bool doVerticalHeaderData() const { return false; }
    /// Sets the number of rows and columns for children on a given parent item.
    bool setRowsColumns(Model * model, const QModelIndex & parent, int rows, int columns) {
        bool rc = model->insertRows(0, rows, parent);
        if (columns > 1) rc = rc && model->insertColumns(1, columns-1, parent);
        return rc;
    }
    /// Returns a streamable representation of the model's configuration.
    ModelConfig modelConfig(const Model *) {
        return true;
    }
    /// Sets the model's configuration from the streamable representation.
    bool setModelConfig(Model *, const ModelConfig &) {
        return true;
    }
};

Such a class must be implemented to capture the requirements of a particular model. The one given above is often sufficient for basic models. A serializer instance takes or default-constructs an instance of the traits class. Thus, traits can have state.

When dealing with streaming and model operations, either can fail. A Status class captures whether the stream and model are ok, and whether it's possible to continue. When IgnoreModelFailures is set on the initial status, the failures reported by the traits class are ignored and the loading proceeds in spite of them. QDataStream failures always abort the save/load.

struct Status {
    enum SubStatus { StreamOk = 1, ModelOk = 2, IgnoreModelFailures = 4 };
    QFlags<SubStatus> flags;
    Status(SubStatus s) : flags(StreamOk | ModelOk | s) {}
    Status() : flags(StreamOk | ModelOk) {}
    bool ok() const {
        return (flags & StreamOk && (flags & IgnoreModelFailures || flags & ModelOk));
    }
    bool operator()(QDataStream & str) {
        return stream(str.status() == QDataStream::Ok);
    }
    bool operator()(Status s) {
        if (flags & StreamOk && ! (s.flags & StreamOk)) flags ^= StreamOk;
        if (flags & ModelOk && ! (s.flags & ModelOk)) flags ^= ModelOk;
        return ok();
    }
    bool model(bool s) {
        if (flags & ModelOk && !s) flags ^= ModelOk;
        return ok();
    }
    bool stream(bool s) {
        if (flags & StreamOk && !s) flags ^= StreamOk;
        return ok();
    }
};

This class could also be implemented to throw itself as an exception instead of returning false. This would make the serializer code a bit easier to read, as every if (!st(...)) return st idiom would be replaced by simpler st(...). Nevertheless, I chose not to use exceptions, as typical Qt code doesn't use them. To completely remove the syntax overhead of detecting traits methods and stream failures, one would need to throw in the traits methods instead of returning false, and use a stream wrapper that throws on failure.

Finally, we have a generic serializer, parametrized by a traits class. The majority of model operations are delegated to the traits class. The few operations performed directly on the model are:

• bool hasChildren(parent)
• int rowCount(parent)
• int columnCount(parent)
• QModelIndex index(row, column, parent)

template <class Tr = BasicTraits> class ModelSerializer {
    enum ItemType { HasData = 1, HasChildren = 2 };
    Q_DECLARE_FLAGS(ItemTypes, ItemType)
    Tr m_traits;

Headers for each orientation are serialized based on the root item row/column counts.

    Status saveHeaders(QDataStream & s, const typename Tr::Model * model, int count, Qt::Orientation ori) {
        Status st;
        if (!st(s << (qint32)count)) return st;
        for (int i = 0; i < count; ++i)
            if (!st(s << m_traits.headerData(model, i, ori))) return st;
        return st;
    }
    Status loadHeaders(QDataStream & s, typename Tr::Model * model, Qt::Orientation ori, Status st) {
        qint32 count;
        if (!st(s >> count)) return st;
        for (qint32 i = 0; i < count; ++i) {
            typename Tr::HeaderRoles data;
            if (!st(s >> data)) return st;
            if (!st.model(m_traits.setHeaderData(model, i, ori, data))) return st;
        }
        return st;
    }

The data for each item is serialized recursively, ordered depth-first, columns-before-rows. Any item can have children. Item flags are not serialized; ideally this behavior should be parametrized in the traits.

    Status saveData(QDataStream & s, const typename Tr::Model * model, const QModelIndex & parent) {
        Status st;
        ItemTypes types;
        if (parent.isValid()) types |= HasData;
        if (model->hasChildren(parent)) types |= HasChildren;
        if (!st(s << (quint8)types)) return st;
        if (types & HasData) s << m_traits.itemData(model, parent);
        if (! (types & HasChildren)) return st;
        auto rows = model->rowCount(parent);
        auto columns = model->columnCount(parent);
        if (!st(s << (qint32)rows << (qint32)columns)) return st;
        for (int i = 0; i < rows; ++i)
            for (int j = 0; j < columns; ++j)
                if (!st(saveData(s, model, model->index(i, j, parent)))) return st;
        return st;
    }
    Status loadData(QDataStream & s, typename Tr::Model * model, const QModelIndex & parent, Status st) {
        quint8 rawTypes;
        if (!st(s >> rawTypes)) return st;
        ItemTypes types { rawTypes };
        if (types & HasData) {
            typename Tr::Roles data;
            if (!st(s >> data)) return st;
            if (!st.model(m_traits.setItemData(model, parent, data))) return st;
        }
        if (! (types & HasChildren)) return st;
        qint32 rows, columns;
        if (!st(s >> rows >> columns)) return st;
        if (!st.model(m_traits.setRowsColumns(model, parent, rows, columns))) return st;
        for (int i = 0; i < rows; ++i)
            for (int j = 0; j < columns; ++j)
                if (!st(loadData(s, model, model->index(i, j, parent), st))) return st;
        return st;
    }

The serializer retains a traits instance, it can also be passed one to use.

public:
    ModelSerializer() {}
    ModelSerializer(const Tr & traits) : m_traits(traits) {}
    ModelSerializer(Tr && traits) : m_traits(std::move(traits)) {}
    ModelSerializer(const ModelSerializer &) = default;
    ModelSerializer(ModelSerializer &&) = default;

The data is serialized in following order:

1. model configuration,
2. model data,
3. horizontal header data,
4. vertical header data.

Attention is paid to versioning of both the stream and the streamed data.

    Status save(QDataStream & stream, const typename Tr::Model * model) {
        Status st;
        auto version = stream.version();
        stream.setVersion(QDataStream::Qt_5_4);
        if (!st(stream << (quint8)0)) return st; // format
        if (!st(stream << m_traits.modelConfig(model))) return st;
        if (!st(saveData(stream, model, QModelIndex()))) return st;
        auto hor = m_traits.doHorizontalHeaderData();
        if (!st(stream << hor)) return st;
        if (hor && !st(saveHeaders(stream, model, model->rowCount(), Qt::Horizontal))) return st;
        auto ver = m_traits.doVerticalHeaderData();
        if (!st(stream << ver)) return st;
        if (ver && !st(saveHeaders(stream, model, model->columnCount(), Qt::Vertical))) return st;
        stream.setVersion(version);
        return st;
    }
    Status load(QDataStream & stream, typename Tr::Model * model, Status st = Status()) {
        auto version = stream.version();
        stream.setVersion(QDataStream::Qt_5_4);
        quint8 format;
        if (!st(stream >> format)) return st;
        if (!st.stream(format == 0)) return st;
        typename Tr::ModelConfig config;
        if (!st(stream >> config)) return st;
        if (!st.model(m_traits.setModelConfig(model, config))) return st;
        if (!st(loadData(stream, model, QModelIndex(), st))) return st;
        bool hor;
        if (!st(stream >> hor)) return st;
        if (hor && !st(loadHeaders(stream, model, Qt::Horizontal, st))) return st;
        bool ver;
        if (!st(stream >> ver)) return st;
        if (ver && !st(loadHeaders(stream, model, Qt::Vertical, st))) return st;
        stream.setVersion(version);
        return st;
    }
};

To save/load a model using the basic traits:

int main(int argc, char ** argv) {
    QCoreApplication app{argc, argv};
    QStringList srcData;
    for (int i = 0; i < 1000; ++i) srcData << QString::number(i);
    QStringListModel src {srcData}, dst;
    ModelSerializer<> ser;
    QByteArray buffer;
    QDataStream sout(&buffer, QIODevice::WriteOnly);
    ser.save(sout, &src);
    QDataStream sin(buffer);
    ser.load(sin, &dst);
    Q_ASSERT(srcData == dst.stringList());
}

Answer 2

The same way you serialize anything, just implement an operator or method which writes each data member to a data stream in sequence.

The preferable format is to implement those two operators for your types:

QDataStream &operator<<(QDataStream &out, const YourType &t);
QDataStream &operator>>(QDataStream &in, YourType &t);

Following that pattern will allow your types to be "plug and play" with Qt's container classes.

QAbstractItemModel does not (or should not) directly hold the data, it is just a wrapper to an underlying data structure. The model only serves to provide an interface for a view to access the data. So in reality you shouldn't serialize the actual model, but the underlying data.

As of how to serialize the actual data, it depends on the format of your data, which as of now remains a mystery. But since it is a QAbstractItemModel I assume it is a tree of some sort, so generally speaking, you have to traverse the tree and serialize every object in it.

Make a note that when serializing a single object, the serialization and deserialization are a blind sequence, but when dealing with a collection of objects, you may have to account for its structure with extra serialization data. If your tree is something like an array of arrays, as long as you use Qt's container classes this will be taken care of for you, all you will need is to implement the serialization for the item type, but for a custom tree you will have to do it yourself.