Issue with boost serialization of IplImage struct

Question

I'm having trouble getting the boost serialization module to work with OpenCV's IplImage struct. Here is my code for serializing an IplImage (along with some JSON data in a custom struct)

    template <class Archive>
    void save(Archive & ar, const unsigned int version) const
    {
        // First things first; save the essential variables
        // These are needed as input to the create function of IplImage
        ar & frame->width;
        ar & frame->height;
        ar & frame->depth;
        ar & frame->nChannels;
        ar & frame->widthStep;
        ar & frame->imageSize;

//            ar & frame->nSize;
//            ar & frame->ID;
//            ar & frame->dataOrder;
//            ar & frame->origin;
//            ar & frame->align;
//            ar & frame->widthStep;
//            ar & frame->imageSize;

        // Then save the actual image data
        ar & boost::serialization::make_array<char>(frame->imageData, (frame->width * frame->height * frame->nChannels) + 1);

        std::string metaString = meta.dump();
        ar & metaString; // ...and don't forget the meta data
    }

Here is my code for deserializing the same struct

    template <class Archive>
    void load(Archive & ar, const unsigned int version)
    {
        int width;
        int height;
        int depth;
        int nChannels;
        int widthStep;
        int imageSize;

        ar & width;
        ar & height;
        ar & depth;
        ar & nChannels;
        ar & widthStep;
        ar & imageSize;

        // Create the image header with this knowledge
        frame = cvCreateImage(cvSize(width, height), depth, nChannels);
        frame->widthStep = widthStep;
        frame->imageSize = imageSize;

        // And grab the rest of the data
//            ar & frame->nSize;
//            ar & frame->ID;
//            ar & frame->dataOrder;
//            ar & frame->origin;
//            ar & frame->align;
//            ar & frame->widthStep;
//            ar & frame->imageSize;

        // Now we have all the variables, we can load the actual image data
        ar & boost::serialization::make_array<char>(frame->imageData, (width * height * nChannels) + 1);

        // Deserialize the json data
        std::string metaString;
        ar & metaString;
        meta = json(metaString);
    }

The resulting image I obtain after deserializing has pixel noise at the bottom of the image

At serialization:

At deserialization:

Answer 1

It appears that the lower part of your image is random memory. Most likely, the length of the char array must be longer. IplImage supports aligned data where widthStep is larger than width*nChannels. Your calculation can only handle 8Bit images without alignment.

Use make_array with imageSize.

ar & boost::serialization::make_array<char>(frame->imageData, imageSize);

Answer 2

Spambot analysis is correct, but there is more.

• Your data maybe contiguous, but the relevant parts (ie the pixel data displayed) aren't.
• The data origin is not frame->imageData. Because of that there is a gap between the last pixel of a row and the first pixel of the next row of size (frame->imageData - frame->imageDataOrigin)
• frame->widthStep and frame->imageSize are values which make sense only given a specific alignment. They should not be set, and should be expected to be different for each object created with cvCreateImage

To serialize you need to do something like

for(int i = 0; i < height; ++i)
{
   ar & boost::serialization::make_array<char>(
         frame->imageData+i*widthStep, 
         width * nChannels);
}

It saves only the visible portion of the buffer (the one you care about)

to deserialize you also need to iterate over horizontal lines (strides) using the same formula.

You can also convert to a cv::Mat with data copy enabled, which will give a real contiguous array of size width * height * nChannels and then serialize it with make_array.