How to hook up Boost serialization & iostreams to

Question

I've been using the Boost serialization library, which is actually pretty nice, and lets me make simple wrappers to save my serializable objects to strings, like so:

namespace bar = boost::archive;
namespace bio = boost::iostreams;

template <class T> inline std::string saveString(const T & o) {
 std::ostringstream oss;
 bar::binary_oarchive oa(oss);
 oa << o;
 return oss.str();
}
template <class T> inline void saveFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 bar::binary_oarchive oa(ofs);
 oa << o;
}
template <class T> inline void loadFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 bar::binary_iarchive ia(ifs);
 ia >> o;
}

The thing is, I just found the need to compress my serialized data, too, so I'm looking at doing that with the filters in boost::iostreams. I figured out how to do it successfully with files:

template <class T> inline void saveGZFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 bio::filtering_streambuf<bio::output> out;
 out.push(boost::iostreams::gzip_compressor());
 out.push(ofs);
 bar::binary_oarchive oa(out);
 oa << o;
}
template <class T> inline void loadGZFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 bio::filtering_streambuf<bio::input> in;
 in.push(bio::gzip_decompressor());
 in.push(ifs);
 bar::binary_iarchive ia(in);
 ia >> o;
}

But can't figure out how to save correctly to a compressed string. The problem is that I'm not flushing the chain of filters, but I've tried popping and syncing and nothing seems to work. Here's my broken code:

template <class T> inline std::string saveGZString(const T & o) {
 std::ostringstream oss;
 bio::filtering_streambuf<bio::output> out;
 out.push(bio::gzip_compressor());
 out.push(oss);
 bar::binary_oarchive oa(out);
 oa << o;
 // XXX out.pop() twice?  out.strict_sync()??  oss.flush()??
 return oss.str();
}

As a result some data gets stuck in the stream buffer somewhere, and I always end up with a a few complete blocks (16K or 32K) of compressed data when I know it should be 43K or so given the (valid) output I get from using my saveGZFile method. Apparently hooking up the ofstream closes and flushes properly, but hooking up the ostringstream doesn't.

Any help? (This is my first stackoverflow question — help me, guys, you're my only hope!)

Answer 1

Returning to this question, I realized I must've fixed it sometime last year (as I'm using saveGZString right now). Digging to see how I fixed it, it was pretty silly/simple:

namespace bar = boost::archive;
namespace bio = boost::iostreams;

template <typename T> inline std::string saveGZString(const T & o) {
        std::ostringstream oss;
        { 
                bio::filtering_stream<bio::output> f;
                f.push(bio::gzip_compressor());
                f.push(oss);
                bar::binary_oarchive oa(f);
                oa << o;
        } // gzip_compressor flushes when f goes out of scope
        return oss.str();
}

Just let the whole chain go out of scope and it works! Neat! Here's my loader for completeness:

template <typename T> inline void loadGZString(T & o, const std::string& s) {
        std::istringstream iss(s);
        bio::filtering_stream<bio::input> f;
        f.push(bio::gzip_decompressor());
        f.push(iss);
        bar::binary_iarchive ia(f);
        ia >> o;
}

Answer 2

I haven't run the code myself, but my best guess is to use out.strict_sync() which applies flush() to every filter/device in the pipeline. I can't seem to tell, though, if gzip_compressor is flushable. If it is not, then strict_sync() will return false, and sync() would be more appropriate.