How to hook up Boost serialization & iostreams to serialize & gzip an object to string?

boost, boost-iostreams, boost-serialization, c++, ostringstream

Solution

Returning to this question, I realized I must've fixed it sometime last year (as I'm using saveGZString right now). Digging to see how I fixed it, it was pretty silly/simple:

namespace bar = boost::archive;
namespace bio = boost::iostreams;

template <typename T> inline std::string saveGZString(const T & o) {
        std::ostringstream oss;
        { 
                bio::filtering_stream<bio::output> f;
                f.push(bio::gzip_compressor());
                f.push(oss);
                bar::binary_oarchive oa(f);
                oa << o;
        } // gzip_compressor flushes when f goes out of scope
        return oss.str();
}

Just let the whole chain go out of scope and it works! Neat! Here's my loader for completeness:

template <typename T> inline void loadGZString(T & o, const std::string& s) {
        std::istringstream iss(s);
        bio::filtering_stream<bio::input> f;
        f.push(bio::gzip_decompressor());
        f.push(iss);
        bar::binary_iarchive ia(f);
        ia >> o;
}

Problem

I've been using the Boost serialization library, which is actually pretty nice, and lets me make simple wrappers to save my serializable objects to strings, like so: ``` namespace bar = boost::archive; namespace bio = boost::iostreams; template <class T> inline std::string saveString(const T & o) { std::ostringstream oss; bar::binary_oarchive oa(oss); oa << o; return oss.str(); } template <class T> inline void saveFile(const T & o, const char* fname) { std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc); bar::binary_oarchive oa(ofs); oa << o; } template <class T> inline void loadFile(T & o, const char* fname) { std::ifstream ifs(fname, std::ios::in|std::ios::binary); assert(ifs.good()); // XXX catch if file not found bar::binary_iarchive ia(ifs); ia >> o; } ``` The thing is, I just found the need to compress my serialized data, too, so I'm looking at doing that with the filters in boost::iostreams. I figured out how to do it successfully with files: ``` template <class T> inline void saveGZFile(const T & o, const char* fname) { std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc); bio::filtering_streambuf<bio::output> out; out.push(boost::iostreams::gzip_compressor()); out.push(ofs); bar::binary_oarchive oa(out); oa << o; } template <class T> inline void loadGZFile(T & o, const char* fname) { std::ifstream ifs(fname, std::ios::in|std::ios::binary); assert(ifs.good()); // XXX catch if file not found bio::filtering_streambuf<bio::input> in; in.push(bio::gzip_decompressor()); in.push(ifs); bar::binary_iarchive ia(in); ia >> o; } ``` But can't figure out how to save correctly to a compressed string. The problem is that I'm not flushing the chain of filters, but I've tried popping and syncing and nothing seems to work. Here's my broken code: ``` template <class T> inline std::string saveGZString(const T & o) { std::ostringstream oss; bio::filtering_streambuf<bio::output> out; out.push(bio::gzip_compressor()); out.push(oss); bar::binary_oarchive oa(out); oa << o; // XXX out.pop() twice? out.strict_sync()?? oss.flush()?? return oss.str(); } ``` As a result some data gets stuck in the stream buffer somewhere, and I always end up with a a few complete blocks (16K or 32K) of compressed data when I know it should be 43K or so given the (valid) output I get from using my saveGZFile method. Apparently hooking up the ofstream closes and flushes properly, but hooking up the ostringstream doesn't. Any help? (This is my first stackoverflow question — help me, guys, you're my only hope!)

Original source