I have some data that was packed with msgpack using the C/C++ api as follows:
msgpack::sbuffer sbuf;
msgpack::packer<msgpack::sbuffer> pk(&sbuf);
int var1 = 10;
std::string var2 = "test";
double var3 = 3.14159; // could be any type
pk.pack_array(3);
pk.pack(var1);
pk.pack(var2);
pk.pack(var3);
At a later point I need to unpack that array, but need direct access to the third element so I can persist to file/db/redis/memcached/etc. While the first two elements of the array are fixed types, the third can be any type acceptable to msgpack (int, string, vector, map, etc).
size_t off = 0;
msgpack::unpacked result = msgpack::unpack(sbuf.data(), sbuf.len(), off);
msgpack::object obj = result.get();
int var1;
obj.via.array.ptr[0].convert(&var1);
std::string var2;
obj.via.array.ptr[1].convert(&var2);
// now here I want to get a pointer & len to the 3rd item so I can persist
// this value that is already msgpack'd.
const char* dataptr = reinterpret_cast<const char*>(&obj.via.array.ptr[2]);
// now what is the length of the data pointed to by dataptr?
Potentially I could do a reinterpret_cast on obj.via.array.ptr[2] as shown above, but in the case of binary data, or a msgpack'd struct, a simple strlen() wouldn't get me the length and I can't see where to get the item's length. I know there is a size variable in many of the types but don't believe this is accurate when that item is an array or map.
Here is a memory model of the msgpack-c:
https://github.com/msgpack/msgpack-c/wiki/v1_1_cpp_unpacker#memory-management
I know there is a size variable in many of the types but don't believe this is accurate when that item is an array or map.
True. obj has already been unpacked. It is not suitable for persistent.
I think that storing msgpack format binary data directly is better approach. First, separate the msgpack to the first two and the thrid one. Then, pack the first two as an array. Finally, simply pack the third value. That is packing process.
pk.pack_array(2); // for the first two
pk.pack(var1);
pk.pack(var2);
pk.pack(var3); // for the thrid one
When unpacking, unpack the first two data with offset.
// Unpacking
size_t off = 0;
msgpack::unpacked result = msgpack::unpack(sbuf.data(), sbuf.size(), off);
// off has been set
After unpacking them, offset has been set. So you can get the start point of the third data. Then, store the msgpack format binary data.
std::string store; // file/db/redis/memcached/etc
std::copy(sbuf.data() + off, sbuf.data() + sbuf.size(), std::back_inserter(store));
That is storing process.
When you get the msgpack format binary data from storage, unpack them.
Here is a whole code example:
#include <msgpack.hpp>
#include <iostream>
#include <string>
#include <algorithm>
int main() {
msgpack::sbuffer sbuf;
msgpack::packer<msgpack::sbuffer> pk(&sbuf);
int var1 = 10;
std::string var2 = "test";
double var3 = 3.14159; // could be any type
// Separate the data into the two msgpacks
pk.pack_array(2); // for the first two
pk.pack(var1);
pk.pack(var2);
pk.pack(var3); // for the thrid one
// Unpacking
size_t off = 0;
msgpack::unpacked result = msgpack::unpack(sbuf.data(), sbuf.size(), off);
msgpack::object obj = result.get();
auto converted = obj.as<std::tuple<int, std::string>>();
std::cout << std::get<0>(converted) << std::endl;
std::cout << std::get<1>(converted) << std::endl;
// Storing the thrid one
std::cout << "off: " << off << std::endl;
std::string store; // file/db/redis/memcached/etc
std::copy(sbuf.data() + off, sbuf.data() + sbuf.size(), std::back_inserter(store));
{
// Unpack the thrid one from store
msgpack::unpacked result = msgpack::unpack(store.data(), store.size());
msgpack::object obj = result.get();
if (obj.type == msgpack::type::FLOAT) {
auto f = obj.as<float>();
std::cout << f << std::endl;
}
}
}
You can check the behavior of the code above here:
http://melpon.org/wandbox/permlink/uFfRGKQLqnIIiDrv
