Pickling a Python Extension type defined as a C st

Question

I am running C++ code via Python and would like to pickle an extension type.

So I have a C++ struct (py_db_manager) containing pointers to a database object and a object manager object (both written in C++) that I wrapped with a python type object (t_db_manager). My problem is that this python type needs to know how to pickle the two pointers in order to send it to some child multicore processes. So I registered the type with the copy_reg module (this is equivalent to writing a reduce() method on the type. However, I'm not too sure what to put in it. Should I build a tuple with the PyObject* or just the integer pointers?. Can anyone help?

typedef struct
{
  PyObject_HEAD
  PyObject* man_inst_ ;
  PyObject* db_inst_ ;

}py_db_manager;`

Here's the Py_TypeObject

PyTypeObject t_db_manager = {
    PyObject_HEAD_INIT(0)               /* tp_head */
    0,                                  /* tp_internal */
    ".py_db_manager",                  /* tp_name */
    sizeof(py_db_manager)};

And here's the code that would be in the reduce method:

PyObject *pickle_manager(PyObject *module, PyObject *args)
{
    py_db_manager *cpp_manager =0;
    PyObject *values = NULL,
        *tuple = NULL;
    char text[512];

    if (!PyArg_ParseTuple(args, "O!", &t_db_manager, &cpp_manager))
        goto error;
    sprintf(text,"man_inst_, db_inst_");
    if ((values = Py_BuildValue("(sii)", text,
                                cpp_manager->man_inst_, cpp_manager->db_inst_)) == NULL)
        goto error;
    tuple = Py_BuildValue("(OO)", manager_constructor, values);

error:
    Py_XDECREF(values);
    return tuple;
}

Answer 1

Because this will be passed to another process, pickling just integer pointers will not work as you would want it to. Different processes use different memory space therefore they don't see the same things.

So, to answer your question, you should pickle full objects and reconstruct them from the receiving end.