numpy array C api

问题:

I have a C++ function returning a std::vector and I want to use it in python, so I'm using the C numpy api:

static PyObject *
py_integrate(PyObject *self, PyObject *args){
    ...
    std::vector<double> integral;
    cpp_function(integral);  // This changes integral
    npy_intp size = {integral.size()};
    PyObject *out = PyArray_SimpleNewFromData(1, &size, NPY_DOUBLE, &(integral[0]));
    return out;
}

Here's how I call it from python:

import matplotlib.pyplot as plt

a = py_integrate(parameters)
print a
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(a)
print a

What happens is: The first print is ok, the values are correct. But when I plot a they are not; in the second print I see very strange values like 1E-308 1E-308 ... or 0 0 0 ... as an uninitialized memory. I don't understand why the first print is ok.

Partial solution (not working):

static void DeleteVector(void *ptr)
{
    std::cout << "Delete" << std::endl;
    vector * v = static_cast<std::vector<double> * >(ptr);
    delete v;
    return;
}

static PyObject *
cppfunction(PyObject *self, PyObject *args)
{
    std::vector<double> *vector = new std::vector<double>();
    vector->push_back(1.);
    PyObject *py_integral = PyCObject_FromVoidPtr(vector, DeleteVector);
    npy_intp size = {vector->size()};
    PyArrayObject *out;
    ((PyArrayObject*) out)->base = py_integral;
    return (PyObject*)(out);
}

回答1:

Your std::vector object appears to be local to that function. PyArray_SimpleNewFromData does not make a copy of the data you pass it. It just keeps a pointer. So once your py_integrate function returns, the vector is deallocated. The print works the first time because nothing has written over the freed memory yet, but by the time you get to the next print, something else has used that memory, causing the values to be different.

You need to make a NumPy array that owns its own storage space and then copy the data into it.

Alternatively, allocate your vector on the heap. Then store a pointer to it in a CObject. Provide a destructor that deletes the vector. Then, take a look at the C-level PyArrayObject type. It has a PyObject * member called base. Store your CObject there. Then when the NumPy array is garbage collected, the reference count on this base object will be decremented, and assuming you haven't taken a copy of it elsewhere, your vector will be deleted thanks to the destructor you provided.

Fixer-upper

You forgot to actually create the PyArray. Try this:

(You didn't post DeleteVector, so I can only hope that it's right)

std::vector<double> *vector = new std::vector<double>();
vector->push_back(1.);
PyObject *py_integral = PyCObject_FromVoidPtr(vector, DeleteVector);
npy_intp size = {vector->size()};
PyObject *out = PyArray_SimpleNewFromData(1, &size, NPY_DOUBLE, &((*vector)[0]));
((PyArrayObject*) out)->base = py_integral;
return out;

Note: I'm not a C++ programmer, so I can only assume that &((*vector)[0]) works as intended with a pointer to a vector. I do know that the vector reallocate its storage area if you grow it, so don't increase its size after getting that pointer or it won't be valid anymore.

回答2:

You will need to make a copy of the vector, since the vector will go out of scope and the memory will no longer be usable by the time you need it in Python (as stated by kwatford).

One way to make the Numpy array you need (by copying the data) is:

PyObject *out = nullptr;

std::vector<double> *vector = new std::vector<double>();
vector->push_back(1.);

npy_intp size = {vector.size()};

out = PyArray_SimpleNew(1, &size, NPY_DOUBLE);

memcpy(PyArray_DATA((PyArrayObject *) out), vector.data(), vector.size());