Pass a cython allocated buffer to python

Question

I am aware of this post about passing over Cython malloc'ed data to Python. I however wonder if casting to a memoryview before returning avoids the memory leak:

def someCythonFunction():
    try:
        mPtr = <double *>PyMem_Malloc(N * M * sizeof(double)
        for n in range(N):
            for m in range(M):
                mPtr[m + n*M]= ...
        return <double[:N,:M]> mPtr
    finally:
        print("In cython %d" % <int>mPtr)
        PyMem_Free(mPtr)

then in Python:

mView = someCythonFunction()

Is this safe and correct ?

Answer 1

This is unsafe and incorrect. It deallocates the memory as the function ends which means the pointer that you return is immediately invalid.

You need to tie the lifetime of the memory to the lifetime of a Python object (as in the example you linked to, the memory is freed in the destructor). The simplest and recommended way of doing this is to use a numpy array or a standard library array array (or other library of your choice).

If you can't avoid using malloc one option is to use the cython.view.array class. You can assign it a callback function of your choice to use on destruction and it can be happily assigned to memoryviews:

cdef double[:,:] mview
cdef double* mPtr = <double *>PyMem_Malloc(N * M * sizeof(double))
a = cython.view.array(shape=(N, M), itemsize=sizeof(double), format="d",
                      mode="C", allocate_buffer=False)
a.data = <char *> mPtr
a.callback_free_data = PyMem_Free

mview = a

You can safely return either a or mview (if you return a then there's no need to bother with mview) and they will get correctly at the right time.