cython memoryviews slices without GIL

问题:

I want to release the GIL in order to parallelise loop in cython, where different slices of memoryviews are passed to a some function inside the loop. The code looks like this:

cpdef void do_sth_in_parallel(bint[:,:] input, bint[:] output, int D):
     for d in prange(D, schedule=dynamic, nogil=True):
          ouput[d] = some_function_not_requiring_gil(x[d,:])

This is not possible, since selecting the slice x[d,:], seems to require GIL. Running cython -a, and using a normal for loop, I get the code posted below. How can this be done in pure C?

      __pyx_t_5.data = __pyx_v_x.data;
      __pyx_t_5.memview = __pyx_v_x.memview;
      __PYX_INC_MEMVIEW(&__pyx_t_5, 0);
      {
    Py_ssize_t __pyx_tmp_idx = __pyx_v_d;
    Py_ssize_t __pyx_tmp_shape = __pyx_v_x.shape[0];
    Py_ssize_t __pyx_tmp_stride = __pyx_v_x.strides[0];
    if (0 && (__pyx_tmp_idx < 0))
        __pyx_tmp_idx += __pyx_tmp_shape;
    if (0 && (__pyx_tmp_idx < 0 || __pyx_tmp_idx >= __pyx_tmp_shape)) {
        PyErr_SetString(PyExc_IndexError, "Index out of bounds (axis 0)");
        __PYX_ERR(0, 130, __pyx_L1_error)
    }
        __pyx_t_5.data += __pyx_tmp_idx * __pyx_tmp_stride;
}

__pyx_t_5.shape[0] = __pyx_v_x.shape[1];
__pyx_t_5.strides[0] = __pyx_v_x.strides[1];
    __pyx_t_5.suboffsets[0] = -1;

__pyx_t_6.data = __pyx_v_u.data;
      __pyx_t_6.memview = __pyx_v_u.memview;
      __PYX_INC_MEMVIEW(&__pyx_t_6, 0);
      __pyx_t_6.shape[0] = __pyx_v_u.shape[0];
__pyx_t_6.strides[0] = __pyx_v_u.strides[0];
    __pyx_t_6.suboffsets[0] = -1;

回答1:

The following works for me:

from cython.parallel import prange

cdef bint some_function_not_requiring_gil(bint[:] x) nogil:
    return x[0]

cpdef void do_sth_in_parallel(bint[:,:] input, bint[:] output, int D):
     cdef int d
     for d in prange(D, schedule=dynamic, nogil=True):
          output[d] = some_function_not_requiring_gil(input[d,:])

The two main changes I had to make were x to input (because it's assuming it can find x as a python object at the global scope) to fix the error

Converting to Python object not allowed without gil

and adding cdef int d to force the type of d and fix the error

Coercion from Python not allowed without the GIL

(I also created an example some_function_not_requiring_gil but I assume this is fairly obvious)

回答2:

Solution that works for me:

Access the array slice using

input[d:d+1, :]

instead of

input [d,:]

And pass a 2D array.