Cython: C++ Use a vector in a dictionary?

Question

I am using the following code to try to work with C++ vectors:

from libcpp.vector cimport vector                                                                                                                                         

cdef struct StartEnd:
    long start, end 

cdef vector[StartEnd] vect
print(type(vect))
cdef int i
cdef StartEnd j
k = {}
k['hi'] = vect
for i in range(10):
    j.start = i 
    j.end = i + 2 
    k['hi'].push_back(j)
for i in range(10):
    print(k['hi'][i])

The exact functionality here isn't important, this is just a dummy program. The issue is that running this generates the error: AttributeError: 'list' object has no attribute 'push_back' This works if there is no dictionary, but I think that the dictionary is necessary for my use case. Is there a way to make this work?

I do not want to be copying vectors back and forth as these vectors will get to be tens of millions of entries long. Maybe I can store pointers to the vector instead?

Answer 1

The C++ vector automatically gets converted to list at the Cython/Python borderline (hence the error message you see). The Python dict expects to store Python objects rather than C++ vectors. Create a cdef class that holds a C++ Vector and put that in the dict instead:

cdef class VecHolder:
   cdef vector[StartEnd] wrapped_vector

   # the easiest thing to do is add short wrappers for the methods you need
   def push_back(self,obj):
     self.wrapped_vector.push_back(obj)

cdef int i
cdef StartEnd j
k = {}
k['hi'] = VecHolder()
for i in range(10):
   j.start = i 
   j.end = i + 2 
   k['hi'].push_back(j) # note that you're calling 
       # the wrapper method here which then calls the c++ function