issue using deepcopy function for cython classes

Question

I've been playing with Cython recently for the speed ups, but when I was trying to use copy.deepcopy() some error occurred.Here is the code:

from copy import deepcopy

cdef class cy_child:
    cdef public:
        int move[2]
        int Q
        int N
    def __init__(self, move):
        self.move = move
        self.Q = 0
        self.N = 0      

a = cy_child((1,2))

b = deepcopy(a)

This is the error:
can't pickle _cython_magic_001970156a2636e3189b2b84ebe80443.cy_child objects

How can I solve the problem for this code?

Answer 1

As hpaulj says in the comments, deepcopy looks to use pickle by default to do its work. Cython cdef classes didn't used to be pickleable. In recent versions of Cython they are where possible (see also http://blog.behnel.de/posts/whats-new-in-cython-026.html) but pickling the array looks to be a problem (and even without the array I didn't get it to work).

The solution is to implement the relevant functions yourself. I've done __deepcopy__ since it's simple but alternatively you could implement the pickle protocol

def __deepcopy__(self,memo_dictionary):
    res = cy_child(self.move)
    res.Q = self.Q
    res.N = self.N
    return res

I suspect that you won't need to do that in the future as Cython improves their pickle implementation.

---

A note on memo_dictionary: Suppose you have

a=[None]
b=[A]
a[0]=B
# i.e. A contains a link to B and B contains a link to A
c = deepcopy(a)

memo_dictionary is used by deepcopy to keep a note of what it's already copied so that it doesn't loop forever. You don't need to do much with it yourself. However, if your cdef class contains a Python object (including another cdef class) you should copy it like this:

cdef class C:
    cdef object o
    def __deepcopy__(self,memo_dictionary):
        # ...
        res.o = deepcopy(self.o,memo_dictionary)
        # ...

(i.e. make sure it gets passed on to further calls of deepcopy)