Cython: Avoid copy through std::move not working

问题:

Problem

I have a very large std::vector that gets returned from a C++ function, let's call it getVector().

Now I want to wrap that function in Cython:

cdef extern from "XY.h":
    cdef cppclass _XY:
        vector[double] getVector() except +

cdef class XY:
    cdef _XY _this

    ...

    def getVector():
        return self._this.getVector()

As I want to avoid copying this large vector, I would like to make use of std::move. Like this:

cdef extern from "<utility>" namespace "std":
    vector[double] move(vector[double]) # Cython has no function templates

This modifies the Cython source code in the following way:

def getVector():
    return move(self._this.getVector())

Question

The above idea is not working. Cython is (at least) producing 1 copy of the vector. I assume this is because there is no way to move from a vector as this is already a Cython wrapper around the actual std::vector class from C++.

Is there another approach to avoid any copies? I would like to avoid returning a pointer from the C++ method.

There is probably a way by defining a C++ wrapper class that stores the vector and then move this class in Cython but I was wondering whether there is a way without (or very little) modifying the C++ source code.

回答1:

Edit: After @DavidW's warning I realized I misunderstood your question. Below answer just let's you use a templated move from cython directly without explicit declaration for each moving type (e.g. the one you declared for std::vector<double> in your question).

---

You can use this helper function for wrapping std::move call:

# distutils: language = c++

cdef extern from * namespace "polyfill":
    """
    namespace polyfill {

    template <typename T>
    inline typename std::remove_reference<T>::type&& move(T& t) {
        return std::move(t);
    }

    template <typename T>
    inline typename std::remove_reference<T>::type&& move(T&& t) {
        return std::move(t);
    }

    }  // namespace polyfill
    """
    cdef T move[T](T)

Example usage:

# distutils: language = c++

cdef extern from *:
    """
    #include <iostream>

    #define PRINT() std::cout << __PRETTY_FUNCTION__ << std::endl

    struct Test {
        Test() { PRINT(); }
        ~Test() { PRINT(); }
        Test(const Test&) { PRINT(); }
        Test(Test&&) { PRINT(); }
        Test& operator=(const Test&) { PRINT(); return *this; }
        Test& operator=(Test&&) { PRINT(); return *this; }
    };

    void f(const Test&) { PRINT(); }
    void f(Test&&) { PRINT(); }
    """

    cdef cppclass Test:
        pass

    cdef void f(Test)

from move cimport move

cdef Test t1, t2

print("# t1 = t2")
t1 = t2

print("# t1 = move(t2)")
t1 = move(t2)

print("# f(t1)")
f(t1)

print("# f(move(t1))")
f(move(t1))

print("# f(move(move(t1)))")
f(move(move(t1)))

print("# f(move(move(move(t1))))")
f(move(move(move(t1))))

Output (compiled using cythonize -3 -i test.pyx with Cython 0.29.12 and Python 3.7.3):

$ python3 -c "import test"
Test::Test()
Test::Test()
# t1 = t2
Test& Test::operator=(const Test&)
# t1 = move(t2)
Test& Test::operator=(Test&&)
# f(t1)
void f(const Test&)
# f(move(t1))
void f(Test&&)
# f(move(move(t1)))
void f(Test&&)
# f(move(move(move(t1))))
void f(Test&&)
Test::~Test()
Test::~Test()

Note that C++ objects are still default initialized since that's what Cython does currently, yet this helper function allows calling move assignment after initialization.

---

Edit: I packaged this snippet as cymove.