Boost.Python: How to expose std::unique_ptr

Question

I am fairly new to boost.python and trying to expose the return value of a function to python.

The function signature looks like this:

 std::unique_ptr<Message> someFunc(const std::string &str) const;

When calling the function in python, I get the following error:

TypeError: No to_python (by-value) converter found for C++ type: std::unique_ptr<Message, std::default_delete<Message> >

My function call in python looks like this:

a = mymodule.MyClass()
a.someFunc("some string here") # error here

I tried to expose the std::unique_ptr but just cant get it to work.. Does someone know how to properly expose the pointer class? Thanks!

Edit: I tried the following:

class_<std::unique_ptr<Message, std::default_delete<Message>>, bost::noncopyable ("Message", init<>())

;

This example compiles, but I still get the error mentioned above. Also, I tried to expose the class Message itself

class_<Message>("Message", init<unsigned>())

        .def(init<unsigned, unsigned>()) 
        .def("f", &Message::f)
;

Answer 1

My suggestion is to get the raw pointer from the std::unique_ptr container with get(). You will have to careful to keep the unique_ptr in scope for for whole time that you wish to use the raw pointer value, otherwise the object will be deleted and you'll have a pointer to an invalid area of memory.

Answer 2

Boost supports movable semantics and unique_ptr since v.1.55. But in my project I used previous version and made such simple wrapper:

class_<unique_ptr<HierarchyT>, noncopyable>(typpedName<LinksT>("hierarchy", false)
, "hierarchy holder")
    .def("__call__", &unique_ptr<HierarchyT>::get,
        return_internal_reference<>(),
        "get holding hierarchy")
    .def("reset", &unique_ptr<HierarchyT>::reset,
        "reset holding hierarhy")
    ;

to create unique_ptr<HierarchyT> as Python shierarchy and pass it to the function that accepts it by reference.
Python code:

hier = mc.shierarchy()
mc.clusterize(hier, nds)

where C++ function is float clusterize(unique_ptr<HierarchyT>& hier,...).
Then to access results in Python make a call hier() to get the wrapped object from the unique_ptr:

output(hier(), nds)

Answer 3

In short, Boost.Python does not support move-semantics, and therefore does not support std::unique_ptr. Boost.Python's news/change log has no indication that it has been updated for C++11 move-semantics. Additionally, this feature request for unique_ptr support has not been touched for over a year.

Nevertheless, Boost.Python supports transferring exclusive ownership of an object to and from Python via std::auto_ptr. As unique_ptr is essentially a safer version of auto_ptr, it should be fairly straight forward to adapt an API using unique_ptr to an API that uses auto_ptr:

• When C++ transfers ownership to Python, the C++ function must:
  • be exposed with CallPolicy of boost::python::return_value_policy with a boost::python::manage_new_object result converter.
  • have unique_ptr release control via release() and return a raw pointer
• When Python transfers ownership to C++, the C++ function must:
  • accept the instance via auto_ptr. The FAQ mentions that pointers returned from C++ with a manage_new_object policy will be managed via std::auto_ptr.
  • have auto_ptr release control to a unique_ptr via release()

---

Given an API/library that cannot be changed:

/// @brief Mockup Spam class.
struct Spam;

/// @brief Mockup factory for Spam.
struct SpamFactory
{
  /// @brief Create Spam instances.
  std::unique_ptr<Spam> make(const std::string&);

  /// @brief Delete Spam instances.
  void consume(std::unique_ptr<Spam>);
};

The SpamFactory::make() and SpamFactory::consume() need to be wrapped via auxiliary functions.

Functions transferring ownership from C++ to Python can be generically wrapped by a function that will create Python function objects:

/// @brief Adapter a member function that returns a unique_ptr to
///        a python function object that returns a raw pointer but
///        explicitly passes ownership to Python.
template <typename T,
          typename C,
          typename ...Args>
boost::python::object adapt_unique(std::unique_ptr<T> (C::*fn)(Args...))
{
  return boost::python::make_function(
      [fn](C& self, Args... args) { return (self.*fn)(args...).release(); },
      boost::python::return_value_policy<boost::python::manage_new_object>(),
      boost::mpl::vector<T*, C&, Args...>()
    );
}

The lambda delegates to the original function, and releases() ownership of the instance to Python, and the call policy indicates that Python will take ownership of the value returned from the lambda. The mpl::vector describes the call signature to Boost.Python, allowing it to properly manage function dispatching between the languages.

The result of adapt_unique is exposed as SpamFactory.make():

boost::python::class_<SpamFactory>(...)
  .def("make", adapt_unique(&SpamFactory::make))
  // ...
  ;

Generically adapting SpamFactory::consume() is a more difficult, but it is easy enough to write a simple auxiliary function:

/// @brief Wrapper function for SpamFactory::consume_spam().  This
///        is required because Boost.Python will pass a handle to the
///        Spam instance as an auto_ptr that needs to be converted to
///        convert to a unique_ptr.
void SpamFactory_consume(
  SpamFactory& self,
  std::auto_ptr<Spam> ptr) // Note auto_ptr provided by Boost.Python.
{
  return self.consume(std::unique_ptr<Spam>{ptr.release()});
}

The auxiliary function delegates to the original function, and converts the auto_ptr provided by Boost.Python to the unique_ptr required by the API. The SpamFactory_consume auxiliary function is exposed as SpamFactory.consume():

boost::python::class_<SpamFactory>(...)
  // ...
 .def("consume", &SpamFactory_consume)
 ;

---

Here is a complete code example:

#include <iostream>
#include <memory>
#include <boost/python.hpp>

/// @brief Mockup Spam class.
struct Spam
{
  Spam(std::size_t x) : x(x) { std::cout << "Spam()" << std::endl; }
  ~Spam() { std::cout << "~Spam()" << std::endl; }
  Spam(const Spam&) = delete;
  Spam& operator=(const Spam&) = delete;
  std::size_t x;
};

/// @brief Mockup factor for Spam.
struct SpamFactory
{
  /// @brief Create Spam instances.
  std::unique_ptr<Spam> make(const std::string& str)
  {
    return std::unique_ptr<Spam>{new Spam{str.size()}};
  }

  /// @brief Delete Spam instances.
  void consume(std::unique_ptr<Spam>) {}
};

/// @brief Adapter a non-member function that returns a unique_ptr to
///        a python function object that returns a raw pointer but
///        explicitly passes ownership to Python.
template <typename T,
          typename ...Args>
boost::python::object adapt_unique(std::unique_ptr<T> (*fn)(Args...))
{
  return boost::python::make_function(
      [fn](Args... args) { return fn(args...).release(); },
      boost::python::return_value_policy<boost::python::manage_new_object>(),
      boost::mpl::vector<T*, Args...>()
    );
}

/// @brief Adapter a member function that returns a unique_ptr to
///        a python function object that returns a raw pointer but
///        explicitly passes ownership to Python.
template <typename T,
          typename C,
          typename ...Args>
boost::python::object adapt_unique(std::unique_ptr<T> (C::*fn)(Args...))
{
  return boost::python::make_function(
      [fn](C& self, Args... args) { return (self.*fn)(args...).release(); },
      boost::python::return_value_policy<boost::python::manage_new_object>(),
      boost::mpl::vector<T*, C&, Args...>()
    );
}

/// @brief Wrapper function for SpamFactory::consume().  This
///        is required because Boost.Python will pass a handle to the
///        Spam instance as an auto_ptr that needs to be converted to
///        convert to a unique_ptr.
void SpamFactory_consume(
  SpamFactory& self,
  std::auto_ptr<Spam> ptr) // Note auto_ptr provided by Boost.Python.
{
  return self.consume(std::unique_ptr<Spam>{ptr.release()});
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<Spam, boost::noncopyable>(
      "Spam", python::init<std::size_t>())
    .def_readwrite("x", &Spam::x)
    ;

  python::class_<SpamFactory>("SpamFactory", python::init<>())
    .def("make", adapt_unique(&SpamFactory::make))
    .def("consume", &SpamFactory_consume)
    ;
}

Interactive Python:

>>> import example
>>> factory = example.SpamFactory()
>>> spam = factory.make("a" * 21)
Spam()
>>> spam.x
21
>>> spam.x *= 2
>>> spam.x
42
>>> factory.consume(spam)
~Spam()
>>> spam.x = 100
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Boost.Python.ArgumentError: Python argument types in
    None.None(Spam, int)
did not match C++ signature:
    None(Spam {lvalue}, unsigned int)

Answer 4

I think nowadays there is no way to do what you are looking for... The reason is because std::unique_ptr<Message> someFunc(const std::string &str) is returning by value, which means one of two things:

1. The return value is going to be copied (but unique_ptr is not copyable);
2. The return value is going to be moved (now the problem is that boost::python doesn't provide support to move semantics). (heyy, I'm using boost 1,53, not sure in the newest versions);

Is someFunc() creating the object? In case YES, I think the solution is to create a wrapper, in case NO, you can return by reference:

std::unique_ptr<Message>& someFunc(const std::string &str)

expose the class:

class_<std::unique_ptr<Message, std::default_delete<Message>>, boost::noncopyable>("unique_ptr_message")
    .def("get", &std::unique_ptr<Message>::get, return_value_policy<reference_existing_object>())
;

and also the functions:

def("someFunc", someFunc,  return_value_policy<reference_existing_object>());