How to pass list of structs to C in SWIG/Python

Question

I have a C++ class I'm exporting via swig, and a function that takes an array of Foos:

typedef class Foo {
  int i;
} Foo;

void func(Foo *all_foos);

Now I'd like to be able to pass a python list containing those into all_foos:

afoo = mymod.Foo()
bfoo = mymod.Foo()
mymod.func([afoo, bfoo])

I have a typemap which doesn't work. See the FIXME line for where I need help.

%typemap(in) Foo ** {
  /* Check if it's a list */
  if (PyList_Check($input)) {
    int size = PyList_Size($input);
    int i = 0;
    $1 = (Foo **) malloc((size+1)*sizeof(Foo *));
    for (i = 0; i < size; i++) {
      PyObject *o = PyList_GetItem($input,i);
      // here o->ob_type->tp_name is "Foo"; could check that
      // FIXME: HOW DO I GO FROM o -> SwigPyObject -> Foo *?  THIS IS WRONG
      $1[i] = (Foo *)(reinterpret_cast<SwigPyObject *>(o))->ptr;
    }
  } else {
    PyErr_SetString(PyExc_TypeError,"not a list");
    return NULL;
  }
}

Basically, I have a PyObject o; I need to get the SwigPyObject from it (do I just cast it? Or is it a member?) and then get my Foo pointer from the SwigPyObject somehow.

Answer 1

Your example is a little confused because your C++ function takes Foo*, but your typemap takes Foo** (i.e. array of Foos vs array of pointer to Foos). I'm assuming you meant the latter, because that's the only sane way to tell how long the array is from the function declaration you've given.

In terms of the immediate question "how can I convert Python objects to C++ pointers of a given type?" I usually solve that question by letting SWIG generate some code for me and then inspecting it. So for example if you have a function void bar(Foo*); then SWIG will generate some code in the wrapper:

SWIGINTERN PyObject *_wrap_bar(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  Foo *arg1 = (Foo *) 0 ;
  void *argp1 = 0 ;
  int res1 = 0 ;
  PyObject * obj0 = 0 ;

  if (!PyArg_ParseTuple(args,(char *)"O:bar",&obj0)) SWIG_fail;
  res1 = SWIG_ConvertPtr(obj0, &argp1,SWIGTYPE_p_Foo, 0 |  0 );
  if (!SWIG_IsOK(res1)) {
    SWIG_exception_fail(SWIG_ArgError(res1), "in method '" "bar" "', argument " "1"" of type '" "Foo *""'");
  }
  arg1 = reinterpret_cast< Foo * >(argp1);
  bar(arg1);
  resultobj = SWIG_Py_Void();
  return resultobj;
fail:
  return NULL;
}

The interesting bit of that is the call to SWIG_ConvertPtr which is doing what you want. With that knowledge we just need to put it inside the loop that you've already written for your typemap, so your 'in' typemap becomes:

%typemap(in) Foo ** {
  $1 = NULL;
  if (PyList_Check($input)) {
    const size_t size = PyList_Size($input);
    $1 = (Foo**)malloc((size+1) * sizeof(Foo*));
    for (int i = 0; i < size; ++i) {
      void *argp = 0 ;
      const int res = SWIG_ConvertPtr(PyList_GetItem($input, i), &argp, $*1_descriptor, 0);
      if (!SWIG_IsOK(res)) {
        SWIG_exception_fail(SWIG_ArgError(res), "in method '" "$symname" "', argument " "$argnum"" of type '" "$1_type""'");
      }
      $1[i] = reinterpret_cast<Foo*>(argp);
    }
    $1[size] = NULL;
  }
  else {
    // Raise exception
    SWIG_exception_fail(SWIG_TypeError, "Expected list in $symname");
  }
}

Notice that in the interests of making the typemap generic and reusable I've replaced parts of it with special typemap variables - the same code gets generated as for the single example we saw, but you can reuse it slightly more.

That's sufficient to compile and run the example code you gave (with the one change as noted), however there's a memory leak still. You call malloc(), but never free(), so we need to add a corresponding 'freearg' typemap:

%typemap(freearg) Foo ** {
  free($1);
}

This gets called both on success and error, but that's fine because $1 is initalised to NULL, so the behavior is correct regardless of if we successfully malloc or not.

---

As a general point since this is C++ I think your interface is designed wrong - there's no good reason not to use std::vector, std::list or similar which also makes the wrapping simpler. It's also weird style to use typedefs like you have in your header file.

If it were me writing this though, I'd be looking to use RAII in the wrapper, even if I couldn't change the interface to use a container. So that means I'd write my 'in' typemap as:

%typemap(in) Foo ** (std::vector<Foo*> temp) {
  if (PyList_Check($input)) {
    const size_t size = PyList_Size($input);
    temp.resize(size+1);
    for (int i = 0; i < size; ++i) {
      void *argp = 0 ;
      const int res = SWIG_ConvertPtr(PyList_GetItem($input, i), &argp, $*1_descriptor, 0);
      if (!SWIG_IsOK(res)) {
        SWIG_exception_fail(SWIG_ArgError(res), "in method '" "$symname" "', argument " "$argnum"" of type '" "$1_type""'");
      }
      temp[i] = reinterpret_cast<Foo*>(argp);
    }
    temp[size] = NULL;
    $1 = &temp[0]; // Always valid since we +1
  }
  else {
    // Raise exception
    SWIG_exception_fail(SWIG_TypeError, "Expected list in $symname");
  }
}

which then removes the need for a 'freearg' typemap and never leaks.

---

If for some reason you don't want to write a custom typemap, or change the interface to use a type that already has good typemaps in the SWIG library you can still give your Python users an intuitive pythonic interface by using %rename to 'hide' the default implementation and %pythoncode to inject some additional Python with the same name that "massages" the Python input into the carrays interface transparently to the Python user.

Answer 2

I would try with carrays.i, something like

%include carrays.i
%array_class(Foo, FooArray)

void func(Foo *all_foos);

Then in python:

fooArray = FooArray(10)
func(fooArray)

Typemaps should only be used if there is no other SWIG tool to achieve the API you want. I think it might even be possible to do better than the above, with %inline (separate answer because very different from this).

Answer 3

You should be able to do everything with standard "front end" SWIG tools:

%include <std_list.i>

%ignore func
%rename(func) funcWrap

namespace std {
   %template(FooList) std::list<Foo*>;
}

%include "Foo.h"

%inline %{
    // wrap with const list of non-const Foo*
    void funcWrap(const std::list<Foo *>& all_foos)
    {
         // create Foo* array: 
         Foo* fooArray = new Foo(all_foos.size());
         int count = 0;
         for (std::list<Foo *>::const_iterator ii=all_foos.begin(); ...) 
              fooArray[count] = *ii;
         func(fooArray);
         delete fooArray;
    }
%}

There shouldn't be need for typemap here.