Python: SWIG vs ctypes

Question

In python, under what circumstances is SWIG a better choice than ctypes for calling entry points in shared libraries? Let's assume you don't already have the SWIG interface file(s).

What are the performance metrics of the two?

Answer 1

I have a rich experience of using swig. SWIG claims that it is rapid solution for wrapping things. But in the real life...

---

Cons:

SWIG is developed to be general, for everyone and for 20+ languages. Generally it leads to drawbacks:
- needs configuration (SWIG .i templates), sometimes it is tricky,
- lack of treatment of some special cases (see python properties further),
- lack of the performance for some languages.

Python cons:

1) Code style inconsistency. C++ and python have very different code styles (that is obvious, certainly), the possibilities of swig of making target code more pythonish is very limited. As example, it is butt-heart to create properties from getters and setters. See this q&a

2) Lack of broad community. SWIG has some good documentation. But if one caught something that is not in the documentation, there is no information at all. No blogs nor googling helps. So one have to heavily dig SWIG generated code in such cases... That is terrible, I could say...

Pros:

• In simple cases it is really rapid, easy and straight forward

• If you produced swig interface files once, you can wrap this C++ code to ANY of other 20+ languages (!!!).

• One big concern about SWIG is a performance. Since version 2.04 SWIG includes '-builtin' flag wich makes SWIG even faster than other automated ways of wrapping. At least some benchmarks shows this.

---

When to USE SWIG?

So I concluded for myself two cases when the swig is good to use:

2) If one needs to wrap C++ code for several languages. Or if potentially there could be a time when one needs to distribute the code for several languages. Using SWIG is reliable in this case.

1) If one needs to rapidly wrap just several functions from some C++ library for end use.

---

Live experience

Update :
It is a year and half passed as we did a conversion of our library by using SWIG.

First we made a python version.There were several moments when we experienced troubles with SWIG - it is true. But right now we expanded our library to Java and .NET. So we have 3 languages with 1 SWIG. And I could say that SWIG rocks in terms of saving a LOT of time.

Update 2:
It is two years as we use SWIG for this library. SWIG is integrated in our build system. Recently we had major API change of C++ library. SWIG worked perfectly. The only thing we needed to do is to add several %rename to .i files so our CppCamelStyleFunctions() now looks_more_pythonish in python. First I was concerned about some problems that could arise, but nothing went wrong. It was amazing. Just several edits and everything distributed in 3 languages. Now I am confident that it was a good solution to use SWIG in our case.

Update 3:
It is 3+ years we use SWIG for our library. Major change: python part was totally rewritten in pure python. The reason is that python is used for the majority of applications of our library now. Even if pure python version works slower than C++ wrapping, it is more convenient for users to work with pure python, not struggling with native libraries.

SWIG is still used for .NET and Java versions.

The Main question here "Would we use SWIG for python, if we started the project from the beginning?". We would! SWIG allowed us to rapidly distribute our product to many languages. It worked for a period of time which gave us the opportunity for better understanding our users requirements.

Answer 2

You can also use Pyrex, which can act as glue between high-level Python code and low-level C code. lxml is written in Pyrex, for instance.

Answer 3

I have found SWIG to be be a little bloated in its approach (in general, not just Python) and difficult to implement without having to cross the sore point of writing Python code with an explicit mindset to be SWIG friendly, rather than writing clean well-written Python code. It is, IMHO, a much more straightforward process to write C bindings to C++ (if using C++) and then use ctypes to interface to any C layer.

If the library you are interfacing to has a C interface as part of the library, another advantage of ctypes is that you don't have to compile a separate python-binding library to access third-party libraries. This is particularly nice in formulating a pure-python solution that avoids cross-platform compilation issues (for those third-party libs offered on disparate platforms). Having to embed compiled code into a package you wish to deploy on something like PyPi in a cross-platform friendly way is a pain; one of my most irritating points about Python packages using SWIG or underlying explicit C code is their general inavailability cross-platform. So consider this if you are working with cross-platform available third party libraries and developing a python solution around them.

As a real-world example, consider PyGTK. This (I believe) uses SWIG to generate C code to interface to the GTK C calls. I used this for the briefest time only to find it a real pain to set up and use, with quirky odd errors if you didn't do things in the correct order on setup and just in general. It was such a frustrating experience, and when I looked at the interace definitions provided by GTK on the web I realized what a simple excercise it would be to write a translator of those interface to python ctypes interface. A project called PyGGI was born, and in ONE day I was able to rewrite PyGTK to be a much more functiona and useful product that matches cleanly to the GTK C-object-oriented interfaces. And it required no compilation of C-code making it cross-platform friendly. (I was actually after interfacing to webkitgtk, which isn't so cross-platform). I can also easily deploy PyGGI to any platform supporting GTK.

Answer 4

CTypes is very cool and much easier than SWIG, but it has the drawback that poorly or malevolently-written python code can actually crash the python process. You should also consider boost python. IMHO it's actually easier than swig while giving you more control over the final python interface. If you are using C++ anyway, you also don't add any other languages to your mix.