Pythonic wrapping using

Olly Betts

Me

SWIG

• Automates wrapping of C and C++ code
• Supports many backends - P*rl, R*by, J*v*, G*, ***, etc
• Most alternatives seem language-specific (Boost.Python, ctypes)

Overview

Basics

Example

namespace Xapian {
  int major_version();
  const char * version_string();
}

>>> import xapian
>>> xapian.major_version()
1
>>> xapian.version_string()
'1.2.5'

Ambiguous Cases

Ambiguous Cases

Same syntax, different semantics:

void set_entry(const char * name, unsigned value);

>>> set_entry('answer', 42)

Ambiguous Cases

Same syntax, different semantics:

void set_entry(const char * name, unsigned value);

>>> set_entry('answer', 42)

void write_data(const char * ptr, unsigned len);

Ambiguous Cases

Same syntax, different semantics:

void set_entry(const char * name, unsigned value);

>>> set_entry('answer', 42)

void write_data(const char * ptr, unsigned len);

>>> write_data('hello\x00world', 11)

Ambiguous Cases

Same syntax, different semantics:

void set_entry(const char * name, unsigned value);

>>> set_entry('answer', 42)

void write_data(const char * ptr, unsigned len);

>>> data = 'hello\x00world'
>>> write_data(data, len(data))

Ambiguous Cases

Same syntax, different semantics:

void set_entry(const char * name, unsigned value);

>>> set_entry('answer', 42)

void write_data(const char * ptr, unsigned len);

%typemap(in) (const char * ptr, unsigned len) {
  $1 = PyString_AsString($input);
  $2 = PyString_Size($input);
}

>>> write_data('hello\x00world')

Renaming

namespace Xapian {
  class Stem {
    // [...]
    std::string get_description() const;
  };
}

>>> stem = xapian.Stem('en')
>>> stem.get_description()
'Xapian::Stem(english)'

Renaming

%rename(__str__) Xapian::Stem::get_description;

namespace Xapian {
  class Stem {
    // [...]
    std::string get_description() const;
  };
}

>>> stem = xapian.Stem('en')
>>> str(stem)
'Xapian::Stem(english)'

Renaming

%rename(__str__) get_description;

namespace Xapian {
  class Stem {
    // [...]
    std::string get_description() const;
  };
}

>>> stem = xapian.Stem('en')
>>> str(stem)
'Xapian::Stem(english)'

Pattern renames

Remove a pseudo-namespace prefix from all functions

%rename("%(regex/^mylib_//)s",$isfunction) "";

Style renames

Convert to match PEP8 recommendations:

%rename("%(undercase)s",%$isfunction) "";

%rename("%(camelcase)s",%$isclass) "";

Ignoring features

%ignore max_size;

Exceptions

%exception {
  [...]
  $action
  [...]
}

Translating C++ Exceptions to Python

%exception {
  try {
    $action
  } catch (std::range_error & e) {
    SWIG_exception(SWIG_IndexError, e.what());
  }
}

Throwing wrapped objects in Python

%exception {
  try {
    $action
  } catch (const Xapian::Error & e) {
    PyObject * o;
    o = SWIG_NewPointerObj(new Xapian::Error(e),
			   SWIGTYPE_p_Xapian__Error,
			   SWIG_POINTER_OWN);
    SWIG_Python_Raise(o,
		      "Xapian::Error",
		      SWIGTYPE_p_Xapian__Error);
    SWIG_fail;
  }
}

Refactored for smaller glue code

static void SetPythonException() {
  try {
    throw;
  } catch (const Xapian::Error & e) {
    // Call SWIG_Python_Raise as before.
  }
}

%exception {
  try {
    $action
  } catch (...) {
    SetPythonException();
    SWIG_fail;
  }
}

Error codes to exceptions

%exception {
  $action
  if (result < 0) {
    SWIG_exception(SWIG_IndexError, strerror(errno));
  }
}

Subclassing

"Directors" allow wrapped classes to be subclassed, and virtual methods get routed as you'd hope.

Dynamic typing example

Dynamic typing example

%typemap(in) const vector<Xapian::Query> &
    (vector<Xapian::Query> v) {
  PyObject * fastseq = PySequence_Fast($input, "bad");
  if (!fastseq) SWIG_fail;

  int num_items = PySequence_Fast_GET_SIZE(fastseq);
  v.reserve(num_items);
  for (int i = 0; i < num_items; ++i) {
    // Loop body on next slide...
  }
  $1 = &v;
  Py_DECREF(fastseq);
}

Dynamic typing example continued

PyObject *obj =
    PySequence_Fast_GET_ITEM(fastseq, i);
if (PyString_Check(obj)) {
  char * p;
  Py_ssize_t len;
  (void)PyString_AsStringAndSize(obj, &p, &len);
  v.push_back(Xapian::Query(string(p, len)));
} else {
  Xapian::Query *subqp = Xapian::get_py_query(obj);
  if (!subqp)
    SWIG_exception(SWIG_TypeError, "bad");
  v.push_back(*subqp);
}

Adding extra methods to wrap

%extend Xapian::Query {
  Query(Query::op op, const vector<Query> & v) {
    return new Xapian::Query(op, v.begin(), v.end());
  }
}

Multiple return values

int decode(const char *s, int & width, int & height);

>>> decode(input)
(10, 100)

Iterators

(Mostly) Automatically Wrapped C++ Iterator

i = matches.begin()
while i != matches.end():
    print i.get_docid()
    i.next()

With a little work

for m in matches:
    print m.docid

Docstrings

Generated by SWIG

%feature("autodoc");
%feature("autodoc", 0);

Overridden

%feature("autodoc", "GetPosition() -> (x, y)") GetPosition;
void GetPosition(int* OUTPUT, int* OUTPUT);

Additional

%feature("docstring", "This is some more info.") GetPosition;

Translating documentation comments

• For Xapian, we generate %feature("docstring", ...) using doxy2swig.py
• GSoC project to support something similar inside SWIG (not merged so far)

Summary

• Most C/C++ API features wrapped in a usable way automatically
• With more effort, you can get a more natural wrapped API
• This was just a brief stroll through the SWIG landscape

The End

Links

• Email: olly@survex.com
• SWIG Website
• Xapian (see xapian-bindings for full versions of examples)