I’m wrapping up a bug right now that has to do with translating python exceptions into C++ exceptions. The translation occurrs as follows:

Catch boost::python::error_already_set

Import python modules which contain the exception types that I want to compare against

Compare the thrown exception with the imported types using PyErr_ExceptionMatches

Take appropriate actions (e.g. throw a C++ exception) based on the match

This technique looks something like this:

using namespace boost::python; void main() { Py_Initialize(); try { do_something(); } catch (const error_already_set&) { object exc = import("my_module").attr("MyException"); if (PyErr_ExceptionMatches(exc.ptr())) throw MyException(); throw; } ... }

In the real code the import calls were only made the first time any translation was requested, but that that’s a minor detail.

Do you see the bug? If you do, I imagine that either this has happened to you before or you know a lot about the CPython’s implementation of module imports.

The import() call eventually calls PyImport_Import(), the recommended method for importing module programatically. This calls PyEval_GetGlobals() to get the global dict for the current execution frame. In my case, there apparently is no current execution frame, and this fact sends PyImport_Import down a branch where it calls…*drumroll*…PyErr_Clear().

What this means is that by the time I call PyErr_ExceptionMatches(), there is no active python exception. It has been silently erased. The simple (and, I think, correct) fix for this is to stash the exceptions before importing, import, and then restore the exceptions:

using namespace boost::python; void main() { Py_Initialize(); try { do_something(); } catch (const error_already_set&) { PyObject *e, *v, *t; PyErr_Fetch(&e, &v, &t); object exc = import("my_module").attr("MyException"); PyErr_Restore(e,v,t); if (PyErr_ExceptionMatches(exc.ptr())) throw MyException(); throw; } ... }

In the absence of any commentary in the python source, this behavior is a bit puzzling. There may well be a good reason for it, but there is no indication anywhere that I’ve read that PyImport_Import() might clear your error variables. This bears some looking into.

It took me quite a while to track this bug down for two reasons. First, I made the assumption that the bug must exist in my code. I spent all sorts of time looking at things like destructors triggered by unwinding, implicit conversions, and any other crazy thing I could think of that might somehow be triggering clearing of the errors. The one basic fact I had was that a) an error_already_set was reaching my catch and b) when it arrived, there was no active python exception. Based on what I knew of boost::python and CPython, plus my faith in the two, it seemed wise to look for bugs in my code first.

Second, and more egregiously, I was debugging with release versions of boost and python. In hindsight, this fact cost me more time than anything else. I say this because, as soon as I got debugging versions (unoptimized and with symbols), the bug had no place to hide and was easily found. A few well-placed breakpoints plus some stepping through the translation code almost immediately shined a light on boost::python::import(). After that, it was only a few more steps to PyImport_Import() and, ultimately, PyErr_Clear().

So, lesson learned.