This blog post discusses a Python 3 approach that's heavily centered on the 2to3 tool. These days, I'm much more in favor of the "in place" approach, even if still supporting as far back as Python 2.4. Mako 0.7.4 is now an "in place" library, supporting Python2.4-3.x with no changes. For a good introduction to the "in place" approach, see Supporting Python 2 and 3 without 2to3 conversion .

Just the other day, Ben asked me, "OK, where is there an online HOWTO of how to port to Python 3?". I hit the Google expecting to see at least three or four blog posts with the basic steps, an overview, the things Guido laid out for us at Pycon '10 (and maybe even '09 ? don't remember). Surprisingly, other than the link to the 2to3 tool and Guido's original guide, there aren't a whole lot.

So here are my steps which I've used to produce released versions of SQLAlchemy and Mako on Pypi which are cross-compatible for Py2k and Py3k.

So we fix all those things. If our code needs to support old versions of Python as well, like 2.3 or 2.4, we may have to use runtime version and/or library detection for some things - as an example, Python 2.4 doesn't have collections.Callable . More on that later. For now let's assume we can get our whole test suite to pass without any warnings with the -3 flag.

Run your test suite with Python 2.6 or 2.7, using the -3 flag. Make sure there's no warnings. Such as, using the following ridiculous program:

2. Run the whole library through 2to3 and see how we do

This is the step we're all familiar with. Run the 2to3 tool to get a first pass. Such as, when I run the 2to3 tool on Mako:

classics-MacBook-Pro:mako classic$ 2to3 mako/ test/ -w

2to3 dumps out to stdout everything it's doing, and with the -w flag it also rewrites the files in place. I usually do a clone of my source tree to a second, scratch tree so that I can make alterations to the original, Py2K tree as I go along, which remains the Python 2.x source that gets committed.

It's typical with a larger application or library that some things, or even many things, didn't survive the 2to3 process intact.

In the case of SQLAlchemy, along with the usual string/unicode/bytes types of issues, we had problems regarding the name changes of iteritems() to items() and itervalues() to values() on dictionary types - some of our custom dictionary types would be broken. When your code produces no warnings with -3 and the 2to3 tool is still producing non-working code, there are three general approaches towards achieving cross-compatibility, listed here from lowest to highest severity.

2a. Try to replace idioms that break in Py3K with cross-version ones Easiest is if the code in question can be modified so that it works on both platforms, as run through the 2to3 tool for the Py3k version. This is generally where a lot of the bytes/unicode issues wind up. Such as, code like this: hexlify ( somestring ) ...doesn't work in Py3k, hexlify() needs bytes . So a change like this might be appropriate: hexlify ( somestring . encode ( 'utf-8' )) or in Mako, the render() method returns an encoded string, which on Py3k is bytes . A unit test was doing this: html_error = template . render () assert "RuntimeError: test" in html_error We fixed it to instead say this: html_error = template . render () assert "RuntimeError: test" in str ( html_error )

2b. Use Runtime Version Flags to Handle Usage / Library Incompatibilities SQLAlchemy has a util package which includes code similar to this: import sys py3k = sys . version_info >= ( 3 , 0 ) py3k_flag = getattr ( sys , 'py3kwarning' , False ) py26 = sys . version_info >= ( 2 , 6 ) jython = sys . platform . startswith ( 'java' ) win32 = sys . platform . startswith ( 'win' ) This is basically getting some flags upfront that we can use to select behaviors specific to different platforms. Other parts of the library can say from sqlalchemy.util import py3k if we need to switch off some runtime behavior for Py3k (or Jython, or an older Python version). In Mako we use this flag to do things like switching among 'unicode' and 'str' template filters: if util . py3k : self . default_filters = [ 'str' ] else : self . default_filters = [ 'unicode' ] We use it to mark certain unit tests as unsupported ( skip_if() is a decorator we use in our Nose tests which raises SkipTest if the given expression is True ): @skip_if ( lambda : util . py3k ) def test_quoting_non_unicode ( self ): # ... For our previously mentioned issue with callable() (which apparently is coming back in Python 3.2), we have a block in SQLAlchemy's compat.py module like this, which returns to us callable() , cmp() , and reduce() : if py3k : def callable ( fn ): return hasattr ( fn , '__call__' ) def cmp ( a , b ): return ( a > b ) - ( a < b ) from functools import reduce else : callable = __builtin__ . callable cmp = __builtin__ . cmp reduce = __builtin__ . reduce