Share Tweet Share





Per discussion on the pgsql-hackers mailing list this month, the PostgreSQL database really needs a new (or dramatically improved) Python driver. I have been only too aware of this due to the amount of Django work my team is doing lately. Let me sum up the situation so that you don’t have to comb all the way through that thread. Hopefully at least one person reading this will take it as a call to action. py-bpgsql and pg8000 are pure-Python drivers. As such, neither can perform well enough for a production environment, and as a result neither is production-quality nor is fully compatible existing popular frameworks and ORMs. py-postgresql is currently an experimental driver, designed by James Pye to implement a lot of his advanced ideas about drivers. It’s also Python3 only. So while it may be the driver of the future, it is not the driver of the present. PyGreSQL is our oldest driver, by Darcy Cain. It’s got the best licensing (MIT), but is old enough not to be DBAPI2-compatible. It also does not support Bytea, COPY, extensions, or all data types. Psycopg2 is the most popular driver. However, it’s under a bizarre hacked-up license, and the website for it has been unmaintained for the last couple years (even if the driver is still being updated and improved). Further, Psycopg has a number of issues resulting from trying to do too much, such as asynch code problems, transaction control (at least with Django) and using its own escaping instead of libpq’s. (Drivers who do too much and the developers that love them — 10pm, Lifetime) How did we get to this ugly pass? Well, a lot of people blame the extremely weak DBAPI2 specification for Python, which does not define enough interfaces and methods for a full-featured database driver. One can argue, as Marko does, that this specification is merely lean rather than being bloated, but the fact is that of the many programming languages which support PostgreSQL, only Python has this driver fragmentation problem that I know of (although JDBC has issues around over-specification, so one can go too far the other way). Rather than trying to change DBAPI2, though, the critical issue is to get one really good Python driver out there for PostgreSQL for people to use. The ideal driver would be:

MIT, BSD, or Apache-licensed

Works on both Python 2.x and 3

Wraps libpq and relies on it as much as possible

Uses C for speed optimizations

Has both Django and SQLAlchemy bindings

Support for COPY and Bytea streaming

Support for extensions, especially data types

Much more detail is available from the Python Driver TODO page which Greg Smith put up. Also, Jeff Davis has the PostgreSQL Driver development checklist he developed for our Ruby driver. So, what are you waiting for? Help us improve our Python drivers! Volunteer on the pgsql-hackers mailing list. Thanks to Greg Smith, Marko Kreen, James Pye, and Kevin for most of the information in this post.