How I learned to stop worrying and love python packaging - Jannis Leidel (djangocon.eu)¶

Regular python packaging (in our case in combination with buildout) is something we use a lot. So a talk on this at the djangocon.eu is something I look upon with a favourable eye. (I’ll even take the opportunity to point you at my software releases series.)

The past of python packaging¶ Some terms are ambiguous: A python package is a directory with python files and an __init__.py . This is a module.

A release is something like “django 1.3”.

A distribution is a source/binary form of a release, though this is often called a package. It started out with distutils: a standard way for building, distributing and installing python modules. Especially also C extensions. It installs into a library directory. It wasn’t completely up to the task, so several extensions showed up. Some of those came with a PEP, for instance PEP 241 for package metadata such as “version”, “description”, “author”. And also now there are some PEPs that are currently being implemented. There’s continuous work in this area.

Common pitfalls and gotchas and tips¶ The basis for packaging is a setup.py file. A problem with this is that such a setup file is used for a couple of dissimilar tasks: Generating documentation.

Creating a release (source/binary).

Actually installing the software. Combining these tasks in one file is a bit of a hazard and leads to problems now and then. Tip regarding documentation: add a so-called “long description” by concatenating the readme and changelog and other documentation files into the description field in your setup.py. This is rendered as restructured text on the pypi page if you upload it. List all additional non-python files that you need in package_data , otherwise they won’t get installed. Likewise, add a MANIFEST.in that lists all your files.

Everyday software development¶ Use sane versions. 1.2 , 1.3.2 , not .000001 or unreleased.unofficialdev . And use release names in any prose instead of version numbers: release “batman” is nicer than release “1.2”. Regarding regular package management versus python-only installations? Just use your regular package management (“apt-get install xyz”) in case it are binary packages. They have dependencies and header file requirements that are best handled globally by the OS (PIL, lxml, database adapters, etc.) Install often-updated python-only packages locally, on the other hand. Don’t install them globally. Isolate with, for instance, virtualenv. Tip when you’re using virtualenv: http://www.doughellmann.com/docs/virtualenvwrapper/ . Combine virtualenv with pip for sane installation and uninstalling. It can also “freeze” and “unfreeze” your requirements and your versions. You can just copy a text file to another machine and re-create your environment there. Pip also has automatic support for pypi mirrors (the http://d.pypi.python.org/ ones). You can configure pip to globally cache your downloaded packages, handy in case you’re without internet. You can also make your own simple package index with apache’s DirectoryIndex . There’s also a pypi clone made with django: Chishop.