Deploying Django sites using RPM

Why?

Django is a great framework for developing websites but as with most projects there isn’t a particular focus on the system administration side of running a real site. There are great instructions describing the source-level changes you’ll want to make and what you’ll need to configure your webserver to do but … what about afterwards?

The process of deploying any site has a few basic steps: update the code, apply any database changes and reload the running site. How people choose to do this varies wildly but in the Python world it tends to involve a lot of manual work setting up the Python environment and running commands by hand or using a tool such as Fabric or Buildout to run those commands for you.

This approach works but it has a few drawbacks:

If you’re installing anything which uses a native library (images, databases, clients for things like memcache, etc.) you’ll need install a ton of extra dependencies on your production servers: gcc, development headers, etc.

It’s extremely slow compared to normal Linux software installation

You’re exposed to failures in outside services such as PyPI, Sourceforge, Github, Bitbucket, etc. This can be a problem if you can’t download the package and disastrous if the upstream source has updated to a newer version which you haven’t tested

Adding a second server requires you to duplicate everything, which is time consuming, and you then have to apply your changes in lockstep across every server to avoid requests being processed depending on which server handles the request

It’s hard to tell in advance what you’ll need to install an application unless you tediously compare the installed dependencies on a known-working server

You can have silent failures which only show up in production - the classic example being the Python Imaging Library, which can install “successfully” without optional components like JPEG support if it fails to detect them, which will only show up the first time someone attempts to use a JPEG file with your site.

None of these are new problems - in fact, the BSD and Linux communities have been working on package managers for the last decade or two, which is why you can install a brand new Linux system with hundreds of applications in less time than it takes to bring a large Python website up on a new server.

How

UPDATE: This has turned into a Github project and the latest version of these instructions are on Github pages.

Structuring the application

A well-behaved application is going to do a few things:

Install in a well-defined location not used by other applications - i.e. /opt/ my_app rather than /var/www/html.

rather than /var/www/html. Provide a clean way to customize your app with server-specific settings (e.g. file locations, database info, etc.) which doesn’t involve editing the packaged files - otherwise they’ll be overwritten on the next release. This also makes it easy to cleanly install the same package on development, test, and production servers.

For Django apps, I’m using the following conventions:

Everything installs in /opt/ my_app - this includes a virtualenv which is pre-loaded with our dependencies, avoiding the possibility of conflicts between projects. Want to have a Django 1.1 app installed on the same server as an older Django 1.0 app? This makes that easy.

- this includes a virtualenv which is pre-loaded with our dependencies, avoiding the possibility of conflicts between projects. Want to have a Django 1.1 app installed on the same server as an older Django 1.0 app? This makes that easy. Apache configuration is split into a separate common configuration (e.g. WSGI config, media expiration, etc) designed to be included by a server-specific file which specifies things like hostnames, SSL config, etc.: <VirtualHost example.org:80> ServerName example.org ErrorLog logs/ my_app .errors.log CustomLog logs/ my_app .access.log combined LogLevel info SSLEngine on SSLProtocol all -SSLv2 SSLCipherSuite ALL:!ADH:!EXPORT56:+HIGH:+SSLv3:+TLSv1 SSLCertificateFile "/etc/httpd/ssl.crt/ my_app .crt" SSLCertificateKeyFile "/etc/httpd/ssl.key/ my_app .key" Include /opt/ my_app /deploy/apache_common.conf </VirtualHost>

Django customization is managed by a local_settings.py file which is imported by settings.py. try: import local_settings except ImportError: logging.warning("No local settings - running in development mode") This is where you put things like database username and password, production email contact addresses, etc.

Building an RPM

Setup your RPM build environment Create your specfile (see below) in SPECS/my_site.spec Create source archives for everything you need to install: this be as simple as downloading a tarfile from the library provider or creating your own from your version control system: Git archive:

git archive --format=tar --prefix= my_app - 1.0 / my_app - 1.0 | gzip -9 > ~/rpmbuild/SOURCES/ my_app -1.0.tar.gz

Subversion:

svn export . /tmp/my_package

tar -C /tmp -cjf ~/rpmbuild/SOURCES/my_app.tar.bz2 Now you’re ready to compile the actual RPM:

rpmbuild -ba --clean SPECS/my_site.spec Install the RPM on your test server

If you want to see what files your RPM will install, use RPM’s query options: rpm -q --fileprovide -p RPMS/noarch/my_site.rpm

For future releases the process is simple: update the specfile if you’ve changed your dependencies (add, remove, change versions, etc.) and recompile.

Here’s an example project containing an RPM specfile and the general recommended site structure. There are a few key things you will want to customize: