Building a Django App Server with Chef: Part 3¶

Alternate title: Show the world what you’ve got.

This is Part 3 of my Chef tutorial. Today we’re talking about deployment. You can check out the first 2 parts of the series:

Today’s code will be in the git repo under the tag blog-post-3.

What we’ll need¶ We’ll be taking the Django application that we have on the server and actually deploying it. Let’s make a list of what we’ll need: A web server to sit in front and proxy requests

A WSGI server

A way to keep both of these things running

A caching layer We’ll be using Nginx, Memcached, Upstart, and Gunicorn. This is my preferred deployment stack as of late, mainly because of the simple setup. Let’s get started

A web server¶ Getting Nginx up and running should be old hat by this point. We’re going to need the package and service Resources, which will tell Chef to install and run it. cookbooks/main/recipes/nginx.rb package "nginx" do : upgrade end service "nginx" do enabled true running true supports : status => true , : restart => true , : reload => true action [: start , : enable ] end cookbook_file "/etc/nginx/sites-enabled/readthedocs" do source "nginx/readthedocs" mode 0640 owner "root" group "root" notifies : restart , resources (: service => "nginx" ) end cookbook_file "/etc/nginx/nginx.conf" do source "nginx/nginx.conf" mode 0640 owner "root" group "root" notifies : restart , resources (: service => "nginx" ) end As you can see, we’re providing our own nginx.conf and a readthedocs site configuration. I’m not going to paste these in, as they are pretty application specific, but you can look at them on Github if you’re curious. I also wrote about it a while back. The only new part here is the notifies command, which is pretty nifty. It basically means that whenever you change the nginx.conf file, it should restart Nginx, which is a really nice feature.

A WSGI Server¶ Yesterday, when we installed the deploy_requirements.txt with pip, it pulled in gunicorn. So we actually have Gunicorn already installed in our virtualenv, waiting for us to use it. The only difference is I actually committed a change to the ReadTheDocs source so that it will pull Gunicorn from the git master, which I’ll explain below.

Upstart¶ Note: I use upstart because it ships with Ubuntu, so you don’t need to install a separate package. However, it has pretty horrible documentation, with the Stanzas doc probably the best clue as to what it supports. Here is where things get interesting. I spent a bunch of time trying to get gunicorn and upstart to play nicely yesterday night, but it wasn’t working. I went on the #gunicorn IRC channel on Freenode today, and talked with benoitc. He was awesome and patched gunicorn to work with Upstart for me. Here is the upstart script that we’re using to keep gunicorn running: cookbooks/main/files/default/gunicorn.conf description "Gunicorn for ReadTheDocs" start on runlevel [2345] stop on runlevel [!2345] #Send KILL after 5 seconds kill timeout 5 respawn env VENV="/home/docs/sites/readthedocs.org" #Serve Gunicorn on the internal rackspace IP. script exec sudo -u docs $VENV/bin/gunicorn_django --preload -w 2 --log-level debug --log-file $VENV/run/gunicorn.log -p $VENV/run/gunicorn.pid -b 10.177.69.207:8888 $VENV/checkouts/readthedocs.org/settings/postgres.py end script As you can see, an Upstart script is a pretty clean way to do this. If you’ve ever tried to write an old SysV-style init script, this will look beautiful. You’ll notice that we aren’t passing the –daemon parameter to gunicorn, this is because upstart will background the process for us, and keep track of everything, so we don’t need gunicorn’s daemonizing behavior. It should be pointed out how awesome it is that we can run a production ready WSGI server with a single line of bash. If you’ve ever set up a mod_wsgi install, needing to fuddle with your apache.conf and a WSGI file and everything makes it a chore. This is quite simply the easiest way to deploy a WSGI application. Then we need some additions to cookbooks/main/recipes/readthedocs.rb : cookbook_file "/etc/init/readthedocs-gunicorn.conf" do source "gunicorn.conf" owner "root" group "root" mode 0644 end service "readthedocs-gunicorn" do provider Chef :: Provider :: Service :: Upstart enabled true running true supports : restart => true , : reload => true , : status => true action [: enable , : start ] end Here you can see we’re doing a similar thing to the other service declarations. We however need to tell Chef to use Upstart for this service, instead of defaulting to init.d. Other than that, everything here should look similar to the other files and services we’ve set up.

Memcached¶ As you would expect, installing memcached is just like nginx: cookbooks/main/recipes/memcached.rb package "memcached" do : upgrade end service "memcached" do enabled true running true supports : status => true , : restart => true action [: enable , : start ] end cookbook_file "/etc/memcached.conf" do source "memcached.conf" mode 0640 owner "root" group "root" notifies : restart , resources (: service => "memcached" ) end The memcached.conf is so short, I might as well include it here: - d logfile / var / log / memcached . log - m 64 - p 11211 - u nobody - l 127.0 . 0.1 Memcache’s config file is pretty neat, because it’s basically just a list of arguments to pass to the daemon when it’s started. A little bit like a pip requirements file is just commands to pass to pip install when it’s run.