Caching pages, or even portions of pages is one of the easier ways to speed up a website's performance. Minifying markup can help save space in memory-based data stores. Using a CDN means your users can potentially download the content closer to where they are.

In this post I'll walk through performing all the above in a Django project.

Dependencies and boilerplate The principle orchestrating package being used for crushing is django-compressor. It's a ~5-year-old project that has had 233 closed tickets and 106 contributors. It uses a lot of other packages to crush various scripts (hence the long requirements list below). Other packages that I'll use include django-redis-cache for cache storage communications and for HTML minification I'll use django-htmlmin. The STATIC_DEPS=true prefix below is for lxml. $ sudo apt install \ libxml2-dev \ libxslt-dev \ python-dev \ libzip-dev \ redis-server $ STATIC_DEPS = true pip install \ Django == 1 .7.1 \ django-compressor == 1 .4 \ django-appconf == 0 .6 \ BeautifulSoup == 3 .2.1 \ html5lib == 0 .999 \ slimit == 0 .8.1 \ lxml == 3 .4.0 \ django-redis-cache == 0 .13.0 \ hiredis == 0 .1.5 \ django-htmlmin == 0 .7.0 As of this writing Ubuntu's repositories finally have the latest, stable version of redis, 2.8.17 in this case: $ redis-server --version Redis server v = 2 .8.17 sha = 00000000 :0 malloc = jemalloc-3.6.0 bits = 64 build = 64186bb5bffe2061 I've created a new project called 'compressed', created an app called 'coupon' within it, setup the boilerplate database and finally, downloaded Twitter Bootstrap and an example JPEG to an external folder. This folder will be one of the static folder's sources of content. $ django-admin startproject compressed $ cd compressed/ $ django-admin startapp coupon $ mkdir templates static external $ python manage.py syncdb --noinput $ curl -O https://github.com/twbs/bootstrap/releases/download/v3.3.1/bootstrap-3.3.1-dist.zip $ unzip bootstrap-3.3.1-dist.zip $ mv dist/* external/ $ rm -r bootstrap-3.3.1-dist.zip dist $ mkdir external/img $ curl -o external/img/mark.jpg \ http://tech.marksblogg.com/theme/images/mark.jpg $ touch templates/base.html \ external/css/app.css \ external/js/app.js I've also created external/css/app.css and external/js/app.js for some small, project-specific code. The file and folder layout of this project now looks like this: $ tree . ├── compressed │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── coupon │ ├── admin.py │ ├── __init__.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py ├── db.sqlite3 ├── external │ ├── css │ │ ├── app.css │ │ ├── bootstrap.css │ │ ├── bootstrap.css.map │ │ ├── bootstrap.min.css │ │ ├── bootstrap-theme.css │ │ ├── bootstrap-theme.css.map │ │ └── bootstrap-theme.min.css │ ├── fonts │ │ ├── glyphicons-halflings-regular.eot │ │ ├── glyphicons-halflings-regular.svg │ │ ├── glyphicons-halflings-regular.ttf │ │ └── glyphicons-halflings-regular.woff │ ├── img │ │ └── mark.jpg │ └── js │ ├── app.js │ ├── bootstrap.js │ ├── bootstrap.min.js │ └── npm.js ├── manage.py ├── static └── templates └── base.html

The codebase In compressed/urls.py I mapped all URLs to a single view and added in a static content endpoint for times when the code is run in debug mode. When running on production I'll change STATIC_URL to a CDN endpoint stub. from django.conf import settings from django.conf.urls import patterns , include , url from django.conf.urls.static import static from coupon import views urlpatterns = patterns ( '' , url ( r '^$' , views . home , name = 'index' ), ) if settings . DEBUG : urlpatterns += static ( settings . STATIC_URL , document_root = settings . STATIC_ROOT ) I created a unit test for the single view. I didn't patch the cache file generation as I'll use the test to generate the cache files on a continuous integration server. With those files generated, I'll deploy them to a CDN. A cleaner way to approach this would be to patch the unit tests so they don't create file artefacts and use a management command to generate the static cache files. coupon/tests.py: from django.core.urlresolvers import reverse from django.test import TestCase from django.test.client import Client class ViewTest ( TestCase ): def setUp ( self ): self . client = Client () def test_index ( self ): resp = self . client . get ( reverse ( 'index' )) self . assertEqual ( resp . status_code , 200 ) The single view in coupon/views.py renders a template and returns it. from django.shortcuts import render def home ( request ): return render ( request , 'base.html' ) I created a templates/base.html file which is mostly an example file from the bootstrap project. I added some template tags for identifying sections of markup to minify and wrapped asset URLs with a static template tag so their path prefix can be controlled via settings.STATIC_URL . {% load staticfiles %} {% load compress %} <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Bootstrap 101 Template</title> {% compress css %} <link href=" {% static "css/bootstrap.min.css" %} " rel="stylesheet"> <link href=" {% static "css/app.css" %} " rel="stylesheet"> {% endcompress %} <!--[if lt IE 9]> <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> <![endif]--> </head> <body> <h1>Hello, world!</h1> <img src=" {% static "img/mark.jpg" %} " /> <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script> {% compress js %} <script src=" {% static "js/bootstrap.min.js" %} "></script> <script src=" {% static "js/app.js" %} "></script> {% endcompress %} </body> </html> The compressed/settings.py file is the longest and probably most complicated of the project. Here is a summary of the key settings being put in place: Fetch the SECRET_KEY environment variable.

environment variable. Add compressor to the installed apps tuple.

to the installed apps tuple. Add caching and minification middleware.

Use redis as a caching backend (using database #3).

Fetch the STATIC_URL environment variable and fall-back to using /static/ as the default value if the environment variable is unavailable.

environment variable and fall-back to using as the default value if the environment variable is unavailable. Define the minification behaviours, key among them: KEEP_COMMENTS_ON_MINIFYING which will keep HTML comments in place so we can use conditional statements and COMPRESS_CSS_HASHING_METHOD which will cause our cache file names to be a hash of their respective contents. import os BASE_DIR = os . path . dirname ( os . path . dirname ( __file__ )) SECRET_KEY = os . environ . get ( 'SECRET_KEY' , None ) assert SECRET_KEY is not None , \ 'SECRET_KEY environment variable is needed' DEBUG = True TEMPLATE_DEBUG = True INSTALLED_APPS = ( 'django.contrib.auth' , 'django.contrib.contenttypes' , 'django.contrib.sessions' , 'django.contrib.messages' , 'django.contrib.staticfiles' , 'compressor' , ) MIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware' , 'htmlmin.middleware.HtmlMinifyMiddleware' , 'django.contrib.sessions.middleware.SessionMiddleware' , 'django.middleware.common.CommonMiddleware' , 'django.middleware.csrf.CsrfViewMiddleware' , 'django.contrib.auth.middleware.AuthenticationMiddleware' , 'django.contrib.auth.middleware.SessionAuthenticationMiddleware' , 'django.contrib.messages.middleware.MessageMiddleware' , 'django.middleware.clickjacking.XFrameOptionsMiddleware' , 'django.middleware.cache.FetchFromCacheMiddleware' , 'htmlmin.middleware.MarkRequestMiddleware' , ) ROOT_URLCONF = 'compressed.urls' WSGI_APPLICATION = 'compressed.wsgi.application' DATABASES = { 'default' : { 'ENGINE' : 'django.db.backends.sqlite3' , 'NAME' : os . path . join ( BASE_DIR , 'db.sqlite3' ), } } CACHES = { 'default' : { 'BACKEND' : 'redis_cache.RedisCache' , 'LOCATION' : '127.0.0.1:6379' , 'OPTIONS' : { 'DB' : 3 , 'PARSER_CLASS' : 'redis.connection.HiredisParser' , 'CONNECTION_POOL_CLASS' : 'redis.BlockingConnectionPool' , 'CONNECTION_POOL_CLASS_KWARGS' : { 'max_connections' : 25 , 'timeout' : 4 , } }, }, } LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True USE_TZ = True STATIC_URL = os . environ . get ( 'STATIC_URL' , '/static/' ) assert STATIC_URL is not None and len ( STATIC_URL ), \ 'STATIC_URL environment variable is needed' STATICFILES_DIRS = ( os . path . join ( BASE_DIR , "external" ), ) STATIC_ROOT = os . path . join ( BASE_DIR , "static" ) STATICFILES_FINDERS = ( 'django.contrib.staticfiles.finders.FileSystemFinder' , 'django.contrib.staticfiles.finders.AppDirectoriesFinder' , 'compressor.finders.CompressorFinder' , ) TEMPLATE_DIRS = [ os . path . join ( BASE_DIR , 'templates' )] COMPRESS_ENABLED = True COMPRESS_CSS_HASHING_METHOD = 'content' COMPRESS_CSS_FILTERS = [ 'compressor.filters.css_default.CssAbsoluteFilter' , 'compressor.filters.cssmin.CSSMinFilter' , ] HTML_MINIFY = True KEEP_COMMENTS_ON_MINIFYING = True The caching system is setup to work on a per-view basis but template fragments are also supported in Django and can be useful on pages with dynamic content. Finally, to help demonstrate asset concatenation, I've added the following frontend files: external/css/app.css: h1 { display : none ; text-align : center ; } external/js/app.js: $ ( document ). ready ( function () { $ ( 'h1' ). fadeIn ({ 'duration' : 2000 }) });

Initialising Django When setting environment variables I like to use read . It will allow you to type or paste in values for environment variables. This means those values won't appear when you run history . $ read SECRET_KEY 0x $01 e345$ ) v74-!k9gfdzhmybw0 = -0g+1jh@v#8- = s+^_ $ export SECRET_KEY $ history | tail ... 1320 read SECRET_KEY 1321 export SECRET_KEY With the settings in place I've run collectstatic which will take static files from our external folder and any applications (such as Django's admin) and place them in the static folder. $ python manage.py collectstatic --noinput Copying '/home/mark/compressed/external/img/mark.jpg' Copying '/home/mark/compressed/external/css/bootstrap.min.css' ... Copying '/home/mark/compressed/external/fonts/glyphicons-halflings-regular.svg' 16 static files copied to '/home/mark/compressed/static' .

Making a web request I started up the reference WSGI server, cleaned redis' database #3 and fetched the project's homepage: $ python manage.py runserver & $ redis-cli -n 3 flushdb OK $ curl --silent localhost:8000 | fold -w50 <!DOCTYPE html> < html lang = "en" >< head >< meta charset = "utf-8" />< meta content = "IE=edge" http-equiv = "X-UA -Compatible" />< meta content = "width=device-width, i nitial-scale=1" name = "viewport" />< title > Bootstrap 101 Template </ title >< link href = "/static/CACHE/css/ 1afa57f03e30.css" rel = "stylesheet" type = "text/css" /> <!--[if lt IE 9]><script src="https://oss.maxcdn .com/html5shiv/3.7.2/html5shiv.min.js"></script> < script src="https://oss.maxcdn.com/respond/1.4.2/r espond.min.js"></script><![endif]--> </ head >< body >< h1 > Hello, world! </ h1 >< img src = "/static/img/mark.jp g" />< script src = "https://ajax.googleapis.com/ajax/ libs/jquery/1.11.1/jquery.min.js" ></ script >< script src = "/static/CACHE/js/2c7f836f5f1d.js" type = "text /javascript" ></ script ></ body ></ html > As you can see, there is no excess white space in the HTML and the two CSS files are now concatenated together into /static/CACHE/css/ 1afa57f03e30.css . If I run the response through gzip it shows the content is only 440 bytes, about half of its original size. $ curl --silent localhost:8000 | gzip | wc -c 440 I ran the monitor command on redis and made a subsequent request for the homepage. I could see the caches for the headers and body fetched by Django: $ redis-cli -n 3 monitor OK 1415882400 .531810 [ 3 127 .0.0.1:43471 ] "GET" ":1:views.decorators.cache.cache_header..8f95444a9dd16027d8bb2b1e8ed2fb75.en-us.UTC" 1415882400 .532644 [ 3 127 .0.0.1:43471 ] "GET" ":1:views.decorators.cache.cache_page..GET.8f95444a9dd16027d8bb2b1e8ed2fb75.d41d8cd98f00b204e9800998ecf8427e.en-us.UTC"

Embedding assets within HTML If you're running a single page application then external assets might not be faster than embedding them within the HTML. If multiple pages aren't being loaded then the web request to fetch each asset can mean more overhead. django-compressor supports an inline flag in its template tag which will load the assets (if they're on the local system) and place them inline. {% compress css inline %} <link href=" {% static "css/bootstrap.min.css" %} " rel="stylesheet"> <link href=" {% static "css/app.css" %} " rel="stylesheet"> {% endcompress %} Now the contents of css/bootstrap.min.css and css/app.css will be embedded in the page. $ curl --silent localhost:8000 | fold -w50 | head -n20 <!DOCTYPE html> < html lang = "en" >< head >< meta charset = "utf-8" />< meta content = "IE=edge" http-equiv = "X-UA -Compatible" />< meta content = "width=device-width, i nitial-scale=1" name = "viewport" />< title > Bootstrap 101 Template </ title >< style type = "text/css" > /*!* Bo otstrap v3.3.1(http://getbootstrap.com) * Copyrigh t 2011-2014 Twitter,Inc. * Licensed under MIT(http s://github.com/twbs/bootstrap/blob/master/LICENSE) *//*!normalize.css v3.0.2 | MIT License | git.io/ normalize */html{font-family:sans-serif;-webkit-te xt-size-adjust:100%;-ms-text-size-adjust:100%}body {margin:0}article,aside,details,figcaption,figure, footer,header,hgroup,main,menu,nav,section,summary {display:block}audio,canvas,progress,video{display :inline-block;vertical-align:baseline}audio:not([c ontrols]){display:none;height:0}[hidden],template{ display:none}a{background-color:transparent}a:acti ve,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,strong{font-weight:700}dfn{font-style:it alic}h1{margin:.67em 0;font-size:2em}mark{color:#0 The weight of the page totals 28,424 bytes when piped through gzip. $ curl --silent localhost:8000 | gzip | wc -c 28424

Batch image compression I try to always crush images before committing them to a code base rather than leave them for a request process or continuous integration server to handle. It's a CPU-intensive process that only needs to be run once and few compression tools are smart enough to know ahead of time if they'll make a significant dent in the file size or not. I have a script I use which relies on mozjpeg, gifsicle and optipng to crush JPEG, GIF and PNG-formatted images within an given folder and child folders as well. Below are some condensed installation commands. As of this writing Mozilla doesn't provide a binary for mozjpeg so I've found a community-built one instead: $ sudo apt install \ optipng \ gifsicle $ curl -O http://mozjpeg.codelove.de/bin/libmozjpeg_2.1_amd64.deb $ sudo dpkg -i libmozjpeg_2.1_amd64.deb $ sudo ln -s /opt/libmozjpeg/bin/jpegtran /usr/bin/mozjpeg The image crushing script itself: find . -iname "*.jpg" -type f \ -exec mozjpeg -copy none -optimize -outfile {} {} \; find . -iname '*.gif' -type f \ -exec gifsicle -O2 -b {} \; find . -iname '*.png' -type f \ -exec optipng -o9 -q {} \;