How to configure your Django project for multiple environments?

If you’re about to configure Django settings for multiple environments, you need to think ahead. Your simple project may grow significantly and you’ll have to introduce changes in order to run your app on different environments.

In this article, I will show you how to configure Django settings for multiple environments, based on The Twelve-Factor App methodology for building software-as-a-service apps. I have also used Cookiecutter Django framework by Pydanny.

This project uses several third-party tools including PostgreSQL, Sentry, AWS, WhiteNoise, Gunicorn, Redis, Anymail.

TL;DR If you’d like to just take a quick glance at the code, take a look at this SlideShare prezentation. Below, I’ll explain how to configure Django project step by step.

But before we set up the new project, let’s tackle this question:

Django settings for multiple environments

The no 1 reason to configure Django settings for multiple environments is that when you first start a new project, it lacks such arrangement. Not having the bundles split makes it difficult to configure the project later without having to alter the code.

Also, without setting the Django project for multiple environments, there are no dedicated solutions for production like dedicated path for admin panel, logging errors (e.g., Sentry), cache configuration (memcache/redis), saving the data uploaded to cloud by the user (S3), HSTS or secure cookies.

Testing environment, on the other hand, lacks dedicated solutions either, including turning off debugging templates, in-memory caching, sending mails to console, password hasher, storing the templates.

As a result, setting Django project for multiple environments gives you:

more manageable code with fewer duplications

more accurate settings depending on environment type

Starting a Django project

First, you need to set up our new Django project. To do it, install virtualenv/virtualenvwrapper and Django: pip install Django==1.11.5 or whatever Django version you want to use for your project.

Then, create a new project: django-admin: django-admin startproject djangohotspot .

At this point, your project should look like this:

(djangohotspot) ╭ ~/Workspace/ ╰$ tree djangohotspot djangohotspot ├── djangohotspot │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py └── manage.py

Now you need to set it up for multiple environments. You do it by splitting the requirements first.

Splitting requirements

Create base.txt that will cover the common requirements. Then write down the environment-specific requirements in separate files:

local.txt

production.txt

test.txt

The structure of the project is now as follows:

(djangohotspot) ╭ ~/Workspace/ ╰$ tree djangohotspot djangohotspot ├── djangohotspot │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── manage.py └── requirements ├── base.txt ├── local.txt ├── production.txt └── test.txt

Base.txt

django==1.11.5 # Configuration django-environ==0.4.4 whitenoise==3.3.0 # Models django-model-utils==3.0.0 # Images Pillow==4.2.1 # Password storage argon2-cffi==16.3.0 # Python-PostgreSQL Database Adapter psycopg2==2.7.3.1 # Unicode slugification awesome-slugify==1.6.5 # Time zones support pytz==2017.2 # Redis support django-redis==4.8.0 redis>=2.10.5

Production.txt

-r base.txt # WSGI Handler gevent==1.2.2 gunicorn==19.7.1 # Static and Media Storage boto3==1.4.7 django-storages==1.6.5 # Email backends for Mailgun, Postmark, # SendGrid and more django-anymail==0.11.1 # Raven is the Sentry client raven==6.1.0

Test.txt

-r base.txt coverage==4.4.1 flake8==3.4.1 factory-boy==2.9.2 # pytest pytest-cov==2.4.0 pytest-django==3.1.2 pytest-factoryboy==1.3.1 pytest-mock==1.6.0 pytest-sugar==0.9.0

Local.txt

This file combines production and test files:

-r test.txt -r production.txt django-extensions==1.9.0 ipdb==0.10.3

It’s time to configure the settings of each environment.

Splitting settings

First, you need to remove settings.py from the main folder of your Django app djangohotspot/djangohotspot/settings.py and create new Python module named config , where you create another module, settings , where all the settings files will be stored.

The structure of your project has changed and should look like this now:

(djangohotspot) ╭ ~/Workspace/ ╰$ tree djangohotspot djangohotspot ├── config │ ├── __init__.py │ └── settings │ ├── base.py │ ├── __init__.py │ ├── local.py │ ├── production.py │ └── test.py ├── djangohotspot │ ├── __init__.py │ ├── urls.py │ └── wsgi.py ├── manage.py └── requirements ├── base.txt ├── local.txt ├── production.txt └── test.txt

config.setting.base

To configure settings in base.py in this example, I have used the django-environ library.

ROOT_DIR = environ.Path(__file__) - 3 # djangohotspot/ APPS_DIR = ROOT_DIR.path('djangohotspot') # path for django apps INSTALLED_APPS = DJANGO_APPS + THIRD_PARTY_APPS + LOCAL_APPS AUTOSLUG_SLUGIFY_FUNCTION = 'slugify.slugify' # allows you to define a function for unicode-supported Slug DATABASES = { 'default': env.db('DATABASE_URL', default='postgres:///djangohotspot'), } DATABASES['default']['ATOMIC_REQUESTS'] = True # allows you to open and commit transaction when there are no exceptions. This could affect the performance negatively for traffic-heavy apps. EMAIL_BACKEND = env('DJANGO_EMAIL_BACKEND', default='django.core.mail.backends.smtp.EmailBackend') ADMIN_URL = env('DJANGO_ADMIN_URL', default=r'^admin/') PASSWORD_HASHERS = ['django.contrib.auth.hashers.Argon2PasswordHasher', (...)] # add this object at the beginning of the list

config.settings.local

Configuring local settings, you need to import base settings:

from .base import *

Now, add debug toolbar:

MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware', ] INSTALLED_APPS += ['debug_toolbar', ] DEBUG_TOOLBAR_CONFIG = { 'DISABLE_PANELS': [ 'debug_toolbar.panels.redirects.RedirectsPanel', ], 'SHOW_TEMPLATE_CONTEXT': True, }

Define allowed IP addresses:

INTERNAL_IPS = ['127.0.0.1']

And add Django extension: INSTALLED_APPS += ['django_extensions', ]

config.settings.production

In production settings, you should focus on the security for your project.

# security configuration SECURE_HSTS_SECONDS = 60 SECURE_HSTS_INCLUDE_SUBDOMAINS = env.bool( 'DJANGO_SECURE_HSTS_INCLUDE_SUBDOMAINS', default=True) SECURE_CONTENT_TYPE_NOSNIFF = env.bool( 'DJANGO_SECURE_CONTENT_TYPE_NOSNIFF', default=True) SECURE_BROWSER_XSS_FILTER = True SESSION_COOKIE_SECURE = True SESSION_COOKIE_HTTPONLY = True SECURE_SSL_REDIRECT = env.bool('DJANGO_SECURE_SSL_REDIRECT', default=True) CSRF_COOKIE_SECURE = True CSRF_COOKIE_HTTPONLY = True X_FRAME_OPTIONS = 'DENY' ADMIN_URL = env('DJANGO_ADMIN_URL')

At this point it’s worth to add DJANGO_ADMIN_URL to the production settings. Change it from default to avoid attack attempts on the default URL admin panel.

Next, you need to add your domain or domains:

ALLOWED_HOSTS = env.list('DJANGO_ALLOWED_HOSTS', default=['djangohotspot.pl', ])

And add a Gunicorn:

INSTALLED_APPS += ['gunicorn', ]

Finally, add django-storage for AWS:

INSTALLED_APPS += ['storages', ] AWS_ACCESS_KEY_ID = env('DJANGO_AWS_ACCESS_KEY_ID') AWS_SECRET_ACCESS_KEY = env('DJANGO_AWS_SECRET_ACCESS_KEY') AWS_STORAGE_BUCKET_NAME = env('DJANGO_AWS_STORAGE_BUCKET_NAME') AWS_AUTO_CREATE_BUCKET = True AWS_QUERYSTRING_AUTH = False AWS_EXPIRY = 60 * 60 * 24 * 7 MEDIA_URL = 'https://s3.amazonaws.com/%s/' % AWS_STORAGE_BUCKET_NAME DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

To efficiently track errors you may add Sentry:

INSTALLED_APPS += ['raven.contrib.django.raven_compat', ] RAVEN_MIDDLEWARE = ['raven.contrib.django.raven_compat.middleware.SentryResponseErrorIdMiddleware'] MIDDLEWARE = RAVEN_MIDDLEWARE + MIDDLEWARE SENTRY_DSN = env('DJANGO_SENTRY_DSN') SENTRY_CLIENT = env('DJANGO_SENTRY_CLIENT', default='raven.contrib.django.raven_compat.DjangoClient') SENTRY_CELERY_LOGLEVEL = env.int('DJANGO_SENTRY_LOG_LEVEL', logging.INFO) RAVEN_CONFIG = { 'CELERY_LOGLEVEL': env.int('DJANGO_SENTRY_LOG_LEVEL', logging.INFO), 'DSN': SENTRY_DSN, }

And if you want to serve static files, add WhiteNoise:

WHITENOISE_MIDDLEWARE = ['whitenoise.middleware.WhiteNoiseMiddleware', ] MIDDLEWARE = WHITENOISE_MIDDLEWARE + MIDDLEWARE STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

config.setting.test

Configuring test settings, start with turning debugging off:

DEBUG = False TEMPLATES[0]['OPTIONS']['debug'] = False

Store sent mails in memory. They are available in django.core.mail.outbox:

EMAIL_BACKEND = 'django.core.mail.backends.locmem.EmailBackend'

Set the cache:

CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache', 'LOCATION': '' } }

Set the password hasher to speed up the tests:

PASSWORD_HASHERS = ['django.contrib.auth.hashers.MD5PasswordHasher', ]

If you use Django templates, you can set them to be stored in memory:

TEMPLATES[0]['OPTIONS']['loaders'] = [ ['django.template.loaders.cached.Loader', [ 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', ] , ] , ]

uwsgi.py and urls.py files

Because we’ve split the main settings file into dedicated files with configuration for each environment, we need to point a file which will be used by default when it’s not clearly indicated.

In urls.py file we define the 4xx and 5xx pages. We also add debug toolbar here.

Move uwsgi.py and urls.py files from djangohotspot/djangohotspot catalogue to config module and add following changes to config.settings :

WSGI_APPLICATION = 'config.wsgi.application' ROOT_URLCONF = 'config.urls'

At the end of the config.urls file add the following code to debug 4xx and 5xx pages:

if settings.DEBUG: urlpatterns += [ url(r'^400/$', default_views.bad_request, kwargs={'exception': Exception('Bad Request!')}), url(r'^403/$', default_views.permission_denied, kwargs={'exception': Exception('Permission Denied')}), url(r'^404/$', default_views.page_not_found, kwargs={'exception': Exception('Page not Found')}), url(r'^500/$', default_views.server_error), ] if 'debug_toolbar' in settings.INSTALLED_APPS: import debug_toolbar urlpatterns = [ url(r'^__debug__/', include(debug_toolbar.urls)), ] + urlpatterns

In our example, config.uwsgi file will look like this:

import os import sys from django.core.wsgi import get_wsgi_application app_path = os.path.dirname(os.path.abspath(__file__)).replace('/config', '') sys.path.append(os.path.join(app_path, 'djangohotspot')) if os.environ.get('DJANGO_SETTINGS_MODULE') == 'config.settings.production': from raven.contrib.django.raven_compat.middleware.wsgi import Sentry os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") application = get_wsgi_application() if os.environ.get('DJANGO_SETTINGS_MODULE') == 'config.settings.production': application = Sentry(application)

Summary

As you’ve seen in this article, setting a Django project for multiple environments is a toilsome task. But trust me, it pays off quickly once you start working on the project.

From this point on, you can think of some containerization with Docker, which will give you portability and easiness of setup for your project regardless the environment it will be run on.