Introduction

For those just joining us, the previous section of the Modern Django guide can be found here: Modern Django — Part 0: Introduction and Initial Setup.

The GitHub for this project and all its sections can be found here.

At this point, we have successfully created a Django project. However, we have a lot of legwork to do before we can even claim it is ready for local development and creating our first application. First, we must extend our project’s requirements. Then, we must extend the project’s settings. We will then create a set of environment variables to hold secret values. Doing these steps will increase the speed at which we can develop, test, and deploy our application in the future.

I suggest you bunker down with some espresso for this section. It’s dense. PC: Alejandro Escamilla

After extending the settings, we will go over the default portions of the Django’s Settings API used by our project when settings.py was first created. We will also meet some commonly used settings not created by default, such as those surrounding the handling of static files. Understanding Django’s Settings API is the first step to shedding light on all that Django Magic™. We will do some more light refactoring and then ensure our project can run!

It is important to make sure all new users are comfortable with Django’s settings, so I apologize to those of you who know Django well or want to get cracking on a REST API. I can only make learning about configuration portions like this so fun (all the fun pictures are at the bottom this time around).

Note: As always transparency is something I value. With that being said, throughout this portion you will notice a similar structure to one provided by the Cookiecutter Django Team. If we didn’t work off of CCD, we would end up just extending the base Django tutorial until advanced portions are needed, so I believe this is a better strategy. The CCD team has figured out a large number of local and production problems and how to mitigate them. I have taken some notes on their structure, compared to where we need to be. While our project may look similar in the early stages, it is important to explain what is being done, instead of blindly creating the template project. Our project will end up being very different in the future, but for now you may notice structural similarities. For example, a large difference we will implement is the removal of server side rendering, (which CCD and many many other projects makes use of) for a REST API coupled with a frontend JavaScript framework.

Temporary Notes: For those of you following along from an older version of Part 0 (pre 2/24/17), we now have an updated project structure. The django-admin startproject command from Part 0 has been updated. Sorry, gang. After looking over my notes a large number of edits were needed to get to the above structure. It is simpler for everyone, following along prior, to remove your project directory and recreate the project under the new name and location.

To do so, move into the modern-django directory, remove the old project, and create the new one. Commands below:

# Commands

cd ~/git/modern-django/

rm -rf project/

django-admin startproject config .

Technology

Not many technologies to read up on for this portion!

Django Settings: “A Django settings file contains all the configuration of your Django installation.” (source). This is the set of files/configurations we will be learning about today! It is the backbone of any Django project.

Environment Variables: “A set of dynamic named values that can affect the way running processes will behave on a computer. They are part of the environment in which a process runs.” (source) We will be making a bunch of environment variables through this process.

django-environ: “Allows you to utilize 12factor inspired environment variables to configure your Django application. “ (source) We will install this package to make managing environment variables with Django easier!

Walkthrough

Picking up from Part 0, our current project structure looks as follows!

modern-django

├── LICENSE

├── README.md

├── config

│ ├── __init__.py

│ ├── settings.py

│ ├── urls.py

│ └── wsgi.py

├── manage.py

└── requirements.txt

From here on out, I will assume you are in the modern-django directory, unless otherwise specified. To move into the modern-django directory, follow below:

# Command

cd ~/git/modern-django/

1. Extending requirements.txt

In this section, we will refactor the requirements.txt to be far more robust. First, create a directory in the modern-django directory called requirements . Then, copy the current requirements.txt into it and, at the same time, rename it to base.txt . Then, move into the requirements directory and create requirements file for local, production, and test environments.

# Command

mkdir requirements/

mv requirements.txt requirements/base.txt

cd requirements/

touch local.txt production.txt test.txt

Now, we have the ability to change which Python applications our virtualenv (or a Docker image) will use based on the environment (local development, production, or testing).

1-A. Append django-environ to base.txt

In the base.txt , which all environment will build off of, we will add django-environ. This will allow us to simplify our Django Module Settings on two fronts:

Ease usage of environment variables Simplifying file paths

Open the requirments/base.txt file with your preferred text editor, and make the addition under django==1.10.5 :

django-environ==0.4.1

1-B. Ensure Local, Production, and Test Requirements Inherit base.txt

Now, inside of our local.txt , production.txt , and test.txt , we will add a line that will recursively get all of the requirements from base.txt . Add this line to each of the top of the newly created files:

-r base.txt

Just like how we added django to the old requirements.txt in Part 0, this is how we will continue to add more third party applications to the project. More additions will be made to these files as the guide progresses, but for now it is time to turn our attention towards settings.py .

2. Extending settings.py

In this section, we will refactor the settings.py to be far more robust and separate it much like requirements.txt above. We will first create a new directory called settings inside of config :

# Command

mkdir config/settings/

Now, move the settings.py file into the settings directory, and rename it to base.py :

# Command

mv config/settings.py config/settings/base.py

The next step is to create the extended settings for each environment (local, production, and test):

# Command

cd config/settings/

touch local.py production.py test.py

Again, in your text editor of choice, at the top of local.py , production.py , and test.py , extend base.py by adding:

from .base import *

Now that our scaffolding has been made, it is time to walk through and refactor the settings file base.py .

Note: The above line makes use of import * . As a good engineer, you should only import the portions of classes/modules/libraries you need, but with all things, there are exceptions.

3. Meeting the Django Settings API

Currently, the base.py only has the standard settings provided by the django-admin startproject . We will now go down the settings per section; providing a layman’s definition and edits as necessary.

3-A. Adding django-environ

As described in the above sections, the use of django-environ is to simplify environment variable and file path usage.

In base.py , you will find import os under the generated comments. This is currently being used in a few portions to obtain file paths. The use of environ will simplify this functionality for us. Thus, find import os and replace it with importing environ at the top of base.py :

import environ

3-B. File Path Variables: BASE_DIR to ROOT_DIR + APPS_DIR

Now that we have removed import os , we will edit the use of the variable BASE_DIR . BASE_DIR is typically used for building more file path variables off of the project’s top level (commonly known as root directory), i.e. finding another file or a directory of files.

Now that we have environ , we can replace BASE_DIR with another variable called ROOT_DIR . (or simply rename it, if you wish, or don’t, ROOT_DIR just makes more logical sense for the structure). We will then, add the following:

ROOT_DIR = environ.Path(__file__) - 3

This function gets the current files path ~/git/modern-django/config/base.py and moves up three levels to ~/git/modern-django/ . Now, we have a variable that points to the root of the project and we can get any other file in the directory from it.

To demonstrate that we can access any other file, we will create another variable called APPS_DIR . This will point it at a folder we have not created yet that will later hold the applications we create. Add APPS_DIR below ROOT_DIR :

APPS_DIR = ROOT_DIR.path('project')

3-C. Import Environment Variables

Below our newly created ROOT_DIR and APPS_DIR , we will import a large number of environment variables to use in the project from a file called .env (we’ll make that soon). We will use the environ.Env() function that will find a .env file in our project root, then read_env() to make use of them. To do so, add the following:

env = environ.Env() # This section added from an update to standards in CookieCutter Django to ensure no errors are encountered at runserver/migrations

READ_DOT_ENV_FILE = env.bool('DJANGO_READ_DOT_ENV_FILE', default=False) if READ_DOT_ENV_FILE:

env_file = str(ROOT_DIR.path('.env'))

print('Loading : {}'.format(env_file))

env.read_env(env_file)

print('The .env file has been loaded. See base.py for more information')

Notes: The main purpose of adding this functionality in now is so that when using Docker containers, we will be reading environment variables from the .env file, as there are some complications with doing so in Docker.

3-D. Intro to SECRET_KEY and Moving to local.py

!!! IF YOU READ NO OTHER SECTIONS, PLEASE READ THIS ONE !!!

Following the addition of environment variables, we come across the setting SECRET_KEY . The secret key in Django is extremely important and like its name suggests, should be kept secret. However, we have already committed our secret key to what most likely is a public repository on GitHub. Don’t worry, this was intentional for local development between environments and other engineers. Note that we will be creating another secret key value when we begin deploying our application, and that value should not be tracked or shared with others.

The secret key is used in a number of places, including creating user’s sessions, messages, cookies, password resets, and cryptographic signing. So, it is pretty important. Read more about it here in the official docs.

For local development, we will cut and paste (removing the value from base.py ), the variable SECRET_KEY into local.py and edit the line to match the following (your key value will be different):

SECRET_KEY = env('DJANGO_SECRET_KEY', default='^92l&5_l2f-ik5xlav!7*cat904fro-lmdd@0kgz@c*nxua3@p')

3-E. Intro to DEBUG

We now find ourselves with the DEBUG setting. It is currently set to True . When DEBUG is set to True in Django, a number of things happen, including: detailed error pages, all SQL queries made are stored, and ALLOWED_HOSTS can be empty. Read more about DEBUG here. We should always assume that DEBUG will be False , except in local development where it is True . Edit the current value in base.py to set an environment variable to False .

DEBUG = env.bool('DJANGO_DEBUG', False)

Using DEBUG as True should only be used in local development, and thus we should also make a copy in local.py . Copy and paste the DEBUG setting into local.py and overwrite the environment variable by setting it to True .

DEBUG = env.bool('DJANGO_DEBUG', default=True)

Making a debug environment variable allows us to configure deployment tools more efficiently based on the settings provided!

3-F. Intro to ALLOWED_HOSTS

ALLOWED_HOSTS is the list of sites that Django will accept to make connection and requests. For local development when DEBUG is True , we can have this be empty, however, when in production it must be set to a specific domain such as .example.com .

We will touch on this when we return to deployment. For now, simply cut and paste the ALLOWED_HOSTS into production.py below any imports.

3-G. Intro to INSTALLED_APPS

The INSTALLED_APPS array is the list of applications that the Django project is aware of. It is initialized with a number of default Django applications that run the base project. Typically, projects would simply add in applications as they are installed by pip or created by the engineers. However, for organization will split up the INSTALLED_APPS section into three tuples: DJANGO_APPS , THIRD_PARTY_APPS , and LOCAL_APPS . After we populate the three sections, we will add them together to make the INSTALLED_APPS list.

Note: The usage of tuples, over lists, is because they are immutable and the ease of concatenating them together. Their immutability also ensures, that, at runtime, the project’s installed apps cannot be changed.

So, currently the only applications installed are those default Django apps. Change the structure to reflect this setup:

DJANGO_APPS = (

'django.contrib.admin',

'django.contrib.auth',

'django.contrib.contenttypes',

'django.contrib.sessions',

'django.contrib.messages',

'django.contrib.staticfiles',

)

THIRD_PARTY_APPS = (

)

LOCAL_APPS = (

)

INSTALLED_APPS = DJANGO_APPS + THIRD_PARTY_APPS + LOCAL_APPS

As we begin installing more third party applications and create our REST API, we will populate the two empty tuples.

3-H. Intro to MIDDLEWARE

We now find ourselves in front of the illusive MIDDLEWARE . The concept of middleware is taking requests/responses (HTTP communication) as they enter/leave the Django system and applying functions to them before/after being processed in a light weight manner. Such an example is AuthenticationMiddleware , which can associate a user with a request when user sessions are being used. These functions are done extremely fast and help add data we can process against.

IMPORTANT: The order in which middleware are placed in the MIDDLEWARE list is the order in which they will be processed. That means specific middleware must go above others, or else unexpected behavior may occur.

The Django documentation describes this process like layers of an onion, where a View (and its children) is at the center and a request must go through each wrapping layer of middleware in order, before being manipulated by the view.

For a more in-depth explanation on middleware, please read the documentation page.

We can leave this be for now!

3-I. Intro to ROOT_URLCONF

The ROOT_URLCONF points to the URLconf module (a collection of URL guidelines) that Django will handles requests/responses with. Essentially, when a user makes a request to a page/resource in one of our apps, Django looks in that file to determine what to do. That is, the ROOT_URLCONF points to a file that contains a set of URLs such as modern-django/config/urls.py . The current pointer is the default, and we will add more URLs to the file, such as those to our REST API, later. Currently, you can find it extending to the admin app’s URLs.

3-J. Intro to TEMPLATES

The TEMPLATES setting is a list of settings that are applied to the template engines installed. Multiple can be installed, however most projects will only need one. The default settings include:

BACKEND : the processor for creating templates

: the processor for creating templates DIRS : directories that Django will look for template files

: directories that Django will look for template files APP_DIRS : set to True, Django will automatically look in each app in the INSTALLED_APPS list for templates that could possibly be rendered

: set to True, Django will automatically look in each app in the list for templates that could possibly be rendered OPTIONS : extra options that depend on the BACKEND

: extra options that depend on the context_processors : an option that is for the DjangoTemplates engine, that determine how data is rendered on a template. Read more here.

More information on settings can be found here.

For the purposes of transparency, we will most likely gut this portion later, as we intend to use a true JavaScript frontend with our API.

3-K. Intro to WSGI_APPLICATION

“WSGI” stands for Web Server Gateway Interface. It is a standard for Python web frameworks that point a server at an application.

WSGI_APPLICATION points to the wsgi.py file that Django will serve when the runserver command is executed. It contains an application as a callable and will point the current settings being used at that application. This is essential for running our application. More information can be found here.

3-L. Intro to DATABASES

The DATABASES settings are those that connect different databases to our project. They store the settings needed to interact with different databases. The default database is a locally stored sqlite3 database (that has not been created yet, as we have done no database manipulation. Yet.). We will later on update this to include a PostgreSQL database when we begin working with Docker.

Again, we have encountered another use of os , thus we should change it to work with a path created by environ . Replace the NAME line as follows:

'NAME': str(ROOT_DIR.path('db.sqlite3')),

3-M. Intro to AUTH_PASSWORD_VALIDATORS

This configuration is a set of rules that passwords must be verified by when created or edited by users. The defaults are robust enough to reduce a large portion of attacks and require no additions. You can create custom validators for increased security as defined here.

3-N. Intro to Internationalization

The section of LANGUAGE_CODE , TIME_ZONE , USE_I18N , USE_L10N , and USE_TZ are all used for formatting data and languages to different users. It has been requested to talk more on this subject at a later date, however for the majority of US / English-based sites, the defaults are more than enough. The documentation for these settings is defined here.

3-O. Intro to Static Files and Media Files

Static files are files that are considered not changing, static. This includes files, such as images, CSS, and JavaScript that are used in the application. Media files are considered dynamic files created/edited/uploaded by users. The majority of rules below apply to both Static and Media files.

We now find ourselves at one of the hardest concepts to grasp for new Django users. There is one standard use case for how static/media file serving should be done, but many users become confused on how to do it.

It is extremely important to understand that static files currently live in separate directories, per app. That location is not the location they will be served from in production.

An example, the CSS files used on the Admin panel will not be served from its own static directory of the admin application in production. The manage.py command, collectstatic , will be used when deploying. This will copy all static files this settings file knows about, into a root level folder defined by STATIC_ROOT . Then, a server, such as nginx, will retrieve and serve those files.

HOWEVER, on local development, Django will simply serve them from the directories they originally live in.

Currently, STATIC_URL is the only static settings created by default. This is the URL that static files will be served at. Currently, the files will be served at localhost:8000/static , if localhost:8000 is used for local development.

We will add the following:

STATIC_ROOT : where the collected static files will be placed

: where the collected static files will be placed STATICFILES_DIRS : folders Django will look for static files in

: folders Django will look for static files in STATICFILES_FINDERS : specifications on what files to look for

: specifications on what files to look for MEDIA_URL : URL appended to the root URL to serve media data

: URL appended to the root URL to serve media data MEDIA_ROOT : where the collected media files will be stored

Make these additions to Static and Media settings where STATIC_URL is defined in base.py :