This is part two in a three-part series:

Making React and Django play well together - the “hybrid app” model

Last month I discussed the trade-offs involved in choosing an architecture for integrating React with Django.

I described an alternative between two models:

The “single page app” model: a standalone JavaScript frontend makes API requests to a backend running on another domain.

model: a standalone JavaScript frontend makes API requests to a backend running on another domain. The “hybrid app” model: the same backend serves HTML pages embedding JavaScript components and API requests.

I promised that I would describe how to implement each model.

Today I’m starting with the “hybrid app” model.

Let’s bootstrap a todolist app!

Disclaimer: I’m starting with default project templates and making minimal changes in order to focus on the integration between frontend and backend.

As a consequence, I’m ignoring many best practices for Django and React projects. Keep in mind that I’m describing only one piece of the puzzle and that many variants are possible!

Why build a “hybrid app”?

Here are the main reasons:

Your app needs good SEO (e-commerce, classifieds, portfolios, etc.)

You prefer treating your app as a single unit because that’s more convenient for building new features, running integration tests, and deploying new versions.

You have a team of full-stack developers — perhaps a team of one — and you’d rather avoid the overhead of managing separate backend and frontend projects.

You’re comfortable with building full-stack apps with Django and you want to try a modern JavaScript framework while leveraging your experience.

While the “hybrid app” model feels a bit old-school, it’s good for enhancing the frontend of backend-heavy apps — whether new or pre-existing — without disruption.

Initialization

Since the frontend and the backend are deployed together, it makes sense to maintain them in the same code repository.

Let’s initialize Django and React applications in backend and frontend directories at the root of the repository.

Django

Start a shell, go to the root of the code repository and bootstrap the backend:

mkdir backend cd backend pipenv install django pipenv shell django-admin startproject todolist .

For convenience, edit the Django settings as follows. This will make it easier to switch between development and production mode.

# backend/todolist/settings.py # insert these lines after the definition of BASE_DIR BACKEND_DIR = BASE_DIR # rename variable for clarity FRONTEND_DIR = os . path . abspath ( os . path . join ( BACKEND_DIR , '..' , 'frontend' )) # modify the definition of DEBUG and ALLOWED_HOSTS DEBUG = os . environ . get ( 'DJANGO_ENV' ) == 'development' ALLOWED_HOSTS = [ 'localhost' ]

Start the development server:

# in the backend directory, after executing pipenv shell DJANGO_ENV = development ./manage.py migrate DJANGO_ENV = development ./manage.py runserver

Open http://localhost:8000/ in a browser to confirm that everything is working.

React

Start another shell, go to the root of the code repository and bootstrap the frontend:

npx create-react-app frontend cd frontend

Start the development server:

# in the frontend directory yarn start

http://localhost:3000/ opens automatically in a browser.

Starting point

Django and React development servers are now running on http://localhost:8000/ and http://localhost:3000/ but they don’t know anything about each other.

The source tree contains:

. ├── backend │ ├── Pipfile │ ├── Pipfile . lock │ ├── manage . py │ └── todolist │ ├── __init__ . py │ ├── settings . py │ ├── urls . py │ └── wsgi . py └── frontend ├── README . md ├── package . json ├── public │ ├── favicon . ico │ ├── index . html │ └── manifest . json ├── src │ ├── App . css │ ├── App . js │ ├── App . test . js │ ├── index . css │ ├── index . js │ ├── logo . svg │ └── registerServiceWorker . js └── yarn . lock

You can confirm this by running tree -I node_modules .

Running yarn build in the frontend directory adds:

. └── frontend └── build ├── asset - manifest . json ├── favicon . ico ├── index . html ├── manifest . json ├── service - worker . js └── static ├── css │ ├── main . c17080f1 . css │ └── main . c17080f1 . css . map ├── js │ ├── main . a3b22bcc . js │ └── main . a3b22bcc . js . map └── media └── logo . 5 d5d9eef . svg

Production setup

Let’s start with the configuration for running the app in production.

Many developers would think about their development setup first.

However, production has more constraints than development, notably security and performance. As a consequence, I prefer starting with the production setup and then building a development setup that provides adequate dev / prod parity.

Serving HTML

In the “hybrid app” model, Django is in charge of serving HTML pages.

Let’s add a view that renders the index.html generated by create-react-app with the Django template engine.

A catchall URL pattern routes any unrecognized URL with that view. Then the frontend URL router takes over and renders the appropriate page on the client side.

Configure the template engine:

# backend/todolist/settings.py TEMPLATES = [ { 'DIRS' : [ os . path . join ( FRONTEND_DIR , 'build' )], ... , }, ]

Create the view in a new module:

# backend/todolist/views.py from django.views.generic import TemplateView catchall = TemplateView . as_view ( template_name = 'index.html' )

Add a catchall URL pattern to the URLconf:

# backend/todolist/urls.py from django.contrib import admin from django.urls import path , re_path from . import views urlpatterns = [ path ( 'admin/' , admin . site . urls ), re_path ( r '' , views . catchall ), ]

Build the frontend:

# in the frontend directory yarn build

Install waitress, a production WSGI server, and start it locally:

# in the backend directory, after executing pipenv shell pipenv install waitress waitress-serve todolist.wsgi:application

At this point, you can access the HTML page generated by create-react-app and served by Django at http://localhost:8080/. The page title displayed in the browser tab says “React App”, which confirms that we’re getting the intended HTML page.

However, the page is blank because loading the CSS and JS failed. Let’s fix that.

Serving static files

The “hybrid app” model allows us to take advantage of the staticfiles contrib app and its ecosystem. I find it convenient to serve static files with WhiteNoise and cache them with a CDN for performance.

The changes described below:

Configure django.contrib.staticfiles and WhiteNoise;

Declare that frontend/build/static/ contains static files;

contains static files; Serve them at /static/ where create-react-app expects them;

where create-react-app expects them; Serve the few other files in the frontend/build/ directory at the server root.

# backend/todolist/settings.py INSTALLED_APPS = [ ... , # before django.contrib.staticfiles 'whitenoise.runserver_nostatic' , ... ] MIDDLEWARE = [ ... , # just after django.middleware.security.SecurityMiddleware 'whitenoise.middleware.WhiteNoiseMiddleware' , ... , ] STATICFILES_DIRS = [ os . path . join ( FRONTEND_DIR , 'build' , 'static' )] STATICFILES_STORAGE = ( 'whitenoise.storage.CompressedManifestStaticFilesStorage' ) STATIC_ROOT = os . path . join ( BACKEND_DIR , 'static' ) STATIC_URL = '/static/' # already declared in the default settings WHITENOISE_ROOT = os . path . join ( FRONTEND_DIR , 'build' , 'root' )

This setup requires moving the files to serve at the server root in a root subdirectory. That’s everything in frontend/build except index.html and static .

# in the frontend directory cd build mkdir root mv *.ico *.js *.json root cd ..

Stop the WSGI server, install WhiteNoise, collect static files, and restart the server:

# in the backend directory, after executing pipenv shell pipenv install whitenoise ./manage.py collectstatic waitress-serve todolist.wsgi:application

Now, if you refresh the page at http://localhost:8080/, the create-react-app welcome page loads. Success! Also, the browser tab displays the React icon, which means that serving files like favicon.ico from the server root works.

Deploying

This setup is compatible with any method for deploying Django to production, provided the server that builds the application prior to deploying it is able to build the frontend and the backend. This boils down to installing Node.js and yarn.

A complete build script for the application looks like:

cd frontend # 1. Build the frontend yarn build # 2. Move files at the build root inside a root subdirectory mkdir -p build/root for file in $( ls build | grep -E -v '^(index\.html|static|root)$' ) ; do mv "build/ $file " build/root ; done cd .. cd backend pipenv shell # 3. Build the backend ./manage.py collectstatic --no-input cd ..

Development setup

Now that we’re happy with the production setup, let’s build the development setup.

The challenge is to optimize dev / prod parity while preserving all features of the development servers.

Remove all build artifacts to make sure we don’t accidentally rely on them:

# in the backend directory rm -rf static

# in the frontend directory rm -rf build

If the Django and React development servers aren’t running anymore, start them again with:

# in the backend directory, after executing pipenv shell DJANGO_ENV = development ./manage.py runserver

# in the frontend directory yarn start

Serving HTML

In production, we’re loading frontend/build/index.html as a template.

In development, it’s available at http://localhost:3000/.

It would be possible to build a Django template loader that loads templates from an URL. That would be optimal in terms of dev / prod parity but I don’t think it’s worth the complexity.

Instead I’m just going to write an alternative catchall view that proxies requests for index.html to the React development server.

Proxying is more appropriate than redirecting: redirecting could change the behavior of the browser and introduce significant differences between dev and prod.

Install Requests:

# in the backend directory pipenv install requests

Change the views module to:

# backend/todolist/views.py import requests from django import http from django.conf import settings from django.template import engines from django.views.generic import TemplateView def catchall_dev ( request , upstream = 'http://localhost:3000' ): upstream_url = upstream + request . path response = requests . get ( upstream_url ) content = engines [ 'django' ] . from_string ( response . text ) . render () return http . HttpResponse ( content ) catchall_prod = TemplateView . as_view ( template_name = 'index.html' ) catchall = catchall_dev if settings . DEBUG else catchall_prod

I’m keeping the original implementation in production because it benefits from Django’s caching of compiled templates.

Refresh http://localhost:8000/: while the page is blank, the page title says “React App”, which is good.

However, there’s a stack trace and a HTTP 500 error in the logs of runserver : "GET /static/js/bundle.js HTTP/1.1" 500 . Oops, our catchall view is also receiving requests for static files and it crashes when it attempts to compile them as Django templates!

That was our next step anyway, so let’s improve catchall_dev .

Serving static files

Here’s a version that runs only HTML responses through Django’s template engine.

In addition, it avoids buffering static assets in memory. Development builds of the frontend can grow very large because they aren’t optimized for file size like production builds. Using the streaming APIs in requests and Django makes a noticeable difference.

# backend/todolist/views.py import requests from django import http from django.conf import settings from django.template import engines from django.views.generic import TemplateView def catchall_dev ( request , upstream = 'http://localhost:3000' ): upstream_url = upstream + request . path response = requests . get ( upstream_url , stream = True ) content_type = response . headers . get ( 'Content-Type' ) if content_type == 'text/html; charset=UTF-8' : return http . HttpResponse ( content = engines [ 'django' ] . from_string ( response . text ) . render (), status = response . status_code , reason = response . reason , ) else : return http . StreamingHttpResponse ( streaming_content = response . iter_content ( 2 ** 12 ), content_type = content_type , status = response . status_code , reason = response . reason , ) catchall_prod = TemplateView . as_view ( template_name = 'index.html' ) catchall = catchall_dev if settings . DEBUG else catchall_prod

Refresh http://localhost:8000/: the application loads.

Try modifying source files in the frontend. Autoreload works. Hurray!

Optimizing autoreload

At this point, there’s still a couple errors in the logs of runserver :

"GET /sockjs-node/nnn/xxxxxxxx/websocket HTTP/1.1" 400 : our proxy doesn’t support WebSocket connections; "POST /sockjs-node/nnn/xxxxxxxx/xhr_streaming?t=ttttttttttttt HTTP/1.1" 403 : our proxy rejects POST requests because they don’t account for Django’s CSRF protection.

SockJS falls back to Server-Sent Events to wait for autoreload events, as the following line in the logs shows: "GET /sockjs-node/nnn/xxxxxxxx/eventsource HTTP/1.1" 200 . Server-Sent Events happen to be supported as a side effect of the streaming optimization.

Can we avoid these errors?

The question of dev / prod parity no longer matters here because there’s no autoreload in production.

We can easily add support for POST requests and fix the second error by disabling CSRF protection for the catchall_dev view and proxying the HTTP method properly.

Unfortunately there’s no trivial way to support WebSocket connections and fix the first error:

proxying them would require setting up Django Channels and writing quite a bit of code;

redirecting them doesn’t work because browsers don’t follow redirects on WebSocket connections;

routing them to the create-react-app dev server isn’t possible at the time of writing; that’s an open issue.

For lack of a better solution, we can detect WebSocket requests and return a HTTP 501 Not Implemented error code.

An error will still be displayed in the console of the web browser and SockJS will fall back to a less efficient mechanism, XHR Streaming, that keeps a Python thread busy.

Here’s the final version:

# backend/todolist/views.py import requests from django import http from django.conf import settings from django.template import engines from django.views.decorators.csrf import csrf_exempt from django.views.generic import TemplateView @csrf_exempt def catchall_dev ( request , upstream = 'http://localhost:3000' ): """ Proxy HTTP requests to the frontend dev server in development. The implementation is very basic e.g. it doesn't handle HTTP headers. """ upstream_url = upstream + request . path method = request . META [ 'REQUEST_METHOD' ] . lower () response = getattr ( requests , method )( upstream_url , stream = True ) content_type = response . headers . get ( 'Content-Type' ) if request . META . get ( 'HTTP_UPGRADE' , '' ) . lower () == 'websocket' : return http . HttpResponse ( content = "WebSocket connections aren't supported" , status = 501 , reason = "Not Implemented" ) elif content_type == 'text/html; charset=UTF-8' : return http . HttpResponse ( content = engines [ 'django' ] . from_string ( response . text ) . render (), status = response . status_code , reason = response . reason , ) else : return http . StreamingHttpResponse ( streaming_content = response . iter_content ( 2 ** 12 ), content_type = content_type , status = response . status_code , reason = response . reason , ) catchall_prod = TemplateView . as_view ( template_name = 'index.html' ) catchall = catchall_dev if settings . DEBUG else catchall_prod

And we’re done!

I didn’t explain all the design choices, so you may be wondering…

Why this design?

The setup I described takes advantage of features available in create-react-app and django.contrib.staticfiles to optimize page load performance and dev / prod parity with very little code and without introducing any additional dependencies.

Caching

Optimizing how browsers cache static assets is a performance-critical requirement.

The most reliable solution consists in:

Inserting a hash of the contents of each file in the name of the file;

Telling the browser that it can cache static files forever.

If the contents of a file changes, then its name changes and the browser loads the new version.

In practice this is more complicated than hashing and renaming files. For example, when CSS references an image, inserting a hash in the name of the image changes the contents of the CSS, which changes its hash. Implementing this behavior involves parsing static assets and understanding dependencies.

Django is able to parse and to modify CSS to handle dependencies. This is implemented in in ManifestStaticFilesStorage , which WhiteNoise’s CompressedManifestStaticFilesStorage builds upon.

Code splitting

Delivering the application gradually with code splitting is another performance-critical requirement.

When code splitting is enabled, a JS loader downloads chunks that define modules and imports them as needed. This is a fairly new requirement: code splitting wasn’t mainstream until two years ago.

Django is unable to parse JS and understand dependencies between JS files. For this reason, the bundler needs to be responsible for inserting hashes in file names. Generally speaking, since the bundler is responsible for creating an optimized build of frontend assets, it makes sense to let it take care of inserting hashes in file names.

Putting it all together

Regardless of which system performs the hashing, a mapping from original file names to hashed file names is needed in order to substitute the hashed file name automatically whenever the developer references a static asset by its original file name. For example, main.js must be replaced with main.a3b22bcc.js .

The crux of the issue is to transmit this mapping from the bundler, which creates it when it builds the frontend, to the backend, which needs it to reference static files in HTML pages.

In our setup, the mapping is already applied in frontend/build/index.html when Django loads it as a HTML template, so the backend doesn’t need to do anything.

An alternative solution involves dumping the mapping in a JSON file with a webpack plugin, then loading it and applying it with a Django storage engine. webpack-bundle-tracker and django-webpack-loader implement this.

One more thing!

If you’ve been following closely, you noticed that our setup hashes frontend assets twice: webpack hashes them during yarn build and Django hashes them again during ./manage.py collectstatic . This produces files such as backend/static/js/main.a3b22bcc.5c290d7ff561.js .

These double-hashed files are never referenced.

However, generating them makes ./manage.py collectstatic a bit slower than it could be, especially on large apps. It’s possible to optimize this by subclassing CompressedManifestStaticFilesStorage , if you accept relying on private APIs.

Since I know you’re going to ask, here’s an implementation. It’s a fun hack and it works for me. Don’t shout at me it if breaks :-)

# todolist/storage.py import collections import re from django.contrib.staticfiles.storage import ( ManifestStaticFilesStorage ) from whitenoise.storage import ( CompressedStaticFilesMixin , HelpfulExceptionMixin ) class SkipHashedFilesMixin : _already_hashed_pattern = re . compile ( r '\.[0-9a-f]{8}\.' ) def is_already_hashed ( self , path ): """ Determine if a file is already hashed by webpack. The current implementation is quite lax. Adapt as needed. """ return self . _already_hashed_pattern . search ( path ) def post_process ( self , paths , dry_run = False , ** options ): """ Skip files already hashed by webpack. """ if dry_run : return unhashed_paths = collections . OrderedDict () for path , path_info in paths . items (): if self . is_already_hashed ( path ): yield path , None , False else : unhashed_paths [ path ] = path_info yield from super () . post_process ( unhashed_paths , dry_run = dry_run , ** options ) def stored_name ( self , name ): if self . is_already_hashed ( name ): return name else : return super () . stored_name ( name ) class StaticFilesStorage ( HelpfulExceptionMixin , CompressedStaticFilesMixin , SkipHashedFilesMixin , ManifestStaticFilesStorage ): """ Similar to whitenoise.storage.CompressedManifestStaticFilesStorage. Add a mixin to avoid hashing files that are already hashed by webpack. """

After you create this file, enable the storage engine with: