How to Deploy a Serverless WSGI App using Zappa, CloudFront, RDS, and VPC

Posted on:

This post is more of a brain dump than a step by step guide. Every application is different some of the services, libraries, and techniques may not apply to you. Feel free to skip over them. This is how I deploy my apps with Zappa. I hope this post helps.

AWS Steps

These steps are AWS focued and I only listed them here so you know what happens. I recommend you write a CloudFormation template (use Troposphere) to automate these tasks for future deployments.

Create Two S3 Buckets You should only have to complete this stage once even if you create a CloudFormation template in the future. We are creating one bucket for Zappa to store our code when deploying (by default Zappa will only store it there briefly while deploying or updating the Lambda function) and another to store static files (css, images, and js). Configure your VPC Maybe your app doesn't need VPC. I personally believe if you are running a database that uses RDS and ElastiCache you need a VPC. Below is a list of things I recommend to get a Zappa powered app working with VPC, RDS, and Elasticache. What you'll typically need Public and Private Subnets Create at least two public and private subnets in different availability zones.

Create at least two public and private subnets in different availability zones. NAT Gateway Enables Lambda functions and instances (RDS and Elasticache) in a private subnet to connect to the Internet or other AWS services, but prevent the Internet from initiating a connection with those instances. Associate your NAT Gateway with one of your private subnets. This is needed if you are planning on using any API (MailChimp, Twilio, etc.) other than S3's.

Enables Lambda functions and instances (RDS and Elasticache) in a private subnet to connect to the Internet or other AWS services, but prevent the Internet from initiating a connection with those instances. Associate your NAT Gateway with one of your private subnets. This is needed if you are planning on using any API (MailChimp, Twilio, etc.) other than S3's. Route Table Controls the routing for your subnets. You could setup a different route table for each subnet but for the purposes of this guide we're setting one for the private subnets and one for the public ones. After we associate our subnets we will route 0.0.0.0/0 traffic to our NAT Gateway for our private subnets and 0.0.0.0/0 to our Internet Gateway for our public subnets.

Controls the routing for your subnets. You could setup a different route table for each subnet but for the purposes of this guide we're setting one for the private subnets and one for the public ones. After we associate our subnets we will route 0.0.0.0/0 traffic to our NAT Gateway for our private subnets and 0.0.0.0/0 to our Internet Gateway for our public subnets. VPC Endpoint for S3 This is needed so your Lambda function can communicate with S3's API. This comes in handy later when we run zappa manage "collectstatic --noinput". Create a MySQL, Aurora, or MariaDB instance(s) Create a MySQL instance(s) and associate it with the VPC and subnets you created in the previous step. Create Redis Cluster with ElastiCache Create a Redis instance(s) and associate it with the VPC and subnets you created in step 2.

Code & Local Environment Steps

You got it!! These are the steps that will require a change in your code or your local environment. Mainly the stuff you normally do when you start a new Django project.

Install virtualenv, virtualenvwrapper, and eva globally (Optional) This step is only useful if you plan on working on this project locally without something like Docker or Vagrant. If you just install eva it will install the other two. sudo pip install eva Configure virtualenvwrapper Add three lines to your shell startup file (.bashrc,.profile, etc.) to set the location where the virtual environments should live, the location of your development project directories, and the location of the script installed with this package: export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/ source /usr/local/bin/virtualenvwrapper.sh Create virtualenv and .env file mkvirtualenv -a ~/workspace/yourblog yb Next, create an .env (example)file and then run the workon command again to load the environment variables. Install zappa, envs, django, mysql-python, django-storages, boto, and django-redis (Partially Optional) pip install zappa envs django mysql-python pip install django-storages boto django-redis Run django-admin startproject to well...start the project Only needed if you are deploying a Django app. django-admin startproject yourblog Replace the default settings.py values with environment variables, create an .env file, and create dev stage json settings file (yourblog-dev.json). Only needed if you are deploying a Django app. Click here for the example files Create a custom storage backend Only needed if you are deploying a Django app. Create a new package named util and in that package create a file named custom_storages.py (click here for the example files)

Zappa Steps

List of the various Zappa steps that you need to run.

Run zappa init This command just creates a zappa_settings.json configuration file. You will answer questions pertaining to the S3 buckets you created earlier and the default stage for your app. After the initial setup is complete open the settings file to reference the environment variable JSON file you created earlier and the private subnets and security groups you created. Here is an example file. Run zappa deploy dev This command will deploy the app to Lambda and setup API Gateway. You will only run this command one time. After the first time you will only need to run the update command. Run zappa manage dev migrate This command is only needed for Django projects. The zappa manage migrate command is the equivalent of python manage.py migrate generally used for Django apps. Run zappa certify dev Run this command to request a SSL cert from the Let's Encrypt project and apply it to your API Gateway stage. It uses the domain you entered above in the zappa_settings.json file for the stage you are deploying. Run zappa manage "collectstatic --noinput" This command runs another one of Django's management commands, collectstatic. It will gather all of the static files (css, js, images, and videos) and upload them to S3 from inside the Lambda function. Since the files were already uploaded inside of the zip file Zappa deploys the upload to S3 is really fast.

CloudFront Steps (Optional)

Here is where my approach differs from most. I like to use CloudFront for whole site delivery for dynamic websites and APIs. This allows your site to load much faster across the globe. Depending on the TTLs you require and what needs to be cached on an per-user basis you can also bear much heavier loads because the site is served from cache at the edge. There are a lot of large scale applications that use this model (ex. Slack) it is not that different from using Varnish or Fastly when you really think about it.

Create a SSL Cert with AWS Certificate Manager Create a SSL cert for the URL your end users will see. I generally create a request that has the apex domain (yourblog.com) but also supports wildcard (*.yourblog.com). These are not the same thing but can be achieved on the same cert request. Create a Multi-Origin, Multi-Behavior CloudFront Distro Below I'm going to list the some of your CloudFront ditribution's attributes you should put some thought into. Your distribution should be a web (not RTMP) distribution Use the domain returned after you completed the zappa deploy command earlier as the origin domain name (should be something like to dev-yourblog-apigw.yourdomain.com) Leave origin path blank Type whatever you want in Origin Id Origin Custom Headers can be left blank as well Viewer Protocol Policy should be always be set to Redirect HTTP to HTTPS

Allowed HTTP Methods should be set to GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE unless it is the behavior for the static files on S3 then you can just set it to GET, HEAD Forward Headers should be set to None (improves caching) if most of your site doesn't require the user to login and you don't have any forms that require CSRF protection. If require headers then whitelist the following Accept, Accept-Language, Cache-Control, HTTP_X_CSRFTOKEN, x-csrfmiddlewaretoken, Upgrade-Insecure-Requests, and Referer. Object Caching should be set to whatever you're comfortable with. Here is what I did for this blog, for the default behavior I set the TTL to 3600 seconds, for the homepage I set it for 300 seconds, for the django admin I set it for 0 seconds, for the static files and site media I set it for 3600 seconds.

Forward Cookies this falls into the same category of what does your application require. If you need cookies for authentication or some other purpose then you can select All or Whitelist (at this time API Gateway only forwards one cookie so the Zappa project decided to create only one, zappa). Query String Forwarding and Caching should almost always be Forward All Smooth Streaming should be No

Restrict Viewer Access should be No

Compress Objects Automatically should be Hell Yes How many origins should I have? Two, one for your application and one for the S3 bucket that holds your static files. Why do I have an origin for my S3 bucket too?

I like doing it this way because I cut down the DNS latency by converting my references to images, css, and js from something like this https://cdn.yourdomain.com/yourblog-prod-static/cs... to someting like this /yourblog-prod-static/css/base.css. Plus, its cool as hell.

Hopefully this post helps you get started or come up with your own strategy. Above all, I hope it helps your organization save money and time.

Please enable JavaScript to view the comments powered by Disqus.