The Requirements

2 Ubuntu 16.04 LTS VMs in a Private Cloud Create 3 environments (Dev, UAT and Prod) No Containers :( Existing Enterprise System Flask API using Python 3 Encrypted communication: HTTPS

The Questions

How will we validate the changes to the different environments separately? i.e. code and dependencies. How will we serve the Flask API? How will we isolate 2 environments in a VM and make everything ready for automation without a sea of manual configuration on every deployment? And finally, how will we secure the connection to existing systems?

The Solution

Because every problem is potentially a problem that has been fully or partially solved, we rolled up our sleeves and went on the Research Phase. We found a series of common practices in the Python world to solve parts of our problem and incorporated those into the solution.

3 long lived branches on Github: master , staging , dev Python virtualenv: app-env-dev , app-env-test and app-env-prod Gunicorn serving the Flask API as a systemd service. Nginx: reverse proxy, api gateway and certificate validation

The Architecture

How we did it

The next steps are all done from your cloud instance, where the components should be installed and configured.

Get the code

For this part, you should have your Flask Application in your repository (here represented by App ), with Flask and Gunicorn defined as dependencies in requirements.txt .

Create directories for DEV (dev) and TEST (staging).

$ mkdir ~/DEV $ cd ~/DEV $ git clone https://github.com/App $ git checkout dev

Now this location has your dev branch mapped to it. The idea is that a simple git pull is enough to bring changes in and deploy for testing.

Install Python's virtualenv

Each environment should have its own Python virtual environment. The reason for that is: it makes changes to the dependencies testable in an environment updated according to the declared requirements. Similar to what Docker would do wrapping an application environment in an image. Comparably, #2 is your Docker FROM ("clean" room), while #4 is your environment setup. The idea

$ sudo apt-get install virtualenv $ sudo virtualenv -p python3 /opt/app-env-dev $ source /opt/app-env-dev/bin/activate (app-env-dev)$ pip install -r ~/DEV/App/requirements.txt (app-env-dev)$ deactivate

Install Nginx

Nginx will be our reverse proxy and API gateway, re-writing the requests hitting the different environments based on the prefixed URL: /dev and /test .

$ sudo apt-get install nginx $ sudo systemctl status nginx

Create a systemd service to run Gunicorn

Each environment will also have their own Gunicorn service running, through their own Python virtualenv, and binding to a different unix socket.

$ sudo /etc/systemd/system/app-api-dev.service

[Unit]

Description=Gunicorn instance to serve app-api

After=network.target



[Service]

User=admin

Group=www-data

WorkingDirectory=/home/admin/DEV/App/src/web

Environment="PATH=/opt/app-env-dev/bin"

ExecStart=/opt/app-env-dev/bin/gunicorn --log-level debug --worker-class=gthread --threads=5 --bind unix:/opt/app-env-sock/dev.sock --access-logfile /var/log/gunicorn/dev/gunicorn_access.log -m 007 wsgi:app



[Install]

WantedBy=multi-user.target

Configure Nginx

$ sudo vi /etc/nginx/sites-available/api-gateway

server {

listen 80;

server_name app-api.com;

return 301 https://$server_name$request_uri;

}



server {

listen 443 ssl;

ssl_certificate /opt/ssl/crt.pem;

ssl_certificate_key /opt/ssl/key.pem;



server_name app-api.com; location /dev {

include proxy_params;

rewrite ^/dev/(.*) /$1 break;

proxy_pass http://unix:/opt/app-env-sock/dev.sock;

} #Add here location for /test environment }

Notice we have crt.pem and key.pem under /opt/ssl and pass the location to Nginx, which is going to be the piece responsible for:

Redirecting any traffic coming into :80 to :443 Validating SSL certificates against clients' requests Rewriting the incoming request URI to remove the environment prefix and forwarding to the associated Unix socket.

We should now enable Nginx to serve the API gateway in the shared environment.

$ sudo ln -s /etc/nginx/sites-available/api-gateway /etc/nginx/sites-enabled

Start services

With all pieces in place now, we are ready to start and enable the services.

$ sudo systemctl start app-api-dev : to start Gunicorn DEV environment $ sudo systemctl enable app-api-dev : to start Gunicorn DEV environment at VM startup $ sudo systemctl status app-api-dev : to verify the status of the service $ sudo systemctl restart nginx : to restart Nginx.

Conclusion

Although working with Kubernetes and Docker containers makes more sense from an App Development perspective, sometimes we need to take requirements and apply the best of our knowledge to provide a feasible solution and prepared for automation.

In the next post, we will be digging into how we can enhance the Git flow and automate testing and deployment of this solution using TravisCI.