In an ideal world, data scientists would focus their time on analysis and modeling, but there is often seemingly unavoidable overhead in visualizing and presenting their results. Because data scientists generally don’t have an infrastructure background, they often struggle to build out servers and use tools to visualize their modeling and analysis. At Insight, Fellows have to manually set up a web server on an Amazon cloud EC2 instance to deploy a Flask app to showcase their projects, which can take 6–8 hours or more, especially for those without prior experience. But, what if they could shrink that time 100-fold?

In this post, I will share the work I did in the Insight DevOps Engineering program, where I launched an open source project called One-click to deploy web apps in minutes. Check it out on GitHub.

From Local Web Development to the Internet

Local Flask development is so frictionless it conceals the hurdles required to host your application anywhere other than localhost. When you inevitably want your app hosted on the internet, your options, for the most part, fall into two buckets:

Managed services (e.g. Heroku, PythonAnywhere, WebFaction). DIY! Manually configure and maintain a web server yourself.

Anyone who has sifted through hosting options will attest to weaknesses on both sides.

Managed services are great because they hide the details you don’t care about (software updates, firewalls, and ports). Unfortunately, they also hide the details you do care about (SSH access, hardware, and complex dependencies). In contrast, the DIY option offers that missing granular control, but it’s at the expense of a lengthy, and error-prone, setup.

I was inspired to offer an alternative to both of these options at Insight, when I saw how long data science Fellows were taking to deploy their Flask apps. They wanted features that were more advanced than standard managed services, but didn’t have the technical know-how to quickly set up their cloud servers.

How can we get the best of both worlds?

One-click

The goal of my project, One-click, was to solve this deployment riddle for our Data Science Fellows. It is a command line utility that automates the DIY method (so it is as easy as deploying to a managed service, like Heroku) while still exposing access to the underlying resources that it provisions on AWS.

Here’s how you use it:

Give Terraform your credentials to create control AWS by setting the AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY environment variables. Run the shell command:

Two minutes later, the Flask app in the Github repo is hosted on a freshly configured EC2 instance. The new workflow is local development -> One-click -> hosted on the internet.

All resources are accessible through the AWS console.

The fresh EC2 web server automatically provisioned by One-click

One command brings everything down.

one-click destroy

Architecture

One-click is a thin Python CLI wrapper for Terraform configurations that provision an EC2 instance to build and run a custom Docker image for the target Flask application.

Terraform is a cloud-agnostic Infrastructure as Code (IaC) tool. It is a domain-specific language to provision cloud infrastructure on a variety of providers. With Terraform, you write code instead of clicking through a console or using the AWS CLI. The result is cloud resources that are generated as explicitly and automatically as the programs that will run on them.

Creating an EC2 instance with Terraform looks something like this:

resource "aws_instance" "flask_server" { ami = "ami-70e90210", instance_type = "t2.medium" }

The backbone functionality of One-click is to programmatically control the cloud. Terraform was the ideal choice for the following reasons:

Manages state: Terraform knows which resources are actively running and how to kill them.

Terraform knows which resources are actively running and how to kill them. Configuration management: Execute shell commands on remote servers, which is a great way to configure fresh instances.

Execute shell commands on remote servers, which is a great way to configure fresh instances. Flow control: If statements, for loops, simple data structures.

During an automated One-click deployment, Terraform first creates the instance that will be the web server, then configures it by remotely executing shell commands.

These server-side steps are: