In the first part we’ve seen how to set up an Elastic Beanstalk environment on AWS with a simple Flask application running. Suppose you want to change some code in your application and show the result to your client or a testing team. How do you manage to redeploy your application automatically? That’s the goal of the second part of this article. We will be using Gitlab CI/CD to redeploy our application.

Link to the first part of this article

Continuous integration and continuous delivery tools

You certainly know tools like Jenkins or Travis. These tools were designed for continuous integration and continuous delivery (CI/CD). Jenkins or Travis can be easily integrated with other tools, like source controls like GitHub or Gitlab, project management tools / issue trackers like Jira or communication systems like Slack. Sometimes you want to keep your stack of tools as simple as possible and centralize all these need to one place. That’s the idea behind Gitlab which includes some project management features but most importantly for our use case, with the Gitlab CI/CD features we can avoid using a specific tool like Jenkins and Travis.

Diving into Gitlab CI/CD features

So how does it works? Globally, you can configure a CI/CD pipeline (an ordered list of scripts) which will be launched on machines called Gitlab Runners.

These machines can be provided :

On-Demand : the Runners are owned by Gitlab and you have a limited shared resources with other users (scripts can be longer to start and execute)

On-Premise : Runners are part of your infrastructure (like Gitlab can be installed locally), so you can allocate the amount of memory you want to your CI/CD pipeline, which results in better performance.

We will see the On-Demand solution, which is easier and ready to use without too much effort.

We can configure a Gitlab CI/CD pipeline by adding a file to our project named .gitlab-ci.yml. You can check the official documention here.

Step 1 — Add the .gitlab-ci.yml

Now some code ! We will configure a two steps pipeline, the first step for some unit tests and secondly the deployment part.

The first thing to do is to add an empty .gitlab-ci.yml file at the root of our project initialized in the part 1. On the next push, Gitlab will auto-detect this file and activate the Gitlab CI/CD features on your project.

Step 2 — Add unit tests

It’s a good practice to check the new code against regressions before deploying it, so that will be the first goal of our Gitlab CI/CD pipeline.

Create a new python package tests at the root of the project and add a file test_dumbclass.py with this content :

This test do nothing, but in the future you will be able to implement your unit tests based on your own expected behaviors.

Important : naming the tests files with the prefix “test_” will help us automatically discover which files should be run, as we will see further.

Step 3 — Run the unit tests from Gitlab CI/CD

Add this content to the .gitlab-ci.yml :

image: python:3.6-stretch





before_script:

- pip install -r requirements.txt



test:

script:

- python -m unittest discover tests

Commit / push this code and navigate to your Gitlab project. You should see a job running automatically. After few minutes or maybe less, you should see the job indicating “passed” :

And you should also note at the end of the job logs (when you click to have more details) :

$ python -m unittest discover tests

.

----------------------------------------------------------------------

Ran 1 test in 0.000s



OK

Job succeeded

Congratulations for your first Gitlab pipeline ! This one is really minimalistic, but it’s good for a quick start. Let’s understand what we’ve just done in the .gitlab-ci.yml file :

The image parameter indicate to the Gitlab runner to pull and use a Docker image to install Python. The image details can be found on the Docker hub and you can easily choose another version if you want. Using a Docker image is easier than executing a specific command like sudo apt-get install python3.6 which depends on the operating system you are using.

parameter indicate to the Gitlab runner to pull and use a Docker image to install Python. The image details can be found on the Docker hub and you can easily choose another version if you want. Using a Docker image is easier than executing a specific command like which depends on the operating system you are using. As a second step, we can define a before_script section containing commands we want to be executed before the pipeline start. For example, we may want to install the project dependencies described in the requirements.txt.

section containing commands we want to be executed before the pipeline start. For example, we may want to install the project dependencies described in the requirements.txt. Finally, we are configuring our test stage with a single script to automatically discover our unit tests and run them : python -m unittest discover tests .

Step 4 — Configure the deployment to Elastic Beanstalk

First, we will refactor our configuration file by cleanly divide our pipeline into stages. Just add this code after the image parameter :

image: python:3.6-stretch stages:

- unit_test

- deploy ...

We should also edit our test section with by specifying the name of the stage :

test:

stage: unit_test

script:

- python -m unittest discover tests

Then, we will add a deploy_aws section : it will use the EB CLI to deploy our application in the Elastic Beanstalk environment named “flask-starter-dev”, the one we created in the first part of this article.

deploy_aws:

stage: deploy

before_script:

- pip install awsebcli --upgrade --user

script:

- /root/.local/bin/eb deploy flask-starter-dev

We added a local before_script section to the deploy_aws section, meaning this will be only executed in the deploy stage. We need the before_script to install the EB CLI and then we just run the eb deploy command. Notice that we use the same commands in the part one of this article, but we run it on a remote machine !

If we push the configuration file, we have now two jobs running :

But oops, deploy_aws failed ! By accessing the job logs you can see what’s going on :

$ /root/.local/bin/eb deploy flask-starter-dev

WARNING: Git is in a detached head state. Using branch "default".

WARNING: Git is in a detached head state. Using branch "default".

WARNING: Git is in a detached head state. Using branch "default".

ERROR: InvalidProfileError - The config profile (eb-cli) could not be found

ERROR: Job failed: exit code 1

The main error is that we didn’t provide the necessary AWS credentials to the EB CLI, remember the part one ? We are going to do quite the same.

Step 5 — Configure the AWS credentials

In the part one of this article, we have configured the AWS credentials with the command eb init and by typing the asked aws-access-id and the aws-secret-key . This command is creating a file ~/.aws/config which contains :

[profile eb-cli]

aws_access_key_id = XXXXXXXXXXXXXXXXXXXXXXX

aws_secret_access_key = XXXXXXXXXXXXXXXXXXXX

The previous error “InvalidProfileError — The config profile (eb-cli) could not be found” tell us that this exact same file is missing. So let’s do a bash script creating this file for us !

We should not hardcode the aws_access_key_id and the aws_secred_access_key , instead we will use Gitlab variables. For each projects, Gitlab let you define variables which can be used in your CI/CD pipeline, with the syntax ${MY_VARIABLE_NAME} . Go on your Gitlab project, then on the “Settings > CI / CD” menu. You should fill the “Variables” section with these two variables :

We are all good, we just need to execute the .eb-config.sh bash script from our Gitlab CI/CD pipeline by editing the .gitlab-ci.yml file :

deploy_aws:

stage: deploy

before_script:

- pip install awsebcli --upgrade --user

- chmod +x ./.eb-config.sh

- ./.eb-config.sh

- git checkout master

script:

- /root/.local/bin/eb deploy flask-starter-dev

Don’t forget to add execution rights like we did with the chmod +x command, unless it will not work.

Notice that we also solved the “WARNING: Git is in a detached head state. Using branch default” message. We want to deploy the code located on the master branch, not the “default” branch, so we are using git checkout master .

So, if we put all together, our final .gitlab-ci.yml config file look like this :

image: python:3.6-stretch



stages:

- unit_test

- deploy



before_script:

- pip install -r requirements.txt



test:

stage: unit_test

script:

- python -m unittest discover tests



deploy_aws:

stage: deploy

before_script:

- pip install awsebcli --upgrade --user

- chmod +x ./.eb-config.sh

- ./.eb-config.sh

- git checkout master

script:

- /root/.local/bin/eb deploy flask-starter-dev

Step 6 — Make a change in your app

Now make a change on your application.py file, replace for example :

@application.route('/')

def hello_world():

return 'Hello, World!'

With :

@application.route('/')

def hello_world():

return 'This is my new app'