An evolution of Continuous Integration

Continuous Integration (CI) is ubiquitous. Virtually all software industries today use the practice to automate their integration processes, helping to ensure that software development has the required quality standards through the various processes, using a Version Control System (VCS) and Automated Tests. Typically, CI uses a build server to implement the continuous processes of applying general quality control and of reducing the manual Quality Assurance (QA) processes.

Continuous Delivery is the natural evolution of CI, since it provides the processes to ensure that any of the developments in the CI cycle are in an easily deployable state. In the context of mobile development, Continuous Delivery is the ability to produce valid builds that are ready to be published in an app store. Continuous Deployment is the final status of the process, which ensures that any of the changes applied to the code is automatically deployed to the production environment.

In this post, I’m going to break down the advantages and challenges posed by Continuous Delivery, and demonstrate how you can look to implement it for Android mobile development.

Is Continuous Delivery right for you?

The common CI scenario

Forming the backbone of CI is Version Control, built on rock-solid Unit Tests that reduce error-prone code and raise the quality of the final result. Version Controls are used by everyone from small dev teams all the way up to enterprise organisations. While traditionally, CI uses a locally-built and maintained server, many companies have now begun migrating their CI process to a cloud-based solution, primarily because of the flexibility and control that cloud computing can give you over costs.

Breaking down the challenges

Despite the proliferation of CI, not all companies will be able to implement Continuous Delivery — let alone Continuous Deployment — because of the challenges involved. There are several reasons why mobile developers might struggle to automate deployments. One example is a review step in the iOS deployment process, while some of the Android QA processes can require tests on the alpha or beta channel before proceeding with the final deployment. But for those who can overcome these hurdles, replacing a CI process with Continuous Delivery can bring huge improvements to mobile development.

Meeting company policy

It’s important to consider a company’s security policies when looking to implement Continuous Delivery. Locally-built CI servers can give you greater control over the information being stored and processes being run during development, and some companies have very tight rules on what can be run outside the organisation’s local network. This means that for some, the sharing of release keys and certificates will not be an attractive option.

Key signing… and a possible solution for mobile developers

The Android release deployment process requires a key to sign off the production build before it moves to the Google Play store. Keeping keys and passwords on VCS is bad practice, because VCS is available to any member with access to the project, which includes any temporary 3rd parties. It’s common practice therefore to create release builds on a local computer, but this erodes one of the great advantages to Continuous Delivery: having a scalable online platform and the ability to easily deploy any stages of the CI at any time.

Because of the need to secure the signing keys of the app, building Continuous Delivery for mobile development can be tricky. For Android app development, you could improve security further by using the new Google Play App Signing, which applies a second key when it is uploaded to the Google Play Store. But even after employing this method, there will still be at least a key and a couple of passwords required to build a release (signing key, key and upload password). The CI system needs these to build an APK ready for release, but as mentioned it’s not a good idea to put them on the Version Control. However, using Encrypted Cloud Storage services is one solution to this problem; they can keep the keys safe and are only limited by the integration capabilities of your chosen CI system.

A speed and cost boost

While a local CI server does not require a powerful computer to run build processes, the execution of tests can be time-consuming. If a regression test takes dozens of hours to complete, it becomes potentially expensive in terms of time lost to testing. Furthermore, a locally-based server will have associated costs for maintenance and eventual replacement.

By comparison, a cloud-based solution gives you the opportunity to use only what you need and when you need it. This means you can scale up or down at will, making the process cost-effective and without the need for ongoing maintenance. It offers a highly-efficient configuration in terms of compute power, reliability and flexibility.

Implementing Continuous Delivery

Picking the right tools for the job

Assuming your Version Control System is based on GIT or Mercurial (either is a valid option), you can choose from a number of CI systems, including:

TeamCity

Travis CI

Jenkins

Bamboo

GitLab CI

CircleCI

This post uses CircleCI, GitHub and AWS as the preferred available solutions, but any of the other services can be valid considerations, as long as they provide the minimum required specifications.

Making a start with GitHub & CircleCI 2.0

Using CircleCI (otherwise known as CI system) begins with the integration of the Version Control System, an easy feat to accomplish with GitHub and CircleCI. You start the process by setting the GitHub Key in CircleCI. You can choose to set a GitHub Deploy Key that can allow access only to a specific repository, or a User Key that allows CircleCI to get access to all the user repos. Once you have the Deploy or User Key, the process to set it in CircleCI is to go to Project Settings > Checkout SSH keys and add the SSH Key.

The process is explained quite well in the GitHub security and SSH keys documentation.

CircleCI 2.0 — Anatomy for Android

CircleCI 2.0 comes with huge improvements for Android processes, but the benefits can be summarised with two main points:

Dockerized Android Images

CircleCI 2.0 provides all the necessary SDK and Build Tools in a Docker image to build an Android app (which means hassle-free SDK downloads and automated licence acceptances!)

provides all the necessary SDK and Build Tools in a image to build an Android app (which means hassle-free SDK downloads and automated licence acceptances!) Workflows

The ability to build specific workflows based on dependencies running in parallel

Furthermore, CircleCI 2.0 drastically reduces the CI process time. We saw a 60% reduction in processing time compared with CircleCI 1.0, so I would strongly urge you to move directly to 2.0 for the new projects or migrate the old 1.0 to the 2.0.

Here a CircleCI 2.0 config.yml sample

version: 2 references: # ----------------------------

# Cache Configuration

# ---------------------------- cache_key: &cache_key

key: cache-{{ checksum "gradle/wrapper/gradle-wrapper.properties" }}-{{ checksum "build.gradle" }}-{{ checksum "app/build.gradle" }}

restore_cache: &restore_cache

restore_cache:

<<: *cache_key

save_cache: &save_cache

save_cache:

<<: *cache_key

paths:

- ~/.gradle

- ~/.m2 # ----------------------------

# Docker Configuration

# ---------------------------- android_config: &android_config

working_directory: ~/your-project-root

docker:

- image: circleci/android:api-27-alpha

environment:

TERM: dumb

ANDROID_SDK_VERSION: 27

ANDROID_BUILD_TOOLS_VERSION: 27.0.2

GRADLE_OPTS: '-Dorg.gradle.jvmargs="-Xmx2560M -XX:+HeapDumpOnOutOfMemoryError"' # ----------------------------

# Job Section

# ---------------------------- jobs: # Job Debug

build_debug:

... # Job Release

build_release:

... # Unit Tests

unit_test:

... # ----------------------------

# Workflows

# ---------------------------- workflows:

version: 2

debug_build_deploy:

jobs:

- build_debug: release_build_deploy:

jobs:

- build_release:

In the reference area, the Cache and Docker Configuration ensures that the environment for the Android build is installed and working correctly, handling the cache strategy and loading the prepared Docker image for Android. Before CircleCI 2.0, these steps were not Dockerised, which meant that they were much more time consuming. This step in 2.0 is what makes up the majority of your efficiency savings for the build time, when compared with 1.0.

The Job sections points to the single available atomic tasks grouped under a custom name (i.e. build_debug, build_release, etc…).

Last but not least, the Workflow area defines the execution of the job strategy above.

Fabric, AWS and the custom helper

To support the delivery process, the Fabric Beta service is an excellent choice to internally distribute the build to the proper departments (QA, Marketing, Advertisement, etc…) while AWS stores, in an encrypted S3 bucket, the app signing key and certificates.

Since the Fabric Beta deployment can easily be configured in the app project and executed through a gradle task, the AWS integration needs a specific customisation for the CircleCI 2.0 config.yml.

It’s good practice to create a specific IAM user within AWS for the CircleCI configuration and to assign it a specific KMS key to be used for the encrypted S3 bucket.

Configure CircleCI to get AWS access

To grant CircleCI access to AWS, within CircleCI, go to Project Settings > AWS permission. Supply CircleCI with the Access Key ID and Secret Access Key for the new AWS IAM user, which are generated when you create the user in AWS.