Continuous Integration and Deployment for Rails using CircleCI Updated Jun 28, 2019

7 minute read

Continuous integration and delivery pipeline can have a significant impact the dev team’s productivity and stability of production releases. In this tutorial, I describe how to automate testing, security checks, and deployments for Ruby on Rails apps using CircleCI. I cover a basic CI setup as well as more advanced features like concurrent specs, dependencies caching, NodeJS/Webpack setup, Heroku deployments, and GitHub integration.

It’s a quickstart introduction and high-level overview of CircleCI continuous integration setup. For more in-depth info you can check out the official docs.

The post is written in the context of Ruby on Rails tech stack, but with small changes, the same approach can be used with any language.

CircleCI workflow phases

CircleCI continuous integration workflow in action





CircleCI pipelines are configured using so-called workflows. Each of them can consist of multiple phases. Our sample Rails pipeline app begins with a setup phase, that is responsible for downloading, installing and caching all the dependencies.

test phase is parallelized to speed up specs execution. Parallelization is not needed for this particular sample app because test suite is small, but mature Rails apps test suites can take even over an hour to execute so parallelizing them is often necessary.

deploy phase pushes the newest code changes to the production server. In this tutorial I describe how to set up Heroku integration.

CircleCI config.yml for Rails apps

Let’s start from a final form of .circleci/config.yml file and down the road I’ll explain it more in detail. This particular config comes from one of my side projects that I’ve recently open sourced. Feel free to check it, and it’s corresponding CircleCI dashboard.

version : 2 jobs : setup : docker : - image : circleci/ruby:2.6.2-node steps : - checkout - run : gem update --system - run : gem install bundler - restore_cache : keys : - bundle-{{ checksum "Gemfile.lock" }} - run : bundle install --path vendor/bundle - save_cache : key : bundle-{{ checksum "Gemfile.lock" }} paths : - vendor/bundle - restore_cache : keys : - yarn-{{ checksum "yarn.lock" }} - run : yarn install --cache-folder ~/.cache/yarn - save_cache : key : yarn-{{ checksum "yarn.lock" }} paths : - ~/.cache/yarn - run : bundle exec rake webpacker:compile - save_cache : key : webpack-{{ .Revision }} paths : - /home/circleci/project/public/packs-test/ test : docker : - image : circleci/ruby:2.6.2-node environment : DATABASE_URL : postgresql://postgres: [email protected] :5432 REDIS_URL : redis:// [email protected] :6379 - image : circleci/postgres:11 environment : POSTGRES_USER : postgres POSTGRES_DB : price_watcher_test POSTGRES_PASSWORD : secret - image : circleci/redis:latest parallelism : 2 steps : - checkout - restore_cache : keys : - webpack-{{ .Revision }} - restore_cache : keys : - bundle-{{ checksum "Gemfile.lock" }} - run : gem update --system - run : gem install bundler - run : bundle install --path vendor/bundle - run : sudo apt install postgresql-client - run : dockerize -wait tcp://localhost:5432 -timeout 1m - run : bundle exec rake db:create - run : bundle exec rake db:schema:load - run : name : Additional checks command : | if [ $CIRCLE_NODE_INDEX = 0 ]; then bundle exec bundle-audit update bundle exec bundle-audit check elif [ $CIRCLE_NODE_INDEX = 1 ]; then bin/rubocop fi - run : name : Specs command : | TESTFILES=$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings) bundle exec rspec $TESTFILES --profile 10 --format RspecJunitFormatter --out ~/spec-timings/rspec.xml --format progress - store_test_results : path : ~/spec-timings deploy : docker : - image : buildpack-deps:trusty steps : - checkout - run : name : Deploy to Heroku command : | cat >~/.netrc <<EOF machine api.heroku.com login $HEROKU_EMAIL password $HEROKU_API_KEY machine git.heroku.com login $HEROKU_EMAIL password $HEROKU_API_KEY EOF chmod 600 ~/.netrc curl https://cli-assets.heroku.com/install.sh | sh git push https://heroku: [email protected] /rails-app.git master heroku run rake db:migrate -a rails-app no_output_timeout : 10m workflows : version : 2 setup_and_test : jobs : - setup - test : requires : - setup workflows : - deploy : requires : - test filters : branches : only : master

Docker images based setup

Docker is a first-class citizen in CircleCI workflows setup. You can define the main image inside which tests and checks are executed, as well as supporting images, e.g., for databases.

Thanks to this approach time spent on setting up the environment can be minimized thus speeding up the whole process. CircleCI offers a whole array of supported images. They should be enough for most CI scenarios, but in case your requirements are more complex you can always use your own Docker image published into the public or private repository.

Caching dependencies

CircleCI offers powerful caching features. For our sample Rails app, we need to install both Ruby Gems and Node modules dependencies, as well as results of Webpack assets compilation. Cache keys for dependencies are a checksum of lock files, so an actual install only takes place when something changed. For Webpack compilation we use .Revision variable as a cache key to refresh it with every new commit.

Using this approach allows minimizing the time spent on installing dependencies. It is done only once in a single-threaded job, then multithreaded test jobs can reuse the cache.

Parallel execution of Rails Rspec specs in CircleCI

A test suit of a legacy Rails app can be very slow to run, discouraging developers from using it. Parallelizing specs execution is a no brainer approach to optimizing your CI pipeline speed.

Rails 6 is going to add native support for parallel specs, but CircleCI already offers a decent solution to perform it.

Check out the following config lines:

- run : name : Specs command : | TESTFILES=$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings) bundle exec rspec $TESTFILES --profile 10 --format RspecJunitFormatter --out ~/spec-timings/rspec.xml --format progress - store_test_results : path : ~/spec-timings

It creates a TESTFILES variable, that contains spec file names, split equally for each parallel job. A cool feature is that thanks to saving specs execution speed in ~/spec-timings , each worker runs specs with similar execution time, so there is no danger of a single overworked job delaying the whole suit.

With this config, you can speed up your reduce spec suite time by adding more workers and increasing the parallelism config option.

Execute custom tasks

Additional tasks like Rubocop linting or bundler-audit security checks are an invaluable addition to a robust CI pipeline.

You could add a seperate workflow job for them, but they are usually much faster to execute than the actual specs, so paying for an additional job is not needed.

Instead, you can add executing those check scripts to single jobs from parallelized spec workers using the following config:

- run : name : Additional checks command : | if [ $CIRCLE_NODE_INDEX = 0 ]; then bundle exec bundle-audit update bundle exec bundle-audit check elif [ $CIRCLE_NODE_INDEX = 1 ]; then bin/rubocop fi

CIRCLE_NODE_INDEX set by CircleCI represents the index of a parallel worker job. With this config first of the parallel workers performs security checks using bundle-audit second lints codebase with Rubocop etc.

Automatic deployments to Heroku

Depending on your infrastructure setup a deployment script will differ. I explain how to setup automatic deployments for Heroku because it is a bit more complex due to Heroku CLI login hack needed:

deploy : docker : - image : buildpack-deps:trusty steps : - checkout - run : name : Deploy to Heroku command : | cat >~/.netrc <<EOF machine api.heroku.com login $HEROKU_EMAIL password $HEROKU_API_KEY machine git.heroku.com login $HEROKU_EMAIL password $HEROKU_API_KEY EOF chmod 600 ~/.netrc curl https://cli-assets.heroku.com/install.sh | sh git push https://heroku: [email protected] /rails-app master heroku run rake db:migrate -a rails-app no_output_timeout : 10m ... workflows : version : 2 setup_and_test : jobs : - setup - test : requires : - setup - deploy : requires : - test filters : branches : only : master

This config assumes that your Heroku app is called rails-app so make sure to change it accordingly. You must add HEROKU_API_KEY and HEROKU_EMAIL variables via CircleCI UI. HEROKU_API_KEY is generated by running:

heroku authorizations:create

Workflows config ensures that deploy workflow phase executes only for a master branch pushes. You might want to tweak this config if your GitHub deployment workflow is different.

Modifying .netrc file is necessary if you want to execute migration or any other rake tasks after each deploy using authenticated Heroku CLI.

GitHub integration settings

How to set up a productive GitHub workflow is a story for another blog post. Just make sure to select building CI pipeline only for pull requests and cancel redundant build in CircleCI options. Otherwise, your CI queue might get stuck if there are more developers pushing commits to the same project.

Those CircleCI GitHub settings can prevent your CI execution queue from getting stuck

Summary

I hope you’ll find some of those tips useful when setting up your continuous integration pipeline using CircleCI. Let me know if you notice some ways how this setup could be improved.