Whoops! We need to add tests to the validate instance method to keep Undercover happy. If we add those and run undercover again, we’ll be back to green.

Compare to branch or commit

We have analysed test coverage for the current diff, but what about bigger chunks of work? Undercover can perform the same analysis with respect to any branch or commit. This is where the -c or --compare option comes in handy.

Try this set of command arguments:

undercover --compare master

It will look at the most recent coverage report and find untested methods in the entire diff between HEAD and what we’ve set in --compare . Comes in handy for various branching workflows!

Automate

We now have what we need to create the beginning of an automated feedback loop. Let’s get started with git hooks.

What are they? Git hooks are customisable scripts that live in .git/hooks of every repository and are executed before or after git events. They can be edited manually, but I suggest to start with a helper gem called overcommit, which does an excellent job managing community standard git hooks and custom ones.

Follow Overcommit’s readme to install it and place the contents of this suggested config in .overcommit.yml . An starter config should have been created during installation.

PrePush:

Undercover:

enabled: true

required: true

command: ['undercover', '--compare', 'master']

This is my config of choice that ensures all tests are in place before code is pushed ( PrePush ). You might want to change this to suite your preferences and perhaps use a PreCommit variant instead. It’s also good to know that hooks can be disabled in an emergency by setting OVERCOMMIT_DISABLE=1 or using the --no-verify option in commands like commit or push .

Set up Continuous Integration

The next steps will cover adding Undercover to analyse commits project-wide on the CI server, so that we close the feedback loop and make it part of the code review process.

Step 1: Build status check

We’ll start with a status check for test coverage that runs tests and then Undercover. I have prepared a few starter configs for a few popular CI services: Travis CI, CircleCI, Semaphore and Codeship. They are also stored in the repo in case you need them later.

Travis CI

The sample .travis.yml config was taken straight from the undercover repository. It fails the build based on coverage results and failures can be inspected within the output log. This is not ideal, but works just fine.

# .travis.yml

language: ruby

rvm:

- 2.5.1

- 2.4.4

- 2.3.4

before_install:

- gem install bundler undercover

- gem update --system

script:

- bundle exec rake

- git pull origin master:master # required for --compare below

- undercover --compare master

It should be also possible to run a coverage check a separate TravisCI job, but I haven’t looked into it yet. It appears that passing a coverage report (or any data, really) between Travis test executors requires an external storage layer like S3.

CircleCI

Sample single-job config for CircleCI that runs specs and validates coverage with Undercover.

# .circleci/config.yml

version: 2

jobs:

build:

docker:

- image: circleci/ruby:2.5-browsers

steps:

- checkout

- run:

name: Install dependencies

command: |

sudo apt-get install cmake

bundle install

- run:

name: Run RSpec

command: bundle exec rspec

- run:

name: Check coverage

command: |

gem install undercover

undercover --compare origin/master

We can go even further and create a separate job for analysing coverage, leveraging features of CircleCI 2.0. That is useful, because we can benefit from workflows and build conditions, but requires a bit more work. Having that in mind, there is another more advanced starter config available on GitHub, where the LCOV report file is shared between multiple CircleCI jobs to create individual build stages for test and coverage .

Semaphore

Semaphore pipeline example with a single job running rspec and undercover :

version: v1.0

name: RSpec + Undercover

agent:

machine:

type: e1-standard-2

os_image: ubuntu1804

blocks:

- name: "RSpec"

task:

jobs:

- name: Run Specs and Check Coverage

commands:

- checkout

- bundle install

- bundle exec rspec

- gem install undercover

- undercover --compare origin/master

Codeship

To run specs and analyse coverage in Codeship, go to Project Settings and place the following instructions in the Setup Commands section:

rvm use 2.5.3 --install

bundle install

gem install undercover

Then, run specs and undercover as part of the Test Pipeline:

bundle exec rspec --format documentation --color # pull origin/master to have a ref to compare against

git remote set-branches --add origin master

git fetch undercover --compare origin/master

One caveat is that Codeship uses a shallow git clone, so we need to specifically add the master branch (or any other branch we want to compare with) to the cloned repository. This way we can use undercover --compare origin/master .

Merging test results from multiple test runners

Another common caveat that applies to most CI services supporting parallel execution is about partial test results. Undercover does not currently support partial coverage reports, so we need to merge them into one before running the undercover command. If this applies to you, this gist might be a good starting point.

Step 2: Automated GitHub comments

We have already closed the feedback loop with the build status, but that’s not all we can do. How about not having to inspect the build output every time untested code is detected? We can create automated pull request comments thanks to Pronto and its integration with Undercover!

Pronto is an open-source automated code review framework that integrates with multiple tools (Rubocop,Reek, Brakeman included) and offers integrations with SCMs like GitHub, GitLab and Bitbucket.

That’s how automated code review looks like at Rainforest