There’s currently no plugins for Jenkins to tie in nicely with any sort of Elixir testing framework. This lack of plug-n-play makes for another good case to leverage more Docker magic.

So for this test, I wanted to leverage Docker-Compose so that I could future proof our unit tests. Even though I know unit tests should not rely on any external backend services, sometimes its unavoidable — so I wanted to make sure we would have room to grow in case I needed to support unit tests that needed a Docker container running SQL or ElasticSearch or something like that.

The first problem we ran into was getting tests out of Elixir into a format Jenkins could parse. JUnit is the obvious choice for Jenkins-Junkies, so we’re using junit_formatter to output our tests results in XML. To get this working edit your deps in mix.exs to include junit_formatter . Below is a sample from what we’re using for our testing dependencies:

#mix.exs defp deps do

[

#Other deps go here.... # dev/test deps

{:exvcr, "~> 0.10", only: :test},

{:stream_data, "~> 0.4.0", only: [:dev, :test]},

{:credo, "~> 0.9.0-rc1", only: [:dev, :test], runtime: false},

{:distillery, "~> 1.5", runtime: false},

{:excoveralls, "~> 0.8", only: :test},

{:dialyxir, "~> 0.5.0", only: [:dev], runtime: false},

{:junit_formatter, "~> 2.2", only: [:test]}

]

end

Then to have our tests use the junit_formatter , we have a helper test file that is configured to use JUnitFormatter

#test_helper.exs ExUnit.configure(formatters: [JUnitFormatter, ExUnit.CLIFormatter])

ExUnit.start()

We’re only launching a single container running our application, so our compose file is pretty simple. Below is our docker-compose-test.yml . A very important part of this is the volumes section — this makes sure that our docker container /opt/app directory is mounted to our $PWD when Jenkins runs the job. This makes it a lot easier to grab test results and publish them out of the ./_build directory after the test runs.

#docker-compose-test.yml version: '3'

services:

web:

build:

context: ./

dockerfile: Dockerfile.test

command: ./infra/build/test/unittest.sh

volumes:

- .:/opt/app

Our Dockerfile.test is very simple as well (I’d like to move to the Alpine Elixir image, but we ran into issues using it, so we’re using the big fat Elixir image instead)

#Dockerfile.test FROM elixir:1.6.5 ENV REFRESHED_AT=2018-04-06 \

# Set this so that CTRL+G works properly

TERM=xterm RUN apt-get update \

&& update-ca-certificates --fresh \

&& rm -rf /var/lib/apt/lists/* \

&& mix local.hex --force \

&& mix local.rebar --force \

&& mix archive.install hex sobelow --force WORKDIR /opt/app

unittest.sh is also a straight forward script to run our Elixir tests and linters:

#!/bin/bash set -e echo "## get deps"

mix do deps.get, deps.compile, dialyzer --plt echo "## check for warnings"

mix do clean, compile --force --warning-as-errors echo "## checking format"

mix format --check-formatted echo "## run credo for linting"

mix credo --strict echo "## running tests"

mix test --trace echo "## run coverage"

mix coveralls.html echo "## run dialyzer"

mix dialyzer echo "## run sobelow security check"

mix sobelow --ignore Config.HTTPS,Config.CSRF,Config.Headers --private --verbose --exit

To run our tests, we do some magic to test if docker-compose-test.yml has any changes since the last build and force a rebuild if it has been changed, otherwise we just run normally. Here’s our docker_wrapper.sh script that runs it all:

#docker_wrapper.sh if $DOCKERCOMPOSE_CHANGED

then

echo "## rebuilding docker-compose"

_cmd="docker-compose --file ./docker-compose-test.yml up --force-recreate --abort-on-container-exit --build web" else

echo "## skipping docker-compose rebuild"

_cmd="docker-compose --file ./docker-compose-test.yml up --abort-on-container-exit"

fi echo "## running tests in a docker container"

echo "> ${_cmd}"

/bin/bash -l -c "${_cmd}"

Tying it all together on Jenkins is now a breeze. Whenever a PR or a check-in comes in to master, we trigger tests.