Backstop is a simple-to-use tool to take screenshots of pages through various viewports.

viewport: a framed area on a display screen for viewing information.

The most common names used to describe viewports are device categories: mobile, tablet and desktop.

Another interesting option is to use breakpoints like those defined by Material Design. Material-UI, a React library on top of Material Design, implemented a subset of them which are quite intuitive.

Overview

Let’s first have a look at BackstopJS’s workflow (once it’s setup)

Generate references with backstop reference (create screenshots that will be compared against) and save them (if using a versioning tool like git, you could commit them, but we’ll see later a better alternative). Modify some code. Run the tests with backstop test to generate screenshots for the current version If a test screenshot differs from the reference, an error will be show ! To fix it, either fix your code because the reference is correct, or… Approve the new screenshot with backstop approve because the reference is inaccurate (for example because you added a new feature to your design). This will replace the reference so you need to save it again.

It is recommended to only version the references and ignore the test screenshot.

The workflow is very straightforward, let’s setup BackstopJS in our project and try it out.

Setup

First run backstop init to setup your project.

backstop creates by default a backstop.json file which contains all the configuration, including all scenarios you’ll want to run.

A very minimal configuration with a single scenario would look like this:

Whaaat ? It is very long already. I could already see a bottleneck appear in the near future when we’ll have hundreds of scenarios (very common for a business application). It will be very hard to maintain such a file.

Moreover, don’t we need several of these configuration files ? One to run all scenarios locally one time on the developer’s machine, one for continuous testing on file changes that will only run a subset of scenarios/viewports, and one for a CI engine like Travis

jq to the rescue !

jq is a lightweight and flexible command-line JSON processor. It is like sed for JSON data

json is a great format for APIs and such, but this is no configuration file material. You always want to document the non-obvious parts, let messages for other developers, and keep the sizes small.

That’s what makes YAML more suitable. So I decided to split off the config in several files

Since backstop still needs a json file, let’s auto-generate it and gitignore the generated file. It’s a good rule of thumb to never commit auto-generated files unless there’s a good reason for it, like lock files to insure all developers use the same packages versions.

let’s dive in the css:config command.

First, we use yq, which is a wrapper around jq, to parse the yaml files and return json content. All the scenarios are slurped together to form a single array of scenarios. Then we pipe the json content to jq which will merge all the scenarios (input) in the `scenarios` key of the configuration object.

(

cat backstop.yaml backstop_local.yaml # read the config file

| yq .; # overrided if needed

cat backstop_data/scenarios/* # read the scenarios files

| yq '.scenarios[]' # extract the scenarios

| jq -s . # slurp all scenarios

)

| # pipe the config and scenarios

jq '.scenarios=input' # merge the scenarios in the config

> backstop.json # Save the json result

Hey, your yq command fails when I try it, do you know why ?

There are different softwares called yq, so my guess is you installed the wrong one. At GOunite, we use kislyuk/yq.

In some cases you may be able to avoid writing a config file by using the node invocation e.g. Pass a config object to the command.

References

Once your setup is ready, execute backstop reference to generate your reference screenshots. It can be either a staging or production application, but we’ll see later why a staging instance makes more sense for dynamic applications.

Since we created a custom configuration, and to insure that a generated json file is present, we can instead run yarn css:reference .

This has created as many images in the directory backstop_data/bitmaps_reference/ as there are viewports times scenarios.

While you want to version these files in order for your team to keep testing new UI screenshots against these references, do not commit them in git! Your git repository’s size would quickly explode, making it very inconvenient for everybody.

A good solution for versioning large files is git lfs. This way, instead of saving full blobs of data in git, you’ll only save the hash to the remotely stored file (github integrates it seamlessly).

$ git lfs install

$ git lfs track "backstop_data/bitmaps_reference/*"

$ git add .gitattributes

$ git commit -am "feat: save backstop references with LFS"

$ git push

Then ask your colleagues to install git lfs and run git lfs pull to be up to date.

Tests

Run backstop test to create your test screenshots that will be compared with the references. Similarly to the references, we’ve wrapped it in a new command yarn css:test to insure the config file is there.

As a developer, if you run an instance of the application locally, you can quickly spot how different your local application and the staging one are.

This is what makes CSS regression testing suites so powerful. Imagine you added a 10px padding to an element to fix a design issue. Many non-designers wouldn’t even be able to point out the issue (I certainly wouldn’t !), but if a screenshot has a long purple band showing the difference between screenshots, you’ll notice it immediately.

Let’s have a look at one screenshot:

reference | test | diff

The screenshot above shows a fix for an address field in the mobile viewport which wasn’t aligned with other fields. While a trained eye could spot the error, this isn’t something obvious, especially when the page containing this information is very long and when it only breaks under special conditions like a viewport.

Approve

Finally run backstop approve whenever an error is spotted but your local version is correct. That will replace the reference with the new screenshot.