When making changes to Sidekiq’s web UI, or any UI, it’s invaluable to know that the UI is being presented as you expect, and that you’re not shipping visual bugs to customers.

Refactoring code, adjusting CSS rules, or altering a completely different part of the app can sometimes introduce unexpected and unwanted visual side effects.

Adding visual regression tests results in confidence when making changes. You can see that what you intended to change has changed (ie. the green button got bigger) and there are no unexpected changes (ie. the other buttons stayed the same).

What is Percy?

Percy is a visual review platform. It makes it easy to add visual regression tests to your application and notifies you when visual changes have occurred. Percy provides a platform to review and approve visual changes, and integrates right into your GitHub pull requests.

Adding Percy to Sidekiq

To add Percy to Sidekiq, we made some small adjustments to it’s testing system. This section explains why.

Sidekiq’s web interface is built as a Rack application. It has a suite of MiniTest tests, and uses Rack::Test to test the web app.

Rack::Test is very fast, and is great if you only care about the html in the response.

However, for visual testing we need more — we need a real browser to be running in tests, handling DOM updates and running JavaScript. We need the page state to be full and complete, but Rack::Test stops short of running a browser or JavaScript. It simply interacts with Rack handlers and HTML directly.

To solve this and get an accurate DOM snapshot, we switched Sidekiq’s UI tests to use Capybara with the PhantomJS headless browser. The Poltergeist driver for Capybara makes working with PhantomJS painless. This allows the page to fully render with javascript updates to the DOM, before taking the snapshots with Percy.

Introducing the headless browser is asking the tests to do more work, so the test suite takes a little longer to complete. Sidekiq’s 279 tests took ~1 second to run with Rack::Test, and they now take ~4 seconds with Capybara and PhantomJS.

To switch to Capybara, in addition to adding the Capybara and Poltergeist gems, we had to make some modifications to the test code.

Switching from Rack::Test to Capybara and Percy

Before updating, the tests using Rack::Test looked like this:

Display busy workers test using Rack::Test

After switching the tests to use Capybara, and adding Percy to take a snapshot, the tests now look like this:

Display busy workers test using Capybara and Percy

We also adjusted Sidekiq’s test helper to initialize and finalize Percy:

Adding initialization and finalization to test_helper.rb

Visual reviews in GitHub pull requests

Now when Sidekiq’s test suite runs in its CI environment, MiniTest, Capybara, and Percy co-operate to take DOM snapshots. The DOM snapshots, supporting images, and CSS resources are automatically uploaded to Percy. Percy immediately gets to work rendering the DOM snaphots in a custom browser farm, capturing screenshots, and running comparisons on them with screenshots from an earlier build.

If Sidekiq has an open pull request associated with the current git commit, Percy updates the pull request status with the results of the visual regression tests, showing whether there are any changes that require review.

GitHub pull request showing visual diffs need review

When there are changes, Sidekiq developers can click the ‘Details’ link to open Percy and review the changes.

Percy.io showing the changes the pull request introduces

Approve the visual changes in Percy.io once they’ve been reviewed

Once the Sidekiq developers are happy Percy’s only showing intended changes, the build can be approved in Percy and the status shown in the pull request will update showing the approval.

The GitHub pull request now shows visual changes have been reviewed and approved

Team Sidekiq can now merge that pull request and ship with confidence that Sidekiq’s UI will display as intended.