As a quality engineer who’s responsible for testing web applications, Selenium WebDriver is arguably the most popular solution for automation. However, using a single WebDriver on a local machine soon becomes insufficient in many ways:

Tests are slow: browser tests are slow compared to unit or integration tests, and when you’re running Selenium tests serially in a single process, running your test suite soon becomes an excuse for a very lengthy nap. Tests are more flaky: when running tests locally, WebDriver actually launches a browser driver, meaning if you’re running your tests with the ChromeDriver, a chrome browser opens up and takes over focus on your mouse and inputs. If you try to do anything else, focus on any other windows or processes outside of the test run, this could cause failures in your test suite if it draws away the input focus.

Automated tests are only useful if they provide confidence that our ever-growing list of features are free from regressions and working as expected. Once the test suite becomes flaky or takes more than a reasonable amount of time to run, tests soon lose their value to the team.

To enable the engineers to run the automation test suite in a more consistent and timely manner, we began looking into different tools to help scale our web automation. This is where Selenium Grid and Docker come in.

Selenium Grid enables us to do distributed testing, allowing us to perform test execution on different machines, with multiple browsers at the same time. We can point our tests against a central hub, which will then distribute the execution to remote nodes. However, setting up our own Selenium Grid means we have to configure browsers across multiple machines, virtual or physical, and making sure each machine is running the Selenium Server correctly. This also means maintaining the Grid and making updates would be a time-consuming, node-by-node process. This is where Docker comes in and saves the day!

Amazon ECS with Docker

Docker provides a central point from where we can configure and update our Selenium Grid hub and nodes, and gives us a way to quickly recover in case a node ends up in a bad state; and Amazon ECS provides a ‘highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of EC2 instances.’ At Lever, we are already using Amazon ECS for cluster and configuration management for most of our deployments, so it became the obvious service for us to use for spinning up our Selenium Grid cluster using Docker containers.

So how did we set this up?

We created an ECS cluster for Selenium Grid, and within the task definition, specified the Docker container images to pull and run across the cluster. We are using the Docker images for Selenium Hub and Node configurations with Chrome found in the docker-selenium repository. We decided to start our Selenium Grid configuration with five Chrome nodes, so we specify a container for the Hub, and 5 Chrome Node containers.

ECS container definitions for Grid

After specifying the images in our ECS task definitions, we start up the Selenium Grid cluster, which should now have one hub and five Chrome nodes. With each node setup to have 10 instances of Chrome available for use!

Selenium Grid web console shows 5 chrome nodes

Using Docker to spin up our automation cluster has been one of the best decisions we’ve made. Docker let’s us take our images and spin them up as a pre-configured Selenium Grid cluster. A huge benefit is its ability to scale.

Another added benefit to running Selenium Grid with Docker is the ability to restart the entire grid. Nodes in Selenium Grid often end up in a bad state if left up and running for a long enough time, when this happens, we can save a lot of the diagnostic overhead by just restarting the grid. With Docker Compose, that takes just a couple of minutes.

Since moving off of running Selenium locally, the Quality Team at lever has managed to improve test stability, as well as reduce the total automation runtime by 93%! If you are thinking about using Selenium Grid for testing your web-based application, Docker can be a great way to improve your productivity.

While we’ve made significant improvements to test stability and runtime, we are still far from achieving our goal of having a seamless test and release process. If that seems like an interesting challenge for you, please check out our jobs page!