In this series, we’ll explore building a scalable execution framework for Selenium using Docker containers and Kubernetes.

Test execution at hyperscale

Containerization with Docker and the Kubernetes orchestration platform have taken the computing infrastructure world by storm and had a profound effect on the way we architect and build software today. Selenium tests are a candidate workload that can benefit from the massive parallelism offered by a modern hyperscale infrastructure.

In this article, we’ll execute Selenium tests in Docker containers using the framework we built out in the Selenium UI Testing with MSTest V2 series. While the sample code used for testing is C# built on .NET Core, this approach could be used with any Selenium test suite that can make use of the RemoteWebDriver.

Executing tests in Linux with .NET Core

First, let’s clone the framework and sample from the jg-testframework repo on Github. In a Linux environment, make sure you can build and run the tests locally and get a passing result. You’ll want to make sure you have the .NET Core 2.1 SDK installed and running (my testing has been against 2.1.301).

In the test/JG.Demo.CoreTests subfolder, open the default.runsettings file and make sure the WebDriver parameter is set to “chrome”:

<Parameter name=”WebDriver” value=”chrome” />

Now you can run the tests and see if you get passing test results:

cd test/JG.Demo.CoreTests

dotnet build

dotnet test --no-build --settings default.runsettings

Let’s also do a sanity check using the RemoteWebDriver with a manually launched chromedriver. Make sure the following values are set in the default.runsettings:

Then start up an instance of chromedriver in the background and repeat the test execution:

./bin/Debug/netcoreapp2.1/chromedriver --port=4444 >/dev/null 2>&1 &

dotnet test --no-build --settings default.runsettings

Aside: if you’re going to continue along, you may want to terminate the background process if you want to reuse port 4444.

Getting started with Docker

The Selenium project maintains a set of Docker images that we’ll use for executing the tests in Docker.

First up, let’s try a simple test using the standalone Chrome image. First, we’ll need to modify the SeleniumHost parameter to include a path (we’ll go into the why in a moment):

Then we’ll spin up a container bound to port 4444 and run our tests again:

docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome

dotnet test --no-build --settings default.runsettings

You can find additional information on the Selenium images, including a brief discussion of /dev/shm and memory, on the docker-selenium repo.

Aside: if you’re going to continue along, you may want to stop the container if you want to reuse port 4444.

Scaling with Selenium Grid

The Selenium project created Grid to help address the scalability problem. Grid abstracts access to a (potentially) heterogeneous population of driver/browser combinations, centralizing infrastructure and providing consumers with a testing cloud.

A grid comprises hub and node elements. Nodes are abstractions for driver/browser pairs, e.g. Chrome nodes or Firefox nodes, and hubs act as concentrators. Nodes register with a hub at startup. Then a client submits a request to start a session and communicates desired capabilities to the hub, which inspects the request and routes the test session to an appropriate node for execution. Subsequent communication all continues to flow through the hub, simplifying client configuration.

To execute tests via a Grid, we add the /wd/hub path to the Selenium host URL. We can infer, then, that the Docker image we used in the prior example was acting as both hub and node and was in fact a self-contained, single node grid.

Aside: Why the extra path? The hub also provides other endpoints, including a console at /grid/console that is useful for managing the grid.

Standing up Selenium Grid in Docker

Let’s take a look at some other images available on Docker Hub that break out the Selenium Grid hub and node roles, specifically selenium/hub and selenium/node-chrome. We’ll employ Docker Compose to simplify standing up the grid by borrowing from the docker-selenium documentation:

Now by simply executing docker-compose up -d from the directory where the docker-compose.yaml was created, we’ve started up two containers: one hub and one node with Chrome. We can repeat our tests and they should run successfully once again.

Scaling Selenium Grid nodes horizontally

Docker makes scaling trivial (assuming you have sufficient compute resources), so let’s see what happens if we scale up with a second Chrome node and run the tests again.

docker-compose up -d --scale chrome=2

I have added some Thread.Sleep() calls into our demo test methods to make the effects of scaling more obvious, but the returns will naturally become more pronounced as we add additional tests and continue to scale out.

Up next

In our next installment, we’ll tackle build and release with VSTS to demonstrate how we can integrate Selenium Grid in Docker into a typical CI/CD process, then we’ll push the execution into Kubernetes in the cloud.

Download the code from the Github repo.