Testing web applications across multiple browsers on different platforms can be challenging even for smaller applications. With Jenkins and the Sauce OnDemand Plugin , you can wrangle that complexity by defining your Pipeline as Code.

I recently started looking for a way to do browser UI testing for an open-source JavaScript project to which I contribute. The project is targeted primarily at Node.js but we’re committed to maintaining browser-client compatibility as well. That means we should run tests on a matrix of browsers. Sauce Labs has an "open-sauce" program that provides free test instances to open-source projects. I decided to try using the Sauce OnDemand Plugin and Nightwatch.js to run Selenium tests on a sample project first, before trying a full-blown suite of tests.

This sample app generates the SauceOnDemandSessionID for each test, enabling the Jenkins Sauce OnDemand Plugin’s result publisher to link results to details Sauce Labs captured during the run.

This pipeline expects to be run from a Jenkinsfile in SCM. To copy and paste it directly into a Jenkins Pipeline job, replace the checkout scm step with git url:'https://github.com/bitwiseman/JS-Nightwatch.js', branch: 'sauce-pipeline' .

Happily, there’s a whole range of sample projects in " saucelabs-sample-test-frameworks " on GitHub, which show how to integrate Sauce Labs with various test frameworks, including Nightwatch.js. I forked the Nightwatch.js sample to bitwiseman/JS-Nightwatch.js and set to writing my Jenkinsfile . Between the sample and the Sauce Labs instructions, I was able to write a pipeline that ran five tests on one browser via Sauce Connect :

I started off by following Sauce Labs' instructions on " Setting up Sauce Labs with Jenkins " as far as I could. I installed the JUnit and Sauce OnDemand plugins, created an account with Sauce Labs, and added my Sauce Labs credentials to Jenkins . From there I started to get a little lost. I’m new to Selenium and I had trouble understanding how to translate the instructions to my situation. I needed a working example that I could play with.

Adding Platforms

Next I wanted to add a few more platforms to my matrix. This would require changing both the test framework configuration and the pipeline. I’d need to add new named combinations of platform, browser, and browser version (called "environments") to the Nightwatch.js configuration file, and modify the pipeline to run tests in those new environments.

This is a perfect example of the power of pipeline as code. If I were working with a separately configured pipeline, I’d have to make the change to the test framework, then change the pipeline manually. With my pipeline checked in as code, I could change both in one commit, preventing errors resulting from pipeline configurations going out of sync from the rest of the project.

I added three new environments to nightwatch.json :

" test_settings " : { " default " : { /*----8<----8<----8<----*/ }, " chrome " : { /*----8<----8<----8<----*/ }, " firefox " : { " desiredCapabilities " : { " platform " : " linux " , " browserName " : " firefox " , " version " : " latest " } }, " ie " : { " desiredCapabilities " : { " platform " : " Windows 10 " , " browserName " : " internet explorer " , " version " : " latest " } }, " edge " : { " desiredCapabilities " : { " platform " : " Windows 10 " , " browserName " : " MicrosoftEdge " , " version " : " latest " } } }

And I modified my Jenkinsfile to call them:

//----8<----8<----8<----8<----8<----8<---- sauceconnect( options : ' ' , useGeneratedTunnelIdentifier : false , verboseLogging : false ) { def configs = [ (1) ' chrome ' , ' firefox ' , ' ie ' , ' edge ' ].join( ' , ' ) // Run selenium tests using Nightwatch.js sh " ./node_modules/.bin/nightwatch -e ${ configs } --test tests/guineaPig.js " (2) } //----8<----8<----8<----8<----8<----8<----

1 Using an array to improve readability and make it easy to add more platforms later. 2 Changed from single-quoted string to double-quoted to support variable substitution.

Test frameworks have bugs too. Nightwatch.js (v0.9.8) generates incomplete JUnit files, reporting results without enough information in them to distinguish between platforms. I implemented a fix for it and submitted a PR to Nightwatch.js. This blog shows output with that fix applied locally.