Automation Testing is a specialized job that is done with QE engineers who are familiar with the frameworks involved. In my team this year we have an ambitious goal of having developers write these automation tests as part of the development process thereby making test coverage (both unit and automation) a metric of the code quality.

As developer on one of the FrontEnd teams in Walmart and a noob to the automation setup and development process, I am sharing my experience on how I went about setting up the automation code base and executing it. My team works on Walmart’s online Returns experience , our front end responsive web framework uses Electrode a framework open-sourced by Walmart for large scale front end development and the Mobile App experience using React Native. This post is centric around the frameworks that WalmartLabs uses and is primarily intended for a developer getting into automation test development.

First the Frameworks

Automation has been around for a while in Walmart Labs and we have a great team dedicated to working on the tools (TestArmada). So I did not really have to go far to figure out which tools I needed to use. For those who are absolutely new to the automation world, it’s useful to know that Selenium is the most ubiquitous test automation framework out there (started out in ThoughtWorks and later enhanced by Google) and many of the tools in various languages are developed on top Selenium’s web driver apis.

Selenium — its a Java based server implements the WebDriver protocol (which is a Rest based protocol that receives commands to execute on a browser). The Server spawns out separate processes which launches a browser that has remote debugging enabled over which it issues commands to the browser.

Magellan — this is the test runner which runs the tests in parallel . Its similar to KarmaJS in that running tests is a separate concern than actually defining the test. Magellan works by finding files or tags and spawning up new worker processes that work on these tests. Each file is a new test case that will be executed within a worker. The worker environment is a sandbox for each test. It can work on these in parallel or in serial mode depending on the settings provided.

NightwatchJS — is the BDD framework that abstracts out Seleniums webdriver apis and provides a framework for defining the test. It supports a design pattern called a page object model through which an HTML page or Mobile page can be abstracted out as page-fragments with commands , sections elements and pages .

Appium — is the Selenium equivalent for mobile app testing. Nightwatch uses Appium to interact with the Mobile device. Appium implements Web Driver specifications making it compatible with any client that uses Web driver api.

Shifu — is a mocking framework developed as plugin with HapiJS. It essentially provides an easy way of detecting and configuring mocks for particular URI route. For local or dev end to end testing, you would want to start by mocking all of the webapp’s dependent downstream services. So when you start your automation setup, you also start a mock server which will handle all your downstream api calls and return a mock response. The cool thing about Shifu is that it has a concept of a session, which allows sending different responses for a route per session. You do this by invoking a variant for a session. Variants are different responses for the same route, tagged differently. The shifu-magellan-nightwatch npm module has out of the box integration with Shifu so that this can be easily plugged into your automation setup.

The Page Object Model

Elements — these are logical placeholders for the elements you will query on your page using CSS or XPath query selectors

elements

Commands — these are abstractions that will act on your elements

commands

Pages — a page is composition of the elements and commands all of which are available to the page object.

page

The Actors in our Setup

Entities in an automation

Note on the actors

The Mocked services are optional — you can just as well do automation testing on a live or pre-prod setup Each Automation tests runs in its own sandbox which starts a Webapp in that process space. The entire setup can be run in one local machine or can be distributed over the network. You could have tests executing from your local machine talking to remote selenium servers that load up webapps from yet another domain.

Setting it Up

So essentially if you are automating a full front end application, you need to be able to load the page, click elements, set values to inputs, click Buttons, assert on element values etc. From the onset, its a good idea to provide a data-automation-id or some such identifier to your elements so as to not have to depend on CSS and element ids. Also where there are dynamically generated elements (such as a grid or list) , selecting elements using CSS parent child hierarchy is next to impossible if all the elements on the grid have the same class attributes. Under the hood, Selenium uses document.querySelectorAll for getting the elements and asserting on these. If you do not have unique id or css selector, this returns a list if there is more than one match for the query. In which case, clicking on the element will not work and Nightwatch Extra will give a warning that looks like

21:39:35 [WARN] [Nightwatch Extra] getEl saw selector .new-order-item-select-container but result length was 26, with 26 of those :visible