Hello my friends, today we are going to look into the most popular solutions for functional web testing. For my review, I listed the most well-known and popular frameworks, sorted them by the number of stars in their GitHub repos and picked top 5. Here are they (the number of stars is specified for the moment when this article was written, and it can differ from the current score).

CasperJS — 6399 stars Site GitHub

stars Site GitHub Protractor — 6351 stars Site GitHub

stars Site GitHub Nightwatch.js — 5975 stars Site GitHub

stars Site GitHub TestCafe — 2106 stars Site GitHub

stars Site GitHub CodeceptJS — 1040 stars Site GitHub

CasperJS is written in Python, i.e. it is not a native Node.js solution. However, I’ve added this framework to my review because it can be installed from npm and so it fits well into the Node.js toolchain.

Further, we will have a detailed look at each of them. We will discuss their main features and try to perform a couple of basic actions with each, which will allow us to understand what each framework is worth. I’ll describe what you need to do to write your first simple test. This review does not encompass all the features — only the first impression from getting started with the framework. As a test scenario, we will use each framework to find its repository on GitHub.

CasperJS

This is the most starred framework in our rating. A special thing about this framework is that it’s tailored to be used with headless browser engines ( PhantomJS or SlimerJS ). On one hand, your tests run much faster compared to real browsers. And you don’t need to install browsers in your continuous integration system. On the other hand, the behavior of headless browsers sometimes differs from real browsers, which makes it impossible to detect browser-specific issues. Nevertheless, these considerations may not affect your choice of framework because you can run tests in PhantomJS with many other frameworks by using plugins or other simple mechanisms. Another advantage of CasperJS is support for CoffeeScript syntax in test code.

Installation

Install PhantomJS npm install phantomjs -g

Install Python 2.6 or newer

Install CasperJS: npm install -g casperjs

Create a file with the test: /casperjs/test.js

Test Code

This test code in plain JavaScript looks a little bit overburdened. It could get better if we used CoffeeScript , but after I rewrote the test in this language, I couldn’t get it to work because the version of PhantomJS should be older than 2. By looking at test code, you can also see that using arrow functions instead of nested plain functions, i.e. ES6 support, could improve code readability.

Run Test

casperjs test test.js

Report

Test Passed

Test Failed

Protractor

A very popular solution positioned as a framework for testing Angular and AngularJs applications. This allows it to attract many Angular users. Protractor has built-in means to work with Angular elements. This simplifies code significantly and allows you to write tests easily and fast. Though, it’s worth noticing that the same can be achieved in other frameworks one way or another. However, if you use Angular heavily, consider using Protractor. In contrast to the previous framework, Protractor tests against real browsers, which is an important advantage. Also, note the automatic waiting feature that, according to the authors, facilitates writing and running tests.

Installation

Install Java Development Kit

Install Protractor: npm install –g protractor

Initialize Web DriverManager: webdriver-manager update

Create a test file ( /protractor/test.js )

) Create a configuration file ( /protractor/conf.js )

Test Code

Run Test

Run Web DriverManager: webdriver-manager start

protractor conf.js

Report

Test Passed

Test Failed

Nightwatch

A very popular framework, the main competitor to Protractor. Its benefits include highly transparent and readable test code. Nevertheless, you still need to use waits in code while Protractor and TestCafe have built-in waiting mechanism. The main drawback is the most complicated environment configuration among all the reviewed frameworks. You can see for yourself how many steps it takes to run a simple test. In general, Nightwatch and Protractor have a lot in common.

Installation

Install Java Development Kit (JDK). The minimum required version is 7

Download Selenium — selenium-server-standalone-{VERSION}.jar

Put Selenium in the same directory with tests

Download the WebDriver for Google Chrome and put it in the same folder ( /nightwatch/chromedriver.exe )

) Install the framework: npm install -g nightwatch

Create a configuration file ( /nightwatch/conf.js ). In this file, you need to specify the path to the Chrome WebDriver.

Create a test file ( /nightwatch/test.js )

Test Code

Run Test

Run Selenium java -jar selenium-server-standalone-3.0.0.jar

nightwatch test.js

Report

Test Passed

Test Failed

TestCafe

A young but already popular framework. Its key feature is that it doesn’t use WebDriver to work with browsers as other solutions do. That is why it requires minimal test environment and it is installed with one command. Besides, this approach allows you to run tests on any physical device without requiring anything but the browser. Other pluses include writing tests in ES6/ES7 , automatic waiting as well as the elegant and informative console reporter. The main drawback of TestCafe is that it emerged not long ago and has a much smaller community than its competitors. Nevertheless, the community grows bigger and TestCafe already has a wide range of plugins that extend its functionality. It is undoubtedly worth your attention.

Installation

npm install testcafe -g

Create a test file ( /testcafe/test.js )

Test Code

Run Test

testcafe chrome test.js

Report

Test Passed

Test Failed

CodeceptJs

This is not a plain framework but a wrapper that provides syntax to write tests that will run in one of the popular test runners. You can choose from the following options

WebDriverIO

Protractor

Selenium WebDriver JS

NightmareJS

or others…

The syntax provides a simple and comprehensible linear way to define the test scenario. You can extend commands in this syntax as you wish, though the set of built-in commands is enough to write basic tests. The support for ES6 syntax makes test code concise and readable. An advantage of such an approach is that you can run your test without substantial changes in any of the popular test runners with no additional effort.

Installation

Install framework npm install -g codeceptjs

Generate codecept.json config with a console command codeceptjs init

config with a console command Create a test by using the command codeceptjs gt

… In addition, you need to install everything required to run the helper of your choice (e.g., Protractor)

Test Code

Run test

codeceptjs run --steps

Report

Test Passed