Problem: We have an unreliable suite of smoke tests for Springer Link, dependent on data, prone to random failures, and taking significant time to compile and execute.

Spiked solution (1 of 4): Write a new suite of tests using CoffeeScript, Jasmine and an assortment of helpful third party libraries with no dependencies on “golden data”.

describe 'The abstract page for an article', -> it 'has PDF and full text links', (done) -> searchWith 'facet-content-type': 'Article' .then findResultWith fulltext, pdf .then loadPageWith SearchPage.resultTargetUri .then checkForValidLink '#pdf-link' .then checkForValidLink '#full-text-link' .fin done

Jasmine

Jasmine is a well-known JavaScript test framework, already in use in some of our projects and the spec syntax is familiar to our team from our Scala tests. It also provides support for asynchronous testing (through the done() function), a necessity when your tests involve making HTTP requests.

CoffeeScript

CoffeeScript is a language which “compiles” to JavaScript with the intention of smoothing over some of JavaScript’s rougher edges with a terser syntax. The latter makes it an attractive prospect for writing readable tests. Consider the following snippet from the above test translated to JavaScript:

describe('The abstract page for an article', function() { it('has PDF and full text links', function(done) { searchWith({ 'facet-content-type': 'Article' }) .then(findResultWith(fulltext, pdf)) // ... .fin(done()); }); });

Your mileage may vary, but the boilerplate around function definitions and visual noise from dots, brackets and semi-colons (don’t forget the semi-colons!) is something I find distracts from the content of the test.

Q

HTTP requests in node are asynchronous and we will be making many of these, often chained (there are two in the example), leading to ugly nesting. One solution is to use the promises implementation provided by Q, hence all the then s above (they are not actually test script syntax, they just nicely dovetail with it). The test might otherwise have looked something like this:

describe 'The abstract page for an article', -> it 'has PDF and full text links', (done) -> searchWith 'facet-content-type': 'Article', (page) -> result = findResultWith(fulltext, pdf)(page) loadPageWith SearchPage.resultTargetUri(result), (page) -> checkForValidLink('#pdf-link')(page) checkForValidLink('#full-text-link')(page) done()

Not only does this deeply nest with just two requests, some boilerplate is back in the form of parameter lists (it will be clear why in the next section). With Q an asynchronous operation can be wrapped in a promise with failure and success handlers for the result, which in turn result in promises that may be chained:

loadPage = (uri, callback) -> // get that page and invoke the callback when you've got it promiseToLoadRobots = -> deferred = Q.defer() loadPage '/robots.txt' , (page) -> deferred.resolve page deferred.promise promiseToLoadRobots().then (robots) -> // do something awesome with robots

A fin function may be invoked at the very end which is analogous to the finally of a try block in some languages - it will always be executed regardless of the success of the promises in the chain. This is important in asynchronous testing as Jasmine’s done function tells the test runner when an async test is complete (without it the test will time out).

Currying

Even with CoffeeScript there is scope for reduction of boilerplate. In the example the functions which are passed to the then s of the promises are actually functions which take some parameters and return functions which take the result of the promise to check something on. This uses a concept known as currying. Consider without:

.then (page) -> checkForValidLink page, '#pdf-link' page .then (page) -> checkForValidLink page, '#full-text-link' page

By defining the function checkForValidLink such that instead of taking a page and a CSS selector:

checkForValidLink: (page, cssSelector) -> // check for valid link

it takes a CSS selector and returns a function taking a page, which in turn returns the page so that it can be wrapped in a promise by Q for chaining:

checkForValidLink = (cssSelector) -> (page) -> // check for valid link, then return the page

which is exactly what the then of the promise is expecting:

.then checkForValidLink '#pdf-link' .then checkForValidLink '#full-text-link'

I think it’s worth noting here how nice the terse syntax of CoffeeScript is for this technique, which would be considerably uglier and have less clear intent in JavaScript:

function(cssSelector) { return function(page) { // check for valid link, then return the page }; };

Cheerio

For interacting with the DOM of a page the result of the request is loaded using the Cheerio library, a subset of JQuery with the browser API stripped out providing a way of querying HTML familiar to those with web development experience:

$ = cheerio.load response.body // assign to $ for familiarity, not strictly necessary title = $('h1').text()

Requests

Requests are made using the Request library, which provides a nice wrapper around nodejs’s native HTTP support. In combination with Q and Cheerio it looks something like this:

loadPage = (url, headers) -> deferred = q.defer() request url: url headers: headers , (error, response) -> if (error or response.statusCode != 200) deferred.reject error or response.statusLine else deferred.resolve cheerio.load response.body deferred.promise

The first argument to request is a JSON object specifying the request, and here showing again how nice the terseness of CoffeeScript can be.

All data considered equal

One problem with our previous smoke test setup was a reliance on “golden data” - data which the tests assumed to be present in the system. There are two problems with this - data can be subject to change, causing tests to fail for reasons which may not be immediately apparent; and it requires that that data be present in all environments you may wish to smoke test.

To work around this the new test uses the site’s search functionality to locate pages which match certain base criteria which it can then load and check:

searchWith 'facet-content-type': 'Article' .then findResultWith fulltext, pdf .then loadPageWith SearchPage.resultTargetUri // then check some things about the page

searchWith takes a list of query string parameters which are used by the site search, in this case faceting on content type to retrieve articles.

findResultWith then takes a list of criteria to locate a search result, in this case we want to make sure that it is for an article that has both PDF and HTML representations available.

loadPageWith takes a function which is used to extract a URI from a provided page or page fragment, in this case a search result entry.

Configuration

Given that we want to be able to run these smoke tests in different environments (and now are able to with the data dependency issue largely resolved) we need to be able to provide environment-specific configuration. The library which stood out here was flatiron’s nconf:

environment = nconf.env().get 'ENV' nconf.file file: "conf/config.#{environment}.json"

The first line in this snippet gets the environment name from an environment variable that we have defined on our servers and dev machines (or that can be provided in the command line), and uses it to select the file to load.

Configuration in nconf is hierarchical, a nice contrast to the repetitious dot syntax familiar from Java properties:

{ "springer": { "host": "localhost:8080" } }

The host key is then accessible like so:

nconf.get 'springer:host'

Additionally, nconf is wrapped in our own config wrapper and nconf.get assigned to an exported config value. This makes it both less verbose and the code less explicitly dependent on a third party:

// config.coffee: // config initialisation goes here exports.config = nconf.get // other file: {config} = require './config' host = config 'springer:host'

Conclusion

Using a few familiar libraries or libraries which provide familiar interfaces, a boiler-plate reducing language and a useful FP paradigm we succeeded in producing tests which are clean, readable and fast-running (the whole suite presently runs in ~3.5s on a MacBook pro, as opposed to > 2 minutes for the Scala tests).

By Jim Kinsey