Results for Visual Regression Test

TLTR: it will capture images as a baseline for the first run, then compare with the images from the next run! That’s it.

The result will be shown in your logs by using Spec Reporter, so you will know if your test is failed or not.

Requirements

There are so many tools to run the Visual Regression Test, and I always prefer the FREE ones. In this post, I will show you how to do your Visual Regression Test with WebdriverIO v4 (it doesn’t work with v5 :( yet), WebdriverCSS, Chromedriver, and GraphicsMagick.

You should read the documents on how to install GraphicsMagick before starting to write your testing codes.

Getting Started

Prepare package.json for your project

// package.json {

"name": "visual-regression-test",

"version": "1.0.0",

"description": "Visual Regression Test",

"main": "index.js",

"scripts": {

"clean": "find . -name '*.diff.png' -delete",

"test": "npm run clean && wdio",

"dev": "wdio --spec"

},

"keywords": [

"webdriverio",

"webdrivercss",

"visual-regression-testing",

"regression-testing"

],

"author": "Dale Nguyen",

"license": "ISC",

"dependencies": {

"webdrivercss": "github:visualregressiontesting/webdrivercss",

"webdriverio": "^4.14.4"

},

"devDependencies": {

"wdio-mocha-framework": "^0.6.4",

"wdio-selenium-standalone-service": "0.0.12",

"wdio-spec-reporter": "^0.1.5"

}

}

After that, you just need to install all the packages.

npm install

Prepare Your WebdriverIO config file

./node_modules/.bin/wdio config

Then answers some questions.

Write Your First Test

// test/home.page.spec.js const assert = require('assert');

const wdio = require('webdriverio');

var getDiffFiles = require('../utils/get_diff_files'); let client;

url: '

// url: '

screenshotRoot: 'screenshots/page1',

failedComparisonsRoot: 'screenshots/page1/diffs'

} const options = {url: ' https://learn.visualregressiontesting.com/' // url: ' https://learn.visualregressiontesting.com/broke.html' screenshotRoot: 'screenshots/page1',failedComparisonsRoot: 'screenshots/page1/diffs' describe('my website should always look the same',function() {

beforeEach('init webdrivercss', () => {

client = wdio.remote({

desiredCapabilities: {

browserName: 'chrome',

chromeOptions: {

args: [

'--headless',

'--disable-gpu',

'window-size=1920,1080'

]

}

}

}).init();



require('webdrivercss').init(client, {

// example options

screenshotRoot: options.screenshotRoot,

failedComparisonsRoot: options.failedComparisonsRoot,

// misMatchTolerance: 0.05,

// screenWidth: [320,480,640,1024]

});

}) it('homepage should look the same', async (done) => {

await client.url(options.url)

.webdrivercss('homepage', [

{

name: 'header',

elem: '.header'

},

{

name: 'benefits',

elem: '.benefits',

screenWidth: [320, 640, 1024]

}

])

.end()

.call(done); // test diff folder

const files = await getDiffFiles(options.failedComparisonsRoot);

// console.table({files})

assert.equal(files.length, 0, `There are changes on the site! Please check in ${options.failedComparisonsRoot} folder.`)

});

})

The test will run in Chrome headless mode. The first time you run, you will use https://learn.visualregressiontesting.com/ site to create base images. The next time, you can use https://learn.visualregressiontesting.com/broke.html, to show the difference between the two sites. If there are any differences, it will be saved under screenshots/**/diffs folder.

The test will check if there are any files under the diffs folder and split out the results.

This is the helper function for waiting for files under the diffs folder.

// utils/get_diff_files.js const fs = require('fs'); const getDiffFiles = (path) => {

return new Promise((resolve, reject) => {

fs.readdir(path, function(err, items) {

var temp = [];

for (var i=0; i<items.length; i++) {

if(items[i].includes('.png')) {

temp.push(items[i]);

}

}

resolve(temp);

});

})

} module.exports = getDiffFiles

That’s it. You just need to run

npm test

in order to show if there are any differences after you update or make changes on your site.

Here is the Git repository in case you want to check the working code.

https://github.com/dalenguyen/visual-regression-testing