By integrating of a set of SaaS services we set up the CI environment.

Setting up a continuous integration environment involves a few steps

that differ depending on the used technology but generally the following

pattern can be recognized.

1) A build is triggered automatically when new code is committed to repository

2) During the build code is compiled and unit tests are run

3) If everything goes well a new distributable is made

4) New software version is automatically deployed to a staging environment

5) Automated regression UI tests are executed against the new software version

Finally there’s the actual release phase that you do as often as it fits

for your project.

Majority of software projects never have UI regression testing automated.

Software is often released fingers crossed – it is barely tried out at all

prior releasing. This leads to trivially observable bugs in production

and angry customers.

We have implemented all CI steps by integrating a set of SaaS services.

Our goal was to build and test a new software version when changes are made,

and report results back to developers as soon as possible. The feedback

time is critical. With fast feedback, regression is easier to understand

and fix.

When we make code changes, we push them to Bitbucket. A commit hook then

triggers our build process that is executed in Codeship. Codeship compiles

our code and runs unit tests. After success, Codeship deploys new

version to our staging server and sends signal to Usetrace to start

automated regression testing against the user interface.

Every step is reported in team chat. We’re using HipChat. Regression

test results are reported in real time. Each failing test is reported

immediately, and once the test suite has completed, a summary

is given. This allows us to notice and fix problems even before

the whole CI cycle has completed.

Our CI cycle currently takes around six minutes. Most of the time is spent

in the compilation phase. Regression testing stands currently at two minutes:

we have automated six most business critical test cases and are running them

on our supported browsers: latest versions of Chrome and Firefox. So we have

total of 12 automated UI tests run in under 120 seconds. On average, one

test runs in ten seconds.

Finally, before we do a release, we check the current software performance from Usetrace dashboard. Up-to-date facts about the business critical features’ performance make the decision of whether or not to release less stressful.

Here’s the whole picture as a diagram: four SaaS developer tools make the whole

CI system.

Disclaimer: Usetrace is a SaaS web testing solution providing a cross-browser platform to create, maintain and run UI tests and have automated testing integrated in the CI process.

Follow Usetrace in Twitter!