Over the last few years I’ve had the opportunity to work on several Service

Oriented Architecture (SOA) applications. I learned that writing integration

tests for such applications is difficult, but important. The challenge lies in the fact that most SOA applications use

testing approaches that are well suited for monolithic applications, but these

approaches are not always suited for testing SOA applications. It is important

because without integration tests it is far too easy for subtle bugs to creep

into your code base.

What is SOA?

Service Oriented Architecture might be best understood by first

understanding monolithic applications. Your typical long-lived Rails

application tends to be a monolithic application. According to Martin Fowler,

monolith applications:

…are often built in three main parts: a client-side user interface (consisting

of HTML pages and javascript running in a browser on the user’s machine), a

database (consisting of many tables inserted into a common, and usually

relational, database management system), and a server-side application. The

server-side application will handle HTTP requests, execute domain logic,

retrieve and update data from the database, and select and populate HTML views

to be sent to the browser. This server-side application is a monolith – a

single logical executable. Any changes to the system involve building and

deploying a new version of the server-side application.

While there might not be a universally accepted definition of SOA, applications

that adhere to this approach exhibit some common characteristics. SOA is a style

of architecting applications where the underlying structure supports

communication between a collection of loosely coupled services over well-defined

interfaces.

An example might help clarify:

Figure 1.



In this example, there are two applications that comprise the SOA, the home

application and the client application. home has a database and exposes a

RESTful interface. The

client does not have a database and communicates with home via HTTP . For

example, if the client wants a list of all the users, it has to issue a HTTP

request to the /users resource of the home application, home queries its

database for all the users, and home responds to the client with a JSON

respresentation of all the users.

Testing Monolithic Applications

The Ruby community has developed a robust suite of testing tools. My go-to tools

when writing tests for a monolithic Ruby on Rails application are rspec-

rails , factory_girl_rails , and database_cleaner . rspec-rails is the testing framework, factory_girl_rails creates test data and database_cleaner cleans up test data. These tools provide a

straightforward and simple syntax that most Rails developers are accustomed to

using. A simple test that employs these tools looks like:

In spec/models/user_spec.rb

Testing Approaches for SOA

The SOA projects in which I’ve been involved had two things in common:

They lacked a robust and integrated test suite.

The developers on these

projects understood the importance of testing, but were unable to effectively

utilize existing tooling to do the job.

Why did these projects have these problems in common? They all required state to

exist in one of the other services in order to be tested. In our client/home

example, home’s database must be seeded with records in order for the

client’s tests to run. factory_girl creates test data in applications that

contain a database and database_cleaner deletes test data in applications that

contain a database. Since the client does not have a database, factory_girl

and database_cleaner are not useful to the client and are not always suited

for applications that comprise SOA. In lieu of being able to use factory_girl

and database_cleaner , the following are several approaches I’ve seen employed:

Stub HTTP requests from client to home.

requests from client to home. Seed home’s test database

with all the required test data needed for the client’s test suite to run

prior to running the client’s test suite.

with all the required test data needed for the client’s test suite to run prior to running the client’s test suite. Create test data in home from

the client by issuing HTTP requests to home’s public API.

Applying a real-world scenario to each of these approaches helps determine the

efficacy of each strategy and emphasizes why integrating home with the

client’s test suite is important. The scenario is as follows:

Given the architecture described in Figure 1, Developer A is working on the

client and Developer B is working on home. home provides a public

resource, /users , which returns a list of all the users. Developer A creates a

feature and a passing test in the client that displays the last name of all

the users on the client’s index page. Then, Developer B renames the resource

in home from /users to /people and does not tell Developer A. Then,

Developer A creates another feature and a passing test in the client that

displays the number of users on the index page. These two features are

deployed to production.

Stubbing HTTP Requests

The most common strategy I’ve seen employed is to test each individual

application in isolation of every other application. With this approach,

home’s server is not running when the client’s test suite is running. As a

result, the client application can not issue HTTP requests to home and all

HTTP requests from the client to home are stubbed. The biggest benefit of

this approach is that test setup and coordination between the client and home is

reduced because home’s server does not have to be running while the client’s

test suite is running.

However, the biggest disadvantage is that stubbing the HTTP requests from the

client to home instills a false sense of security in the client’s test

suite. If all HTTP requests issued to /users in home from the client

were stubbed to always return a list of all users, then renaming the resource in

home from /users to /people would not result in any test failures when the

client’s test suite is run. Because there were no test failures when the

client’s test was run, the developers would feel confident that the code works

as intended and this code would be deployed to production. Yikes, they introduced a

crisis!

Proponents of stubbing HTTP requests might reply that if the client just

updated the stubbed HTTP requests or re-recorded the cassettes (if using

VCR) then running the client’s test suite would

result in failures. The flaw with that approach is that

the developer has to remember to issue updates—and humans have notoriously bad

memories.

It is worth noting that there is nothing inherently wrong with stubs. They are

often effectively used to isolate units under test. However, integration testing

SOAs (client issuing HTTP requests to home in our example) gives us more

confidence in the system than stubbing would otherwise provide. You might then

ask, “When should I stub HTTP requests?” I suggest that you stub HTTP requests

when you cannot manage the test server, such as with a third-party service. If you can

manage the test server, then I suggest that you should not stub HTTP requests.

Seeding Home’s Database

With this approach, prior to running the client’s test suite, home’s test

database is seeded with all the test data required for the client’s test suite

to pass. The biggest benefit of this approach is that home’s server can be

running while the client’s test suite is running, which allows the client to

issue HTTP requests to home (as opposed to stubbing HTTP responses to

home). As a result, renaming the resource /users to /people would result

in test failures when the client’s test suite is run. This bug would be caught

prior to being deployed to production. Crisis averted!

The disadvantage of this approach is that it quickly becomes an unviable long-

term solution by virtue of its implementation. There are many unforeseeable

problems with this approach, but the main concern is that home’s test database

bleeds state between tests. For example, if 20 tests in the client each created a

user, then there would be 20

users in home’s test database when the client’s test suite finished running. A situation like this will likely give rise to

bugs that are difficult to track down.

Create Data with Home’s Public API

It is certainly possible to create test data in home from the client using

home’s public API. Like the seeding of home’s database approach, the biggest

benefit of this approach is that home’s server can be running while the

client’s test suite is running, which allows the client to issue HTTP

requests to home (as opposed to stubbing HTTP responses to home). As a

result, renaming the resource /users to /people would result in test

failures when the client’s test suite is run and this bug would be caught

prior to being deployed to production. Again, crisis averted!

However, this approach can make creating test data difficult in the client.

This is especially true when the data that needs to be created has complex data

constraints or has complicated associations. In some cases, the test data that

needs to be created might not be able available through the public API. I would

also argue that creating test data via home’s public api unnecessarily tests

home’s resources when the test should instead focus on testing a piece of

application code (i.e., a user signing in). Like seeding home’s test database

strategy, this approach bleeds state between tests.

The Fourth Option

Wouldn’t it be great if there was a fourth option? What if something existed in the

client, like factory_girl and database_cleaner , that enabled the client

to easily create and delete test data in home? remote_factory_girl in

conjunction with remote_factory_girl_home_rails allows the client to create

test data in home’s database. remote_database_cleaner in conjunction with

remote_database_cleaner_home_rails enables the client to delete test data in

home’s test database.

A simple test that employs these tools looks like:

In spec/models/user_spec.rb in client

Like the seeding of home’s database and using home’s public API

strategies, the biggest benefit of this approach is that home’s server can be

running while the client’s test suite is running. The client could issue

HTTP requests to /users (as opposed to stubbing HTTP responses to home)

to retrieve a list of users. Renaming of the resource /users to /people

would result in test failures when the client’s test suite is run. The bug

would be caught prior to being deployed to production. Yay, crisis averted!

However, what sets this apart from the other approaches is that:

remote_factory_girl and remote_database_cleaner have a similar interface

and workflow to their counterparts, factory_girl and datbase_cleaner , and

because these tools are popular among Rails developers, the overhead associated

with introducing new developers to this approach is arguably lower.

and have a similar interface and workflow to their counterparts, and , and because these tools are popular among Rails developers, the overhead associated with introducing new developers to this approach is arguably lower. remote_factory_girl leverages factory_girl , which can be beneficial when

creating data with complex data constraints or associations.

leverages , which can be beneficial when creating data with complex data constraints or associations. remote_database_cleaner leverages database_cleaner so test data does not

bleed between tests.

With this approach, SOA applications enjoy (almost) all the benefits of

factory_girl and database_cleaner that monolithic applications do!

Create Test Data in Home from Client

remote_factory_girl lives in the client and is configured to issue HTTP

requests to home at the resource /remote_factory_girl/home on a specified

port ( 4000 in this case). When RemoteFactoryGirl.create is invoked, a HTTP

request is sent to home which includes the factory_girl factory name and

attributes as parameters. remote_factory_girl is configured in the client

like:

In Gemfile in client

In spec/spec_helper.rb in client

remote_factory_girl_home_rails lives in home and exposes the resource

/remote_facotry_girl/home . It creates test data with factory_girl based on

the input provided by remote_factory_girl and responds with a JSON

representation of the record just created. The factory names receieved as

parameters from the client must have a cooresponding factory defined in

home. remote_factory_girl_home_rails is configured in home like:

In Gemfile in home

In config/environments/test.rb in home

In config/routes.rb in home

Note: remote_factory_girl_home_rails can be configured to skip specified

controller actions in home when the client’s test suite is running.

Delete Test Data in Home from Client

remote_database_cleaner lives in the client and issues HTTP requests to

home at the resource /remote_database_cleaner/home/clean on a specified port

( 4000 in this case). When RemoteDatabaseCleaner.clean is invoked, an HTTP

request is sent to the resource /remote_database_cleaner/home/clean in home

on the specified port. remote_database_cleaner is configured in the client

like:

In Gemfile in client

In spec/spec_helper.rb in client

remote_database_cleaner_home_rails lives in home and exposes the resource

/remote_database_cleaner/home/clean . When this resource is hit, it cleans

home’s database. remote_database_cleaner_home_rails is configured in home

like:

In Gemfile in home

In config/environments/test.rb in home

In config/routes.rb in home

Note: remote_factory_girl_home_rails can be configured to skip specified

controller actions in home when the client’s test suite is

running.

Running Client’s Test Suite

remote_factory_girl and remote_database_cleaner issue HTTP requests to

home on a specified port ( 4000 in our example), so home’s server needs to

be running on that port. We also want to ensure that data is created and deleted

in home’s test database (not development), so home’s server needs to be

running in the test environment (assuming remote_factory_girl_home_rails and

remote_database_cleaner_home_rails were included in the test group in the

Gemfile and enabled in config/environments/test.rb ). After home’s server

is running, the client’s test suite can be run. Start home’s server like:

$ rails server --environment = test --pid = /Path/to/home_application/tmp/pids/home_app-test.pid --port = 4000

Note: The pid option is not required. However, providing a pid allows

home’s development and test server to be running simultaneously so you do not

have to shutdown home’s development server to run the client’s test suite.

The Takeaway

Integration testing SOA gives us more confidence in our test

suites. Without integration testing, it is far too easy for bugs to creep into

our code base. Although integration testing might be more difficult in the

short term, I believe it pays dividends in the long run.

Working Demo