REST API Test Automation in Java with Open Source Tools

Vladimir Belorusets, PhD

Introduction

Nowadays, REST API testing has gained much deserved popularity due to the simplicity of verifying backend functionality. Availability of many open source tools and libraries supporting REST API development and testing in Java also added incentives to automate these tests before the GUI development is complete.

REST API regression test automation includes generating code for HTTP calls and comparing the server's actual response with the expected one. In my article "A Unified Framework for All Automation Needs - Part 2" [1], I described how to use the open source Spring Framework to generate REST calls and map JSON and XML responses to Java classes.

In this article, I will describe the automation process in detail. The intended audience is test automation engineers who need practical guidance in navigating through the large number of various tools available on the market. The article does not pretend to be a comprehensive overview of all the tools, but it describes a subset of the tools sufficient for successful and easy regression test automation.

The automation process consists of the following steps:

1. Compose a REST call manually with the help of a REST client tool.

In automating any test case, the first step is to perform that test case manually to see that the application produces the outcomes as expected. Only after that it makes sense to start writing the automation code. REST clients provide a fast and simple way to execute the REST calls manually and observe the responses in a convenient format.

2. Model the endpoint response into an object class using any object-oriented programming language of your choice.

That class will represent the JSON/XML tree structure and provide an easy access to the response values. In the next step, the object of that class will be used to store the response data in a file for future comparisons.

An alternative is to process the REST response as a string without mapping it to an object. Then, you need to apply a parser to that string to extract individual field values. This will make the response validation test much more complex and cumbersome.

3. Generate the same REST call programmatically and serialize the response object into a file as the expected result.

The serialized object represents the reference point. The regression test validates the condition that even after modifying the web service code, the REST call previously implemented continues to return the same data as in the stored object.

4. Create a script that issues the REST API call and compares the response object with the expected object after deserialization.

The comparison between actual and expected results is done via xUnit assertions.

Let's go through the set of tools that supports each step of the process outlined above.

REST Clients

There are plenty of the REST client tools for any taste. They allow generating REST calls without using a programming language and observe the response in JSON or XML format.

We can classify these tools in the following categories:

Desktop REST client applications

Web browser based REST clients

Command line REST clients

Most of the REST clients allow saving REST calls for future reuse either as files or favorite links. They also support common HTTP request methods such as GET, POST, PUT, HEAD, DELETE, OPTIONS, etc.

Desktop Rest Clients

OS Tool Name Web Site Windows I'm Only Resting http://www.swensensoftware.com/im-only-resting Mac OS CocoaRestClient http://mmattozzi.github.io/cocoa-rest-client/ Any OS WizTools.org RESTClient https://github.com/wiztools/rest-client

Table 1. Desktop REST Clients.

The tools differ mostly in their GUI layout. As an example, 'I'm Only Resting' is presented here.

Figure 1. 'I'm Only Resting' HTTP Client

There are also REST client plugins for popular IDEs that allow you to speed up code development by testing REST API in a separate panel without leaving your IDE.

IDE Tool Name Web Site Eclipse RESTClient Tool https://marketplace.eclipse.org/content/rest-client IntelliJ IDEA (Community Edition) RESTClient https://code.google.com/p/restclient-idea-plugin/

Table 2. IDEs REST Plugins

The IntelliJ IDEA plugin above imitates the desktop WizTools.org RESTClient.

Browser REST Clients

Popular browsers have numerous open source REST clients developed as their plug-ins. Some of these are listed below.

Browser Tool Name Web Site Firefox RESTClient https://addons.mozilla.org/en-US/firefox/addon/restclient/ Firefox REST Easy https://addons.mozilla.org/en-us/firefox/addon/rest-easy/ Chrome Advanced REST Client http://chromerestclient.appspot.com/ Chrome Postman https://www.getpostman.com/ Chrome DHC - REST/HTTP API Client http://restlet.com/products/dhc/

Table 3. Browser REST Clients

Firefox clients are presented as buttons on the browser's toolbar. All Google Chrome clients are accessible via the Chrome App Launcher.

As an example, the Chrome DHC client is presented below.

Figure 2. Chrome DHC Client.

Command Line REST Clients

Tool Name Web Site CURL http://curl.haxx.se/ HTTPie https://github.com/jkbrzt/httpie

Table 4. CLI REST Clients

The most widely used CLI client for REST calls is cURL which stands for 'Client for URLs'. It allows getting or sending files using URL syntax. In the example below cURL is used for HTTP POST in Windows.

Figure 3. An Example of Using cURL for HTTP POST.

HTTPie client is a user-friendly cURL replacement featuring intuitive commands, JSON support, and syntax highlighting. The same REST call we used with cURL is presented with HTTPie syntax on Figure 4. By default, the Content-Type header set to application/json, and data fields in the message body are presented in JSON format. When a request contains data fields, HTTPie sets the request method to POST.

Figure 4. An Example of Using HTTPie for POST.

Mapping JSON Response

Now it is time to develop the automated tests. Since Java is today the most popular programming language in the world [2], the implementation details are given in Java.

The first thing to consider is how to validate the response data with assertions. There are many Java libraries that generate HTTP requests and get the responses as strings: java.net package in Java SE and Apache HttpComponents (http://hc.apache.org/), just to name a few. However, string comparison is not a solution since some data may contain dynamic values, such as ids and dates. Each time you produce the REST calls, these data in the response body will be different and your assertions will fail.

One option is to parse the JSON response string and extract the interesting individual name/value pairs. A popular open source library that provides this capability is JSON.simple (https://code.google.com/p/json-simple/).

In this approach, the test case will consist of multiple assert methods. This is not a good design. It makes the test code longer and harder to perceive. If developers later add new fields, the test code must be updated, which should be avoided since the test logic stays the same and only the data have changed. The pattern to use is a single assertion on an expected object instead of one assertion per each object field [3]. It implies mapping the response to a Java class called POJO (Plain Old Java Object), with the class members corresponding to the JSON fields and using this object in assertion, assertEquals(expectedResponseObject, actualObject), given that assertion is customized to compare objects.

To process JSON response as a POJO, I recommend using the open source Spring Framework (http://projects.spring.io/spring-framework/). It comes with the built-in converters that map the web service responses into the relevant Java classes.

MappingJackson2HttpMessageConverter class converts JSON into a POJOs using the Jackson library (http://wiki.fasterxml.com/JacksonDownload), and GsonHttpMessageConverter class does the same with the gson library (https://github.com/google/gson). Both converters are very similar. For the rest of this article, I imply using Jackson. It provides simple data binding by converting JSON to Java data types, such as maps, lists, strings, numbers, booleans and nulls.

After reviewing the JSON response by using one of the REST clients described in the previous section, you need to create a POJO object that Jackson can convert the data to. Jackson requires having getters and setters for each response field.

The Spring Framework contains org.springframework.web.client.RestTemplate class that makes submitting REST calls and converting responses to Jackson POJOs very easy. For each HTTP request method, Spring offers corresponding RestTemplate method that includes Java class for expected JSON response as a parameter: getForObject(), postForObject(), etc.

You can create a POJO manually, which can be time consuming for the complex JSON responses. Open source tools that simplify this procedure are presented below. They generate POJOs online after you paste the JSON string. An example of the jsonschema2pojo web page is presented on Figure 6.

Tool Name Web Site Jsonschema2pojo http://www.jsonschema2pojo.org/ JSON to POJO http://boldijarpaul.github.io/jsontopojo/ Convert XML or JSON to Java Pojo Classes http://pojo.sodhanalibrary.com/ Json to Java http://jsontojava.sinaapp.com/

Table 5. JSON to POJOs Online Converters

The Jackson project also provides classes that can programmatically convert the JSON string into a POJO (https://github.com/FasterXML/jackson-docs/wiki/JacksonSampleSimplePojoMapper).

Figure 6. jsonschema2pojo Online POJO Generator.

Serializing Response Objects

The next step is to perform an end-point call programmatically and serialize the JSON response as the POJOs in files for future references and comparison. However, as I mentioned before, some of the response fields contain data varying from call to call.

To identify those fields, you can issue the same call a few times using one the REST clients. Then, you can compare JSON responses for differences with the help of the online tools listed below.

Tool Name Web Site JsonDiffPatch http://benjamine.github.io/jsondiffpatch/demo/index.html JSON Diff http://tlrobinson.net/projects/javascript-fun/jsondiff/ JSON Diff View (Firefox) https://addons.mozilla.org/en-us/firefox/addon/json-diff-view/ JSON Diff http://jsondiff.com/

Table 6. Tools for Finding JSON Differences

There is no reason to store those data because the tests will always fail in assertions. A solution is to use the transient keyword for the corresponding POJO variables. The values of the transient fields will not be serialized. But we cannot ignore those fields completely since some of them are mandatory, and an important part of the tests is to guarantee that they are not nulls. Field validation can be performed on the backend as well as on the frontend. To address that issue on the client side, we can use the open source Hibernate Validator library (http://hibernate.org/validator/). It provides annotation-based constraints on class variables and transparent integration with the Spring Framework. The annotations of interest are: @NotNull, @Size(min=.., max=...), @Min(...), and @Max(...). You need to decorate mandatory POJO class variables with these annotations and test the response object using javax.validation.Validator.

Response Verification

The last step in the automated test includes comparison of the deserialized expected object from the file with the actual JSON response mapped to the POJOs. In JUnit, verification is accomplished by using assertions, such as assertEquals(). However, if the fields in the response class contain other objects, out-of-the-box assertions fail even when the objects being compared contain the same values. The reason is that comparison is conducted by using '==' operator applied to the reference variables without recursively checking all object fields. I recommend two libraries to compare object hierarchy field by field in one statement.

Library Name Web Site Unitils http://unitils.org/ Shazamcrest https://github.com/shazam/shazamcrest

Table 7. Recursive Assertion Libraries

Unitils assertReflectionEquals(expectedObject, actualObject) loops over all fields in both objects and compares their values using reflection. If a field value itself is an object, it will be recursively compared field by field. Reflection assert automatically excludes transient variables from comparison. You can also ignore non-transient values by specifying ReflectionComparatorMode.IGNORE_DEFAULTS mode in the assertReflectionEquals(). This mode excludes all fields that designated as nulls in the expected object.

Shazamcrest is a library that extends the functionality of Hamcrest assertions with a special matcher sameBeanAs(): assertThat(actualObject, sameBeanAs(expectedObject). The comparison is done field by field including fields in object variables. It also provides capability of ignoring fields from matching by specifying which fields to ignore as follows: sameBeanAs(expectedObject).ignore("field1").ignore("field2").

Code Samples

Let's assume as a result of GET request to http://www.myREST.com/person, JSON response will be mapped to the class Person using the Jackson converter. The explanation comments are provided within the code.

public class Person { // using Hibernate Validator annotation @NotNull private transient String id; private String name; private Address address; public Person(String id, String name, Address address) { this.id = id; this.name = name; this.address = address; } // getters and setters must be added } public class Address { private String street; private String city; private int zip; public Address(String street, String city, int zip) { this.street = street; this.city = city; this.zip = zip; } // getters and setters must be added } @Test public void testREST() { // use Spring Framework for generating GET request RestTemplate restTemplate = new RestTemplate(); Person actualPerson = restTemplate.getForObject( "http://www.myREST.com/person", Person.class); // let's assume for illustration that we received the // following response object actualPerson = new Person(null, "John Doe", new Address("1 Main St", "San Mateo", 94404)); // then validation using Hibernate Validator fails assertTrue("Some required fields are nulls", RestUtils.validateObject(actualPerson)); /* the following error message will be displayed: java.lang.AssertionError: Some required fields are nulls <2 internal calls> at TestPerson.testREST(TestPerson.java:15) <23 internal calls> id may not be null */ // create your methods serialize to and deserialize from the file to extract the // expected POJO Person expectedPerson = Utilities.deserialize("person.ser"); // let's assume that after de-serialization we get the following object expectedPerson = new Person("1234", "John Doe", new Address("1 Main St", "San Mateo", 94404)); // both assertions pass since they ignore transient id assertReflectionEquals(expectedPerson, actualPerson); assertThat(actualPerson, sameBeanAs(expectedPerson)); // if the expected object is as follows expectedPerson = new Person("1234", "John Doe", new Address("1 Main St", "San Mateo", 94107)); assertReflectionEquals(expectedPerson, actualPerson); /* the following error message will be displayed applying Unitils assertion junit.framework.AssertionFailedError: Expected: Person> Actual: Person> --- Found following differences --- address.zip: expected: 94107, actual: 94404 */ assertThat(actualPerson, sameBeanAs(expectedPerson)); /* the following message will be displayed applying Shazamcrest java.lang.AssertionError: Expected: { "name": "John Doe", "address": { "street": "1 Main St", "city": "San Mateo", "zip": 94107 } } but: address.zip Expected: 94107 got: 94404 */ } public class RestUtils { public static boolean validateObject(Type object) { Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); Set > violations = validator.validate(object); if (violations.size() > 0) { for (ConstraintViolation cv : violations) { System.out.println(cv.getPropertyPath() + " " + cv.getMessage()); } return false; } return true; } }

Summary

The article describes a set of open source tools that give testers of all levels ability to conduct REST API verification. Manual testers can use REST clients that provide convenient GUI for hitting the end-points while test automation engineers with Java experience can develop comprehensive tests fast. Although the choice of tools is subjective, I hope you enjoyed using them as I did.

References

1. Belorusets, Vladimir. A Unified Framework for All Automation Needs - Part 2. Testing Experience, Issue No 27, pp. 9-13, 2014.

2. TIOBE Index for October 2015 (http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)

PYPL PopularitY of Programming Language (http://pypl.github.io/PYPL.html)

3. Meszaros, Gerard. xUnit Test Patterns. Addison-Wesley. 2007.

More Software Testing and Java Resources

Software Testing Magazine

Software Testing Tutorials and Videos

Java Tutorials and Videos

Click here to view the complete list of archived articles

This article was originally published in the Winter 2015 issue of Methods & Tools