8 July 2015

web development, Rails, testing Integration testing for Rails APIs part 3: Automating tedious things

In this final part of the series we’ll take a look at the tests we’ve written. Looks like there’s some duplicated code; learn how to eliminate it using automated JSON validation and RSpec’s shared examples.

Validating JSON responses

If you haven’t read the first two parts in this series, I recommend that you start with them: Structuring specs Writing less code

Manually parsing and checking JSON using JSON.parse is a pain: you need to be aware of the structure in each spec.

Wouldn’t it be nice if we could define the structure of an object in a single file and then validate responses using that file? With JSON schema we can!

I first learned about JSON schema from Laila Winner’s post on the Giant Robots blog. I highly recommend reading it!

JSON schema is a format for describing JSON documents structurally. The latest draft is version 4, which I’ll be using in these examples. Even though the spec is still in draft form, v4 is full-featured and good for most use cases.

Describing a user resource with JSON schema

Now, don’t start thinking of XML here: JSON schema is actually easy to read and write unlike XSD.

We’ll use the GET /users/:id endpoint from the previous part as an example. The returned user resource has three attributes and looks like this:

{ "id" : 1 , "email" : [email protected]" , "role" : "admin" } Our user resource with three attributes.

The role attribute is only shown if the current user has the admin role.

Let’s think about the data types for a second. Our resource document’s root is an object with the following properties:

id integer required

email string, looks like an email address required

role string, either “user” or “admin” optional



The schema for the user resource would then look like this:

{ "$schema" : "http://json-schema.org/draft-04/schema#" , "type" : "object" , "required" : [ "id" , "email" ], "additionalProperties" : false , "properties" : { "id" : { "type" : "integer" }, "email" : { "type" : "email" }, "role" : { "enum" : [ "user" , "admin" ] } } } /spec/support/schemas/user.json

We’ll store our schemas in the /spec/support/schemas folder so that we can easily use them for testing.

Schema root

As you can see, the schema is a plain ol’ JSON file. The $schema property lets the validator know which version of JSON schema to use. It’s optional: in case it’s missing, the validator will use the most recent version it supports.

Since we’re in the root element of our schema, the type attribute defines the root element of the described document. The root element of our user resource is an object.

Properties

An object’s properties can be described with the same syntax as all elements. They also have a type property, we’re using the integer and email types.

The enum notation is special: it defines an array of values that the property can have. It’s not limited to strings, you can specify all kinds of values, including null .

Required properties are listed in required . Note that extra properties that haven’t been defined in the schema are allowed by default: to disallow them, set additionalProperties to false .

Advanced goodies

The schema format allows for advanced features such as referencing other schemas and defining your own data types.

I’m not going to describe them here, but to learn how to use them, check out the documentation and examples.

Writing a schema matcher

Using the json-schema gem and a custom RSpec matcher, we can now validate responses against that schema.

Below is the matcher that I use.

Note that you don’t have to copy it into your project. I’ve packaged it into a gem called rspec_json_schema_matcher that you can install.

Improving the matcher This matcher is based on the one from the original post I linked above. It didn’t fully suit my use cases so I modified it. Firstly, the json-schema gem didn’t work as I expected when using strict validation. It detects when there are unexpected attributes, but it also treats all properties as required even if the schema explicitly specifies otherwise. Unfortunately I wanted to whitelist all attribute names, including the optional ones. I disabled strict validation to work around this limitation and changed my schemas accordingly. To whitelist both required and optional attributes: Mark all required attributes in your schema within "required": [] . Add "additionalProperties": false to your object declarations. This disallows any undefined attributes for that object. Secondly, the error messages could be better. This version lists all errors and outputs the JSON that was tested as well. Thirdly, this matcher validates your schema in addition to the tested JSON and tells you when something’s wrong with your syntax.

# Validates JSON responses against a schema RSpec :: Matchers . define :match_schema do | schema | match do | body | schema_directory = " #{ Dir . pwd } /spec/support/schemas" schema_path = " #{ schema_directory } / #{ schema } .json" opts = { validate_schema: true } begin @result = JSON :: Validator . fully_validate_json ( schema_path , body , opts ) rescue JSON :: Schema :: ValidationError => e @result = "Schema ' #{ schema } ' did not pass validation:



- #{ e . message } " end expect ( @result ). to eq [] end failure_message do | body | " #{ messages }



#{ body } " end description do "match the JSON schema ' #{ schema } '" end private def messages if @result . respond_to? ( :join ) @result . join ( "

" ) else @result end end end /spec/support/matchers/match_schema.rb

Assuming that you have a schema called user.json in /spec/support/schemas , you can validate a response body against it like this:

subject { response } context 'when the user exists' do its ( :status ) { should eq 200 } its ( :body ) { should match_schema 'user' } end Using the JSON schema matcher.

What JSON schema doesn’t do

Your API might be returning valid data, but does it make sense? Remember, the schema validates only the structure, not the content.

When testing your API, you should have some tests in place to ensure that the data actually makes sense:

Are you returning the right record?

Does the record change when you update it?

To help test these, it’s useful to have some utilities installed for working with JSON. I’ve found the json_spec gem to be good for most of my use cases.

Shared examples for less duplication

If you output helpful JSON summaries of errors, you’ll soon start to notice that you’re writing the same code again and again when testing for them.

RSpec lets you write shared examples to eliminate duplicated code. I’ve found them to be useful when checking for responses that are pretty much the same everywhere. Errors are prime candidates here.

If you’ve written schemas for your error summaries, validating them becomes suddenly quite easy:

RSpec . shared_examples 'no content' do its ( :status ) { should eq 204 } its ( :body ) { should be_empty } end RSpec . shared_examples 'unauthorized' do its ( :status ) { should eq 401 } its ( :body ) { should match_schema 'errors/unauthorized' } end RSpec . shared_examples 'forbidden' do its ( :status ) { should eq 403 } its ( :body ) { should match_schema 'errors/forbidden' } end RSpec . shared_examples 'not found' do its ( :status ) { should eq 404 } its ( :body ) { should match_schema 'errors/not_found' } end RSpec . shared_examples 'unprocessable entity' do its ( :status ) { should eq 422 } its ( :body ) { should match_schema 'errors/unprocessable_entity' } end /spec/support/shared_examples/http.rb

Usually I have a bunch of shared examples under /spec/support/shared_examples that I require in spec_helper.rb .

Now testing an error response becomes a one-liner in the spec:

subject { response } context 'when the user does not exist' do it_behaves_like 'not found' end Using the shared example to describe a response.

What’s next

This concludes my series on API testing tips. I hope you’ve learned something useful that you can use in your projects!

Let me know in the comments below if you have any questions or feedback.

Even though this series ends here, I’ve got all sorts of goodies lined up. Don’t forget to subscribe to the newsletter below so that you’ll be the first one to know about them!

Related posts