RESTful API Patterns

There are so many ways to write an API that is REST architectural style compliant, I’ve grouped some of the solutions here.

The REST architectural style has well-defined constraints that help a developer to write scalable Web Service Interface, but APIs are not so easy to write. This is why I’ve grouped some tips and solutions here.

Resource and basic operations Listing and Pagination Many to Many relations Fields filtering Long-running operations Concurrency handling Versioning Combine resources into composites Multi-language fields

1. Resource and basic operations

An API is all about resource and interaction. A resource can have all or some of the basic operations that allow a client to interact with it. A client can create, replace, update, delete and retrieve a resource. All these operations have a well-defined mapping to HTTP verbs.

Let’s see an example. Given the resource user, a client can interact with it through the following endpoints:

// create the resource

POST /users // retrieve the details of the resource identified with the given id

GET /users/:id // partially update the content identified with the given id

PATCH /users/:id // replace the content identified with the given id

PUT /users/:id // delete the content identified with the given id

DELETE /users/:id

To create a new user a client submits the following request:

POST /users HTTP/1.1

Host: example.com

Content-Type: application/json {

"name": "bob",

"age": 76

}

To retrieve the resource identified with the value 8646291 a client submits the following request:

GET /users/8646291 HTTP/1.1

Host: example.com

To partially update the resource identified with the value 8646291 a client submits the following request:

PATCH /users/8646291 HTTP/1.1

Host: example.com

Content-Type: application/json {

"name": "bob-update"

}

The PATCH request updates partially, so in the above example, the attribute age is kept unaltered and only the name is updated.

When using PATCH is not an option, use the POST verb instead. Don’t overload PUT. HTTP defines that PUT is for fully updating or replacing a resource [RFC7231].

To replace the content of the user identified with the value 8646291 a client submits the following request:

PUT /users/8646291 HTTP/1.1

Host: example.com

Content-Type: application/json {

"age": 54

}

After the PUT request, the user identified by the given id will miss the attribute name because the PUT request replaces the user information with the body provided within the request.

Finally to delete the resource identified with the value 8646291 a client submits the following request:

DELETE /users/8646291 HTTP/1.1

Host: example.com

2. Listing and Pagination

A client can retrieve multiple elements with the GET verb and filter them with query parameters. The result must be paginated, most used paginations are Cursor-based Pagination and Offset-based Pagination.

Each pagination has an input parameter that limits the number of items the page will contain, it is provided as a query parameter, let’s call it limit .

Cursor-based Pagination

Maybe you have also seen it with the name of Keyset-based Pagination, it is the most efficient way to paginate when you have a very large dataset because performs better than Offset-based Pagination under most implementations.

When a client requires a collection the server reply with the items and within each one provides a cursor that, at the time of the server response, falls on the item it is included. Here an example:

# REQUEST

GET /users?limit=100 HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"items": [

{

"id": 123,

"meta":{

"cursor": "js3Hsji3nj"

}

},

// other items ...

{

"id": 426,

"meta":{

"cursor": "ke3Gdk1xyi"

}

}

]

}

As you can see each item has an attribute meta.cursor , this cursor is a random string of characters that marks a specific item on a list of items, can be used to retrieve next or previous elements providing it as the value of after and/or before parameters.

When after is provided, the returned items must have as its first item the item that is immediately after the cursor. If there are no items after the cursors the returned collection must be empty. When before is provided, the returned collection must have as its last item the item that is immediately before the cursor. A client submits the following request to retrieve next, previous or between range elements:

# RETRIEVE NEXT ELEMENTS

GET /users?after=ke3Gdk1xyi&limit=100 HTTP/1.1

Host: example.com # RETRIEVE PREVIOUS ELEMENTS

GET /users?before=js3Hsji3nj&limit=100 HTTP/1.1

Host: example.com # RETRIEVE ELEMENTS BETWEEN TWO ELEMENT

GET /users?after=ke3Gdk1xyi&before=js3Hsji3nj&limit=100 HTTP/1.1

Host: example.com

Offset-based Pagination

The Offset-based Pagination allows the client to jump to a certain page, but most of the time has worse performance with very big data sets, it is more widely known than the other paginations.

A client submits the following request to retrieve a collection:

# REQUEST

GET /users?limit=100 HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"items": [

// the results

]

}

To retrieve the next page the client must increase the skip query parameters:

// retrieve second page

GET /users?skip=100&limit=100 HTTP/1.1

Host: example.com // retrieve third page

GET /users?skip=200&limit=100 HTTP/1.1

Host: example.com

Page reference

You can simplify the pagination task using a page reference system, providing an opaque pointer to a page, or in other words, a cursor that marks a specific page.

The page reference or page cursor usually encodes (encrypts) the page position, i.e. the identifier of the first or last page element, the pagination direction, and the applied query filters to safely recreate the collection.

Let’s see an example that shows the page reference in action, a client submits a request to retrieve the first page:

# REQUEST

GET /users?limit=100 HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"items": [

// the results

],

"paging": {

"prev": "kewIJbwDS2Bsja...",

"next": "dFRdkdek2KLmcd..."

}

}

The paging.next is the page reference that the client should use to receive the next batch of responses, and the paging.prev is the page reference the client should use to collect the previous batch:

GET /users?page_ref=dFRdkdek2KLmcd&limit=100 HTTP/1.1

Host: example.com

After the first request, the limit parameter is the only parameter in the URL beside page_ref, this is because it is safe to modify the limit in between requests. Parameters that could break the request, such as sort order and filters, are embedded directly inside of page_ref or stored them somehow. Trying to add or modify filters or order parameters will cause the request to fail. If a different order or filtering is needed, then you must restart on the first page. It’s also important to note that the page reference usually is temporary and should not be saved for later use.

3. Many to Many relations

Some times two resources create a many-to-many relation, you can create a new resource that represents this relation, let’s see an example;

Given two resources; student and course, each student can rate a course. We can create a new resource that explicates the relation between student and course, let’s call it student-course-rates and let’s map operations to HTTP verbs;

// add a student-course rate

POST /student-course-rates // list all student-course rates filtering by student or course

GET /student-course-rates // delete a rate identified by the given id

DELETE /student-course-rates/:id

A student can add a rate to a course with the following call:

# REQUEST

POST /student-course-rates HTTP/1.1

Host: example.com

Content-Type: application/json {

"studentId": "3298wdi28dh28wid92",

"courseId": "93710949600282",

"rate": 10

} # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"id": 1239836164989016,

"student": {

"id": "3298wdi28dh28wid92",

"age": 18

},

"course": {

"id": "93710949600282",

"description": "..."

},

"rate": 10

}

Retrieve all rates of a course is really easy:

# REQUEST

GET /student-course-rates?course=93710949600282&limit=10 HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"items": [

{

"id": 1239836164989016,

"student": {

"id": "i-am-a-student@college.com",

"age": 18

},

"course": {

"id": "93710949600282",

"description": "..."

},

"rate": 10

},

// other results ...

]

}

As you can see a student-course-rates resource has an identifier, the attribute id , you can also omit it and use the pair (studentId, courseId), in this way a delete could be done giving the pair via query parameter; DELETE /student-course-rates?course=..&student=...

4. Fields filtering

Sometimes the client needs to choose which attribute should be included in the response due to performance reason, a good way to do that is requiring a query parameter fields that contains a comma-separated list of attributes.

Given the resource user:

# REQUEST

GET /users/12fw342ej1 HTTP/1.1

Host: example.com #RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

{

"id": "12fw342ej1",

"name": {

"familyName": "Muro",

"givenName": "Rupert"

},

"age": 67

}

a client can choose which attributes should be returned with the following request:

# REQUEST

GET /users/12fw342ej1?fields=name.familyName%2Cage HTTP/1.1

Host: example.com #RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

{

"name": {

"familyName": "Muro"

},

"age": 67

}

You can also map a subset of fields to predefined styles, in this way a client can choose the style (a predefined subset of fields) providing the query parameter style .

For example; we can map the fields id , name.familyName and age to the style compact and the id , name.familyName , name.givenName and age to the style complete .

# REQUEST the compact style

GET /users/12fw342ej1?style=compact HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

{

"id": "12fw342ej1",

"name": {

"familyName": "Muro"

},

"age": 67

} # REQUEST the complete style

GET /users/12fw342ej1?style=complete HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

{

"id": "12fw342ej1",

"name": {

"familyName": "Muro",

"givenName": "Rupert"

},

"age": 67

}

5. Long-running operations

To improve scalability and simplify the deploy a web service must keep response time as lower as possible, but sometimes we need to compute long-running operations, how can we do that?

First of all, create a resource that represents the long-running operation, when the client submits a GET request to the operation resource, reply as following depending on the current status of the operation:

the operation is still running : reply with status code 200 (Ok) and a representation of the operation status.

: reply with status code 200 (Ok) and a representation of the operation status. the operation finishes with a success : reply with status code 303 (See Other) and the Location header containing the URI to the created resource.

: reply with status code 303 (See Other) and the Location header containing the URI to the created resource. the operation finishes with a failure: reply with status code 200 (Ok) and a representation of the operation status giving the information about the failure.

Let’s see the solution with an example; design a web service that extracts from a URI a summary; we have two resources; summary and extraction-task;

// retrieve a summary identified by the given id

GET /summary/:id // create a long running task that will extract the summary

POST /extraction-task // return information of the task identified with the given id

GET /extraction-task/:id

A client can create a new extraction task with a POST request, the server will reply with status code 202 (Accepted) and a representation of the new resource with any useful information, e.g. the date after which the client can retry to check the status of the operation (the checkAfter attribute).

# REQUEST

POST /extraction-task HTTP/1.1

Host: example.com

Content-Type: application/json {

"from": {

"uri": "https://extract.from.here.com"

}

} # RESPONSE

HTTP/1.1 202 Accepted

Content-Type: application/json;charset=UTF-8

Content-Location: HTTP/1.1 202 AcceptedContent-Type: application/json;charset=UTF-8Content-Location: https://example.com/ extraction-task /3 48wd39

"id":

"state": "pending",

"checkAfter": "2019-01-10T22:32:12Z",

"info": {

"from": {

"uri": "https://extract.from.here.com"

}

}

} "id": 3 48wd39,"state": "pending","checkAfter": "2019-01-10T22:32:12Z","info": {"from": {"uri": "https://extract.from.here.com"

The client can then monitor the state with a GET request, if the server is still processing the operation it will reply as:

# REQUEST

GET /extraction-task/

Host: example.com GET /extraction-task/ 3 48wd39 HTTP/1.1Host: example.com # RESPONSE

HTTP/1.1 202 Accepted

Content-Type: application/json;charset=UTF-8

"id":

"state": "pending",

"checkAfter": "2019-01-10T22:32:12Z",

"info": {

"from": {

"uri": "https://extract.from.here.com"

}

}

} "id": 3 48wd39,"state": "pending","checkAfter": "2019-01-10T22:32:12Z","info": {"from": {"uri": "https://extract.from.here.com"

when the server completes the operation with a success it will reply with a 303 (See Other), which means that the result can be found under another URI using a GET method, it doesn’t mean that the resource has

moved to a new location:

# REQUEST

GET /extraction-task/

Host: example.com GET /extraction-task/ 3 48wd39 HTTP/1.1Host: example.com # RESPONSE

HTTP/1.1 303 See Other

Location:

Content-Location: HTTP/1.1 303 See OtherLocation: h ttps://example.com/summary/239rfh392Content-Location: https://example.com/ extraction-task /3 48wd39

"id":

"state": "completed",

"info": {

"from": {

"uri": "https://extract.from.here.com"

}

},

"finishDate": "2019-01-10T22:35:11Z"

} "id": 3 48wd39,"state": "completed","info": {"from": {"uri": "https://extract.from.here.com"},"finishDate": "2019-01-10T22:35:11Z"

if the operation fails the server will reply as:

# REQUEST

GET /extraction-task/

Host: example.com GET /extraction-task/ 3 48wd39 HTTP/1.1Host: example.com # RESPONSE

HTTP/1.1 200 OK

Location:

Content-Location: HTTP/1.1 200 OKLocation: h ttps://example.com/summary/239rfh392Content-Location: https://example.com/ extraction-task /3 48wd39

"id":

"state": "failed",

"info": {

"from": {

"uri": "https://extract.from.here.com"

}

},

"finishDate": "2019-01-10T22:35:11Z",

"detail": "The URI doesn't exist (status code 404)."

} "id": 3 48wd39,"state": "failed","info": {"from": {"uri": "https://extract.from.here.com"},"finishDate": "2019-01-10T22:35:11Z","detail": "The URI doesn't exist (status code 404)."

If you prefer a callback-style solution, just require a URI during operation creation on which the client will be notified when the operation ends;

# REQUEST

POST /extraction-task HTTP/1.1

Host: example.com

Content-Type: application/json {

"from": {

"uri": "https://extract.from.here.com"

},

"notifyOn": "https://client.com"

}

6. Concurrency handling

A server can serve many clients at the same time, this increases the chance to incur into concurrency problem, e.g. two clients that modify the same resource at the same time with a PUT or POST request. The solution comes from RFC using Conditional Request (RFC7232).

The Conditional Request requires that the server provides one or both conditional headers; Last-Modified and ETag for each resource. To do a Conditional Request the client must send one or both conditional headers; If-Unmodified-Since and If-Match.

Let’s see an example: the goal is to provide Conditional Request to the resource user. The server must provide one or both of the two headers Last-Modified and ETag, on our example we will provide both;

# REQUEST

GET /users/12fw342ej1 HTTP/1.1

Host: example.com # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

ETag: "abcec491d0a4e8ecb8e14ff920622b9c"

Last-Modified: Sun, 05 Jan 2019 14:14:52 GMT {

"id": "12fw342ej1",

"name": {

"familyName": "Muro",

"givenName": "Rupert"

},

"age": 67

}

To be compliant to Conditional Request the client must provide one or both of two headers If-Unmodified-Since and If-Match. If none of them are provided the server will reply with a 403 (Forbidden) explaining why in the body of the response.

# REQUEST

PUT /users/8646291 HTTP/1.1

Host: example.com

Content-Type: application/json {

"age": 54

} # RESPONSE

HTTP/1.1 403 Forbidden

Content-Type: application/json;charset=UTF-8 {

"code": "120",

"message": "The conditional headers are required; If-Unmodified-Since and/or If-Match"

}

If the client provides at least one conditional headers in the request, the server must compare them with the current values of Last-Modified and/or ETag. If they match, it can process the update and return 200 (OK) or a 204 (No Content).

# REQUEST

PUT /users/8646291 HTTP/1.1

Host: example.com

If-Unmodified-Since: Sun, 05 Jan 2019 14:14:52 GMT

If-Match: "abcec491d0a4e8ecb8e14ff920622b9c"

Content-Type: application/json {

"age": 54

} # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

ETag: "1e0e5a0fb102db75aa36d4356936fe4c"

Last-Modified: Sun, 05 Jan 2019 14:15:02 GMT {

"id": "8646291",

"age": 54

}

If not, the server must return status code 412 (Precondition Failed) explaining why in the body of the response.

# REQUEST

PUT /users/8646291 HTTP/1.1

Host: example.com

If-Unmodified-Since: Sun, 05 Jan 2019 13:14:52 GMT

If-Match: "b1b3833b514f4b4a5207b572405e786f"

Content-Type: application/json {

"age": 54

} # RESPONSE

HTTP/1.1 402 Precondition Failed

Content-Type: application/json;charset=UTF-8 {

"code": "121",

"message": "The provided conditional headers doesn't match current values; The request rely on stale informations"

}

7. Versioning

Sometimes we need to version the API, providing different versions can significantly complicate the understanding and maintaining the API, you should use it as a last resort. You can use the media-type versioning via the headers Accept and Content-Type. Let’s see an example on the resource user;

# REQUEST VERSION 1

GET /users/12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json;version=1 # RESPONSE VERSION 1

HTTP/1.1 200 OK

Content-Type: application/json;version=1;charset=UTF-8 {

"id": "12fw342ej1",

"familyName": "Muro",

"givenName": "Rupert"

"age": 67

} # REQUEST VERSION 2

GET /users/12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json;version=2 # RESPONSE VERSION 2

HTTP/1.1 200 OK

Content-Type: application/json;version=2;charset=UTF-8 {

"id": "12fw342ej1",

"name": {

"familyName": "Muro",

"givenName": "Rupert"

},

"age": 67

}

8. Combine resources into composites

Sometimes there is the need to show many resources in the same place, the client has to call several endpoints and then combine the representation they need. Based on client usage patterns, performance and latency requirements we can create a new resource that aggregates multiple resources.

Let’s do an example; we need to show a page that recaps a financial state of a user, the page needs to show the following resources; user’s information, first 10 investments, last 10 bank records, the total balance, and credit card limits. Requiring each resource separately leads to performance issues.

# REQUIRE EACH REQUEST

GET /users/12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json GET /investiments?user=12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json GET /bank-records?user=12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json GET /credit-card?user=12fw342ej1 HTTP/1.1

Host: example.com

Accept: application/json GET /bank-account/ew239wqw21ui32une HTTP/1.1

Host: example.com

Accept: application/json

To solve the problem create a resource that aggregate results, let’s call it financial-report and because it is strictly related to a user it can be a subresource, the only available verb is GET;

# REQUIRE EACH REQUEST

GET /users/12fw342ej1/financial-report HTTP/1.1

Host: example.com

Accept: application/json # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8 {

"userInfo": { .. },

"lastInvestiments": [...],

"lastBankRecords": [...],

"bankAccount": 12321,

"creditCartLimits": {...}

}

9. Multi-language fields

HTTP provides two headers for language content negotiation; Accept-Language and Content-Language. The Accept-Language header is provided by the client to inform the server about the preferred language, the Content-Language header is provided by the server within the response.

The structure of a multi-language field should contain all the translations and a value that is populated based on the Accept-Language header, let’s see an example, a client requires a resource product which has the description attribute that is a multi-language field.

# REQUEST

GET /products/782hb1yufhd8923 HTTP/1.1

Host: example.com

Accept-Language: en,en-US,it

Accept: application/json # RESPONSE

HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8

Content-Language: en

Vary: Accept-Language {

"id": "782hb1yufhd8923",

"description": {

"localizedValue": "This is a description",

"translations": [

{

"lang": "en",

"value": "This is a description"

},

{

"lang": "it",

"value": "..."

}

]

}

}

As you can see the field description has an attribute named localizedValue that is valorized to the correct translation based on the Accept-Language provided by the client. If the header is missing then choose a default language.

To create a product the attribute localizedValue is not required:

# REQUEST

POST /posts HTTP/1.1

Host: example.com

Content-Type: application/json {

"description": {

// the 'localizedValue' must not be provided

"translations": [

{

"lang": "en",

"value": "This is a description"

},

{

"lang": "it",

"value": "..."

}

]

}

}

Conclusion

In the future, I will create for each topic a separate article to explore deeply the given solution and in some case show alternatives. Let me know if you found something useful or if you have some advice to improve the above solutions.