I’ve seen a number of discussions lately that show there’s a big gap in terms of the basic core knowledge of what APIs are, what they’re for, and how they work.

For a start, the term API stands for Application Programming Interface. It’s a set of instructions and functionality exposed to developers. In development it’s used in two main contexts. For a start, people will often refer to the set of functions and functionalities a piece of software has as its API. Git’s API for example includes things like fetch and checkout. React’s API includes its render functions and bindings. PHP’s API includes a metric buttload of functions with awful names.

None of that is what we’re talking about here. When web developers talk about “an API” they’re most typically talking about a web accessible API. That’s what we’re going to refer to as an API from here on.

What is an API for?

An API has a clearly defined and simple job — to expose functionality of the application to allow it to be called from the web. The most common use of that is specifically to allow another piece of software access to the data of that application. It might be saving a user, loading a list of projects, or unsubscribing from a newsletter, or.. just about anything.

It’s important to note at this point that there are literally no rules about what constitutes an API, how it works or what functionality it provides. There are conventions, but they’re more sort of guidelines.

Because an API is intended to talk to machines, not humans, they typically don’t return any sort of HTML or layout. They return pure data.

There aren’t any specific rules on what format this takes. It could potentially be some sort of microformat like a calendar event, or a contact card. Most commonly it would have been XML until a few years ago, but now it’s nearly universally JSON.

Despite operating on the internet and using HTTP or HTTPS, an API isn’t intended to be used with a browser. Some functionality might be accessable, but it would be a very limited subset.

What kinds of APIs are there and how do they work?

The intention is that you can hit a specific URI with a request that may or may not include some data, it should do a given job. This URI is commonly known as an endpoint. In theory, there could be anything. In practice, APIs are typically one of two types.

SOAP

The first is SOAP: Simple Object Access Protocol. SOAP is an older approach, initially developed by Microsoft. It nearly always uses XML for everything, and is typically used in old and legacy APIs. We’re mostly talking about it here by way of contrast.

SOAP APIs set up a series of functions, in a verbNoun form: saveUser(), loadProducts(), etc. A file called a WSDL provides a list of the available functions, and what their requirements and inputs (and outputs) are. However, SOAP functions can be literally anything, it could be saveUser, newUser, createUser, updateUser, patchUser, it could depend on the context, etc. While the WSDL provides the literal requirements it does a very poor job of conveying the intent or actual real-world usage of complex APIs.

If you’re making an API now there is absolutely no reason to use SOAP. You make everything harder for yourself and everyone else.

REST

REST stands for REpresentational State Transfer, as determined by someone who doesn’t know how acronyms work. REST actually underpins vast amounts of the web in terms of typical browsing. All web pages are a REST resource, and the presence of a URL at all is part of this pattern.

REST implements a combination of resourcing, http verbs, correct http response codes, and hypermedia to provide a consistent and predictable pattern for data access.

What REST involves

In API terms, the REST approach is probably best described using something called the Richardson Maturity Model. Each level gets a bit closer to the pure intent, what people term more RESTful. Using this model makes it possible to understand and even implement REST a bit at a time.

Level 0: no REST

At level 0 (not even a bit RESTful) the API’s endpoints might be anything. The endpoint might be /api/save_user for example, and you send it form post directly. The problem with this is that it’s completely unpredictable. As above with the SOAP calls the functionality could be or do anything, and that’s not mitigated at all by a WSDL. This is very much the worst of all worlds.

Level 1: Resources

At level 1, endpoints are implemented as a resource. The users list will be found at /api/users. A specific user could be found at /api/users/233. That user’s comments might be at /api/users/233/comments. It’s not encouraged to nest resources any more than this, though it’s entirely possible. You could call /api/users/233/comments/28876 if you wanted, but it would make much more sense to call /api/comments/28876.

Level 2: Verbs

At maturity level 2 it’s important to start properly supporting HTTP verbs. Every request made in HTTP has a “type”. Web developers will often see that as the method in a form, typically set as POST. Every request a browser makes has one of these types. Every page retrieved to view is doing a GET request, and almost every form is doing a POST.

But these aren’t the only verbs. Some of them, specifically DELETE, are pretty obvious in intent, but PUT and PATCH deserve further discussion.

Every verb does something different, but predictable, to any resource it’s called on.

GET requests do nothing but read data. They can make no changes. A GET on something like /api/projects will return a list of the projects. A GET on /api/projects/12 will return the single entry for that project.

POST requests create new entries. A POST request to /api/projects will create an entire new project. A POST request to /api/projects/12… actually doesn’t make any sense. It wouldn’t do anything.

PUT is intended to modify an existing entry. A PUT request to /api/projects makes as little sense as a POST to the specific project, and likewise shouldn’t do anything. A PUT on /api/projects/12 is intended to update that project. More specifically it’s actually intended to replace that project.

PATCH modifies small amounts of data. It also makes no sense in a list, but a PATCH request to /api/projects/12 might just include a new title, or a modified contact name.

DELETE is fairly obvious. In theory a DELETE request to /api/projects would delete all the projects, but in practice you wouldn’t do that because what the hell is wrong with you. A DELETE request to /api/projects/12 would obviously delete that project.

Now is a good time to clarify that DELETE is rarely supported at all, and that there is absolutely no obligation to support all or any of these verbs. Merely that if that functionality is required, it’s implemented correctly. Using a POST request to update a record is just wrong. Sending a get request with ?delete=true is also very wrong.

Level 2 Continues: Status Codes

Most people know some basic status codes. A 404 error means the file isn’t found. A 500 error means the server is on fire and all has gone wrong. Almost every page you go to (including this one) is getting a 200 OK. The status codes are grouped. Anything 2xx is a success. 3xx is a redirect. 4xx is a request error. 5xx is a server error.

There are some very specific status codes that provide useful information, that are really more correct than just using defaults. For example, when creating a new project it is more correct to return a 201 Created code, rather than a 200. Attempting to delete the list of projects might well return a 405 Method Not Allowed. Returning the correct status code allows anything consuming it to get a more nuanced understanding of what happened. In particular, just returning a 200 OK (the default response if just passing back data) will fail to trigger jQuery or another promise handler’s .fail() conditions. Returning a 4xx will allow you to put a message on your app to suggest the user did something wrong, while a 5xx might display a generic server error.

Level 3: Hypermedia

It sounds more complicated than it is. Hypermedia is often referred to in the context of the not-at-all-awful acronym HATEOAS. Hypermedia As The Engine Of Application State. The principle is that everything the application needs to drive its interactions with the API should be returned by the API itself. For the most part this is done through headers. For example, while the above creating of a new projects is better done with a 201 Created status code, it’s even betterer done by returning that status code and a link header that shows where to find the newly created resource. The application doesn’t need to go looking for it then, it’s already been told where to go.

Similar examples are pagination, where other link headers contain the current page, back and next links. The application doesn’t need to calculate out the page number, etc. It’s being told by the application. On returning a resource, the response may well include things like where to find the edit or delete endpoint for this resource, what the current sort options are, or where there are child resources.

Is all of this really necessary?

Technically, without fulfilling hypermedia an API is not RESTful. It’s a yes/no state, like you can’t be two thirds pregnant. Levels 1 and 2 of this maturity model are trivially easy to implement and give excellent bang for buck in getting a logical RESTful structure. If you don’t end up with advanced hypermedia controls fully implemented no one is going to hit you with a stick. When people tell you it’s “not technically RESTful” you can just nod and agree and keep doing what you like.

There’s a fair chance that you’re going to be the only one using your API, the sole consumer. The details are largely up to you.

API design is a complex and controversial field, and opinions often ride on extremely specific interpretations of standards and RFCs. Doing it correctly makes consuming the API using something like a Single Page App much easier. API creation using frameworks like Rails, Laravel for PHP or Hapi for NodeJS. In Laravel, for example, you can simply dump out an Eloquent object directly in the route.

Route::get('projects', function(){

return App\Project::all();

});

And this will give you a happily REST compliant JSON payload with a 200 OK status. Most frameworks are similarly optimised for this case.

Auth

If you look at the above code, it should be obvious that the code will return all the projects – regardless of who is making the request. This is a fairly obvious security flaw. It really should be limiting the response to only those from the user.

What needs to be implemented is Auth – authentication and authorisation. These aren’t the same thing. Authentication is about confirming that you are who you say you are. Authorisation is about determining what you do and do not have access to. When you log into a website with a username and password, you authenticate yourself. The system then determines what kind of user you are, and what sort of thing you should be able to access. Are you an admin? Which comments can you edit. That sort of thing.

A common solution is to scope the request like this. This is Laravel code, but something similar is possible anywhere.

return App\Project::forAuthenticatedUser()->get();

The details of authentication of an API could be a whole article in and of itself. The typical approach is to send a request that returns an authorisation token. This token is then sent in the headers with future requests, so the API knows it’s been authorised. JSON Web Tokens are fast becoming the default way to do this.

Standards and Documentation

There is no specific requirement in REST to do just about anything any specific way. This leaves an awful lot up to interpretation. For example, when successfully running a DELETE action, what should be returned? A 200? A 204 No Content? A 200 with the object that was deleted? How should the results be structured?

The JSON-API project attempts to clarify some of these questions, by providing a specification to work towards. The goal is to prevent bike shedding, unproductive argument about trivial details, so you can focus on just getting it done.

The JSON-API spec was originally developed by the Ember framework. It is now an open source spec, but Ember’s data abstraction system defaults to supporting it. Because it is a defined spec there are available transformations for a range of ORMs and frameworks to generate it, including Laravel’s Eloquent, a Django module, and NodeJS Sequelize or Mongoose.

Another interesting point is how to actually document these things. What resources are available? What methods are valid for this resource? Can they be filtered? What about pagination?

The definitive way to document an API is to use Swagger. Swagger is a documentation specification and a set of tools for designing and building APIs. I should quickly clarify that the role of something like Swagger and JSON-API are not at all in conflict. Swagger’s specifications defines how to document the API, not how to build it. Swagger generates a series of pages, static HTML, that document the API’s structure and inputs.

Publishing Swagger's output lets people (including yourself) see what is available, and how to interact with it.

Testing

As a final point, if you’re doing a lot of API work, look at downloading Postman. It lets you poke your endpoints with data, check responses, run automated tests, and a bunch of other stuff on REST APIs. Postman can run as a standalone application, or as a Chrome app. It can run tests in that app, and also has a command-line tool that can be run as part of a CI or TDD process.

Postman can also be used in combination with Swagger, confirming that the documentation matches the actuality.