A New Era for Drupal's JSON:API

20 September, 2019

On Monday (2019-9-16), I published the 8.x-1.0 version of the JSON:API Hypermedia module (hypermedia is just a fancy name for things with links). It took lots of work in my current role at Acquia and I depended on the help of my friends (and colleagues) Peter Weber (zrpnr), Wim Leers (same), and Mateu Aguiló Bosch (e0ipso).

I believe this opens a new era in the decoupled Drupal ecosystem.

What does this module do? It provides an API for modules to add links to JSON:API responses .

Neat, right?!

Oh… why does that matter, you ask? Well, before I answer that, let’s think a little more deeply about the web that we know and love and the links that drive it.

Consider what it would be like to build a website without links. It would have no menus, no login button, no links between related pages, and no pagination. Already, that sounds like a fairly terrible website.

Yet links on the web are far more prevalent than the ubiquitous <a> element. If we look a little deeper, there are many, many more. Without links, the web would be little more than an internet-connected file browser.

Each of these HTML elements represents a kind of link on the web too:

<link rel="canonical" href="...">

<link rel="stylesheet" href="...">

<script type="application/javascript" src="...">

<img src="...">

<audio src="...">

<video src="...">

<form action="...">

Of course, let’s not forget that we use links in CSS as well. What’s background-image: url(...) if not a link?

If we couldn’t build a website with links, we couldn’t build much of a website at all! Beyond helping a user navigate between websites and webpages, links enrich every page we visit. They add images, videos and audio; they add user interactivity; they make a webpage visually appealing and intuitive.

Maybe you paused before and asked yourself, “is <form> really a link?” Sure it is. A <form> element’s action attribute takes a URL and its method attribute takes an HTTP method ( GET or POST ). Similarly to how your web browser knows that when a user clicks on an <a> element that:

it should follow the href URL using a GET HTTP request

URL using a HTTP request it should then refresh the current browser tab with the HTTP response

Your web browser also knows that when a user clicks on a <form> element’s inner <button> or <input type=”submit”> element that:

it should gather some <input> values

values it should send those values to the action URL using a GET or POST HTTP request

URL using a or HTTP request it should then refresh the current browser tab according to the HTTP response

While the browser’s steps to follow the <form> link are slightly more complicated, we can see that fundamentally, the <a> and <form> elements both represent instructions to the browser and each represents a relationship between one resource on the web and another (a location that the user can visit and a location where a form can be submitted, respectively).

Likewise, an <img> element tells your browser:

it should request the src URL using a GET HTTP request

URL using a HTTP request it should render the HTTP response (containing the image’s binary data) in the current browser tab

However, unlike the previous two examples, processing an <img> link doesn’t refresh a browser tab. Instead, it updates the existing rendering with the newly fetched data.

Going further, the <script> link is the most powerful of them all. It tells your browser:

it should request the src URL using a GET HTTP request

URL using a HTTP request it should parse the HTTP response containing the script

it should execute that script in the context of the current browser tab

If that link is not awesome enough already, consider that the script itself can add links to the DOM (say, to lazy load images or fetch other scripts).

Finally, links are not just for browsers. Other utilities, like a search engine crawler, can process links for purposes not visible to a user at all. The <link rel="canonical"> element is a familiar example. This link is often used to establish a “canonical URL” for a search result. Under the hood, the link is giving instructions to the crawler. It tells the crawler:

it should deprioritize the current document

it should request the href URL using a GET HTTP request

URL using a HTTP request it should index that document as a primary source

How does this understanding of links relate to decoupled Drupal and a new contributed module? I believe that when many of us think of REST APIs, we don’t realize that all of the richness provided by links in HTML is also available to our JSON-based APIs. Moreover, little prevents those links in JSON from being just as functional as a <form> element. In fact, we’ll see later that they can be made more functional than anything we’ve already seen. That’s because links in our own APIs can be extended and customized for our particular applications.

Let’s stop for a moment and internalize this: our web browsers are, at their core, highly sophisticated REST clients that understand responses of Content-Type: text/html .

Now, what if our JSON:API clients could take as much advantage of links as browsers do? Could our single-page JavaScript applications mature into sophisticated REST clients that understand responses of Content-Type: application/json , or, in Drupal’s JSON:API case, Content-Type: application/vnd.api+json ?

Absolutely! In the same way that a browser “knows” how to parse and process links in a text/html document (based on whether the link is an <img> or <form> element) a client that understands application/vnd.api+json documents can parse and process many different kinds of links too. This means that if we can enrich JSON:API responses with more dynamic and powerful links, we can unlock a new realm of sophistication for our decoupled projects.

A hyperdrive engine for decoupled applications

As Drupal modules start to provide those more dynamic and powerful links, Drupal will be able to power native applications, kiosks, web apps and more in ways that it previously hasn’t been able to. Drupal will grow from being a mere content repository for those decoupled applications into an engine that drives those decoupled applications.

What I find most exciting about this new era is that decoupled Drupal implementations will be able to do much more than replicate the HTML links we examined above, they’ll be able to create new types of links of their own!

So, let’s do it! Let’s define a format for JSON link objects—and don’t worry if these definitions seem dense, there’s an interactive example further below that will bring it all together. Let’s say that a link object can have the following properties:

rel : this is short for link relation type and it tells the client what type of link the link object represents. Using different values for this property is like using different HTML elements (e.g. form vs img ). There’s already a registry of link relation types, but developers can invent their own using “extension” link relation types in order to uniquely serve their application.

: this is short for link relation type and it tells the client what type of link the link object represents. Using different values for this property is like using different HTML elements (e.g. vs ). There’s already a registry of link relation types, but developers can invent their own using “extension” link relation types in order to uniquely serve their application. title : this is a title for the action supported by the link. A client might choose to render this as button text (which means that the text doesn’t need to be hardcoded into the client!)

: this is a title for the action supported by the link. A client might choose to render this as button text (which means that the text doesn’t need to be hardcoded into the client!) confirm : the presence of this custom target attribute hints that the client should ask for confirmation before following the link. To help the client, it can suggest language for that confirmation.

: the presence of this custom target attribute hints that the client should ask for confirmation before following the link. To help the client, it can suggest language for that confirmation. data : this is another custom target attribute. In our example below, it provides the client with specific data to send as part of an HTTP request.

As a concrete example, let’s imagine that we operate an online store and users can purchase a product if it’s in stock or save it to their wishlist otherwise.

If we write a client that understands the generic link object defined above, we’d be able to add an action link to every product. If a product is in stock, the server would set the title to “Buy now!” and, if not, it’d set the title to “Save for later”. Take a look at the example below and try changing the value of the Stock field, observe how the link adapts. Notice that when the title property changes, the button adapts accordingly. Go ahead and click on the link. When the product is out of stock, the confirm property becomes false and the button processes the link immediately. When the confirm property is a string, there’s an extra confirmation button to click.

There’s something very exciting hiding in plain sight: because the link adapts to product availability, the client doesn’t have to. In fact, the client doesn’t need to know anything about how a product is stocked at all! Without that dynamic link, the client would otherwise have had to check some stock field or to have had to make an HTTP request to determine if the product was available.

This pattern means that months or years later, when some developers realize that products can be discontinued, all that must be changed is the link. By updating the link to direct the client to a different product and without changing a single byte of client code, the application could evolve to support an entirely new scenario! Try checking the Discontinued checkbox above. The new feature just works.

Had the client been checking the availability of the product, not only would the server have had to make a new endpoint or field to communicate that a product is discontinued, the client would need to been updated in lock-step with that change 😨.

You don’t have to take my word for it, the button-generating code in the example doesn’t “know” anything about products, wishlists, stocks or carts. It only knows how to process the generic link we defined. Here’s a link to the demo’s source code.

When you have complete control over both the client and the server (e.g. in a progressively decoupled block) this pattern might seem nice, but not necessary. However, consider that as applications—and teams—grow in size and complexity, this pattern helps you move more quickly. What if your client were a single-page JavaScript application? What if it were a native mobile application on a phone that rarely gets updated? What if you had to maintain both? Finally, even if you do have complete control over the client and the server, what if links could be defined by contrib modules? Wouldn’t you want them to use a similar pattern?

A toolbox, not an engine

I began this post by asking what the JSON:API Hypermedia module does. The answer was simple: “It provides an API for modules to add links to JSON:API responses.” That answer hasn’t changed. However, I hope that by seeing all of what this actually enables, you’re as excited as I am!

It would certainly have been possible to add lots of new, hardcoded links to the JSON:API module in order to create a tighter integration with Drupal core’s features (like publishing and unpublishing content), but Drupal shines at its brightest when users can combine specialized modules to serve their exact needs. Now, modules have a place to provide those specialized behaviors for the decoupled ecosystem too.

That’s why I believe this opens a new era in the decoupled Drupal ecosystem.

Imagine if the Flag module starts decorating JSON:API responses with links to create bookmarks, or if the Commerce module started decorating them with links to purchase products, or that your module might start decorating responses with links to drive your own custom components!

Wrapping up

Over the coming weeks, I will be publishing more posts that explore the possibilities that the JSON:API Hypermedia module unlocks in greater detail. Each post will dive deeper into the ideas that informed this blog post. To do that, the posts will be broken up into different categories—links that navigate, links that instruct, and links that describe—and each will give examples of when, where and how you can use those links to solve otherwise difficult and inelegant problems. My goal will be to show that many of the “hardest problems” in decoupled Drupal development are simplified through the use of hypermedia—be it decoupled menus, access control and user interactions, or schemas and developer tools.