Part I: Generating API Metadata

Background

When developing a web application, a common design pattern is to divide the project into two repositories: 1) the API, or “back end”, which handles database or service interaction (“back end”), and 2) the client, or “front end”, which handles user interaction.

This series of articles will describe a method to super-power a Vue.js client — or any JavaScript client — using metadata exported from a Django REST Framework API.

But before we look at how to share metadata from the API to the client, let's dig into why it's worth the effort.

Motivation

Resolving one API-client discrepancy is a time sink. Resolving multiple discrepancies is a time bathtub.

Photo by Iz & Phil on Unsplash

Maintaining distinct API and client repositories is good for separating server-side and presentation logic, but those repos can be difficult to keep in sync.

If your application is form-heavy, there are a number of aspects of client and API code that need to match:

Field names, like givenName vs firstName or email vs email_address

vs or vs Variable casing, like firstName vs first_name

vs Which fields are required

Which options to show in select elements

Default field values

Min/max length requirements for text fields

Min/max value requirements for number fields

Regex patterns for text fields

Which fields are read-only

Whew. Look at everything that can get out of whack. Resolving one API-client discrepancy is a time sink. Resolving multiple discrepancies is a time bathtub.

What if the client knew what the API expected? If the API could tell the client ahead of time about its data structures, it would be simple to code the client to match those API requirements.

In other words, the client needs data about what data the API expects.

Data about data? We're talking metadata.

Metadata

Django REST Framework (DRF) has built-in support for generating metadata about API endpoints.

For an API that uses ModelViewSet s, performing an OPTIONS request on an endpoint returns a response with useful information about the Django model corresponding to that endpoint.

The journey of metadata from model to OPTIONS response is:

Model → Serializer → View Set → Metadata class → OPTIONS Response

Note: metadata is generated for each field specified on the serializer. If your serializer specifies only a subset of a model's fields, the metadata will represent only those fields and not the entire model.

To see what metadata DRF gives us for free, here's a sample OPTIONS response for an API endpoint that lists and creates Users:

{ "name" : "User List" , "description" : "" , "renders" : [ "application/json" , "text/html" ], "parses" : [ "application/json" , "application/x-www-form-urlencoded" , "multipart/form-data" ], "actions" : { "POST" : { "id" : { "type" : "integer" , "required" : false , "read_only" : true , "label" : "ID" }, "password" : { "type" : "string" , "required" : true , "read_only" : false , "label" : "Password" , "max_length" : 128 }, "last_login" : { "type" : "datetime" , "required" : false , "read_only" : false , "label" : "Last login" }, "is_superuser" : { "type" : "boolean" , "required" : false , "read_only" : false , "label" : "Superuser status" , "help_text" : "Designates that this user has all permissions without explicitly assigning them." }, "username" : { "type" : "string" , "required" : true , "read_only" : false , "label" : "Username" , "help_text" : "Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only." , "max_length" : 150 }, "first_name" : { "type" : "string" , "required" : false , "read_only" : false , "label" : "First name" , "max_length" : 30 }, "last_name" : { "type" : "string" , "required" : false , "read_only" : false , "label" : "Last name" , "max_length" : 150 }, "email" : { "type" : "email" , "required" : false , "read_only" : false , "label" : "Email address" , "max_length" : 254 }, "is_staff" : { "type" : "boolean" , "required" : false , "read_only" : false , "label" : "Staff status" , "help_text" : "Designates whether the user can log into this admin site." }, "is_active" : { "type" : "boolean" , "required" : false , "read_only" : false , "label" : "Active" , "help_text" : "Designates whether this user should be treated as active. Unselect this instead of deleting accounts." }, "date_joined" : { "type" : "datetime" , "required" : false , "read_only" : false , "label" : "Date joined" }, "groups" : { "type" : "field" , "required" : false , "read_only" : false , "label" : "Groups" , "help_text" : "The groups this user belongs to. A user will get all permissions granted to each of their groups." }, "user_permissions" : { "type" : "field" , "required" : false , "read_only" : false , "label" : "User permissions" , "help_text" : "Specific permissions for this user." } } } }

The first four attributes relate to the endpoint itself. For metadata about the User model, actions is where the action is.

We can see that in a POST request, this API endpoint expects a number of fields, only two of which are required: username and password . Each field contains some or all of the following metadata:

type : the expected value type (string)

: the expected value type (string) required : whether the field is required (it is)

: whether the field is required (it is) read_only : whether the field is read-only (it isn't)

: whether the field is read-only (it isn't) label : a user-facing name (“title”)

: a user-facing name (“title”) max_length : the value's maximum number of characters (100)

We can't see this in the User model's metadata above, but additional information will be included if applicable:

choices : model field choices as a list of objects with display_name and value keys

: model field choices as a list of objects with and keys child : metadata about a ForeignKey 's to model or an ArrayField 's base field

min_length , max_value , min_value : other model field validations

Not bad! Out of the box, we can get useful information about an API endpoint's data expectations, and this information corresponds to quite a few items in the above list of time sinks to sync. That means we should be able to use this metadata in our client to help keep our client code in sync with API expectations.

To use this metadata while developing a client, however, we need this metadata offline (i.e., without doing an OPTIONS request), and we need more.

Extending Metadata

Means and motive to muster more metadata.

DRF supports custom metadata classes, which can generate the metadata to show in an OPTIONS response — or offline, as we'll see in a bit.

Django's default metadata class extracts a good amount of information from the view set's ModelSerializer , as we saw above, but we can be greedy and get even more.

Take a look:

from rest_framework.metadata import SimpleMetadata from rest_framework.schemas.openapi import AutoSchema class APIMetadata ( SimpleMetadata ): """Extended metadata generator.""" def get_field_info ( self , field ): field_info = super () . get_field_info ( field ) # Add extra validators using the OpenAPI schema generator validators = {} AutoSchema () . _map_field_validators ( field , validators ) extra_validators = [ 'format' , 'pattern' ] for validator in extra_validators : if validators . get ( validator , None ): field_info [ validator ] = validators [ validator ] # Add additional data from serializer field_info [ 'initial' ] = field . initial field_info [ 'field_name' ] = field . field_name field_info [ 'write_only' ] = field . write_only return field_info

This custom APIMetadata class adds:

initial : a field's default value

: a field's default value pattern : a Regex pattern a field's value must match

: a Regex pattern a field's value must match format : the format of a string value, like email , uri , or uuid

: the format of a string value, like , , or field_name : a field's name

: a field's name write_only : whether a field should be written and not read

( pattern and format come care of the validators from DRF's OpenAPI schema generator.)

Although we didn't add very many pieces of metadata, the attributes we added are useful in a client:

initial can populate fields with a default value

can populate fields with a default value pattern can be used with HTML5 input constraint validation for input Regex validation

can be used with HTML5 input constraint validation for input Regex validation format can help determine when to use an <input type="email"> or <input type="url">

can help determine when to use an or field_name ensures that all field metadata is in the field metadata object instead of distributed across the key and value

ensures that all field metadata is in the field metadata object instead of distributed across the key and value write_only can help determine when to use an <input type="password">

Now that we've got means and motive to muster more metadata, let's get it out of the API and into the client.

Exporting Metadata

To export metadata in a format that's useful in a client code base, we'll use APIMetadata to generate metadata for a User serializer, then write it to a JSON file.

I recommend writing a management command to export JSON metadata for each model you plan to use in your client, but to keep things moving, here's a quick way to do that from a Django shell ( ./manage.py shell ):

>>> import json >>> from your_app.serializers import UserSerializer >>> metadata_generator = APIMetadata () >>> metadata = metadata_generator . get_serializer_info ( UserSerializer ()) >>> with open ( 'User.json' , 'w' ) as json_file : ... json . dump ( metadata , json_file , indent = 2 , sort_keys = True )

In contrast to the OPTIONS response, this metadata has a shorter journey from model to JSON:

Model → Serializer → Metadata class → JSON file

And here's the resulting User.json file:

{ "date_joined" : { "field_name" : "date_joined" , "initial" : null , "label" : "Date joined" , "read_only" : false , "required" : false , "type" : "datetime" , "write_only" : false }, "email" : { "field_name" : "email" , "format" : "email" , "initial" : "" , "label" : "Email address" , "max_length" : 254 , "read_only" : false , "required" : false , "type" : "email" , "write_only" : false }, "first_name" : { "field_name" : "first_name" , "initial" : "" , "label" : "First name" , "max_length" : 30 , "read_only" : false , "required" : false , "type" : "string" , "write_only" : false }, "groups" : { "field_name" : "groups" , "help_text" : "The groups this user belongs to. A user will get all permissions granted to each of their groups." , "initial" : [], "label" : "Groups" , "read_only" : false , "required" : false , "type" : "field" , "write_only" : false }, "id" : { "field_name" : "id" , "initial" : null , "label" : "ID" , "read_only" : true , "required" : false , "type" : "integer" , "write_only" : false }, "is_active" : { "field_name" : "is_active" , "help_text" : "Designates whether this user should be treated as active. Unselect this instead of deleting accounts." , "initial" : false , "label" : "Active" , "read_only" : false , "required" : false , "type" : "boolean" , "write_only" : false }, "is_staff" : { "field_name" : "is_staff" , "help_text" : "Designates whether the user can log into this admin site." , "initial" : false , "label" : "Staff status" , "read_only" : false , "required" : false , "type" : "boolean" , "write_only" : false }, "is_superuser" : { "field_name" : "is_superuser" , "help_text" : "Designates that this user has all permissions without explicitly assigning them." , "initial" : false , "label" : "Superuser status" , "read_only" : false , "required" : false , "type" : "boolean" , "write_only" : false }, "last_login" : { "field_name" : "last_login" , "initial" : null , "label" : "Last login" , "read_only" : false , "required" : false , "type" : "datetime" , "write_only" : false }, "last_name" : { "field_name" : "last_name" , "initial" : "" , "label" : "Last name" , "max_length" : 150 , "read_only" : false , "required" : false , "type" : "string" , "write_only" : false }, "password" : { "field_name" : "password" , "initial" : "" , "label" : "Password" , "max_length" : 128 , "read_only" : false , "required" : true , "type" : "string" , "write_only" : false }, "user_permissions" : { "field_name" : "user_permissions" , "help_text" : "Specific permissions for this user." , "initial" : [], "label" : "User permissions" , "read_only" : false , "required" : false , "type" : "field" , "write_only" : false }, "username" : { "field_name" : "username" , "help_text" : "Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only." , "initial" : "" , "label" : "Username" , "max_length" : 150 , "pattern" : "^[\\w.@+-]+$" , "read_only" : false , "required" : true , "type" : "string" , "write_only" : false } }

Look at all that delicious metadata!

Wrap-up

In this post, we looked at some of the pain points of maintaining a web application with a separate API and JavaScript client, and how we can use metadata to help keep a client in sync with its API.

By extending DRF's default metadata class, we unlocked even more information about API models and exported that data into a client-friendly JSON format.

Now that we've got all this metadata, we should do something with it.

In Part II we'll start using this metadata for client development, with a little help from TypeScript.