Motivation

This summer we started our transition from REST to GraphQL. As the size of our REST API’s were increasing, we became ever more frustrated with the amount of time we spend on training new team members in Restful thinking and maintaining documentation. We had been using OpenAPI specifications but still found the development experience too cumbersome. The strongly typed query language, that is GraphQL, has enabled us to maintain documentation easily, carry typing to our frontend, and build intuitive customer specific endpoints in a developer-friendly way, right out of the box!

We’ve found, however, that when it comes to APIs, one size seldom fits all. Our internal APIs work on abstract terms such as entities and relations, but our customers need to get insights about companies and people. Allowing customers to integrate with our system would often require us to perform a lecture in our terminology before a single line of code could be written. Hardly a smooth onboarding.

Furthermore, when it comes to authorization, it’s never easy to express the right separation between data-access. Some customers may have licenses to specific sources that others do not. In our experience, traditional access scopes are hard to express, maintain, and have a tendency to blow up.

Our initial solution was to create a new GraphQL API for each customer. With the impressive code-generation features of the server-side framework we are using, sangria, this wasn’t too hard. However, as our customer base grew, we saw a rise in code-duplication which was hard to maintain and prone to errors. Building a new API with almost the same features as the last one, would require us to write hundreds of lines of boilerplate code and add new features multiple places.

Transforming a schema

If only there was a way to create and maintain a single API and expose a curated subset to our customers, i.e., specify a schema for our internal API and then transforming that into a customer API, with its schema.

Transformation tools aren’t something new, and there exist some JavaScript frameworks which allows you to filter and update the GraphQL schema programmatically, e.g., Apollo Schema transforms. However, as with most tools that can do everything, they tend to do nothing. Programming is hard, and sometimes you need a way to express simple features without mastering a general-purpose programming language. Even though JavaScript is the language of gods and should be thought to everyone from first grade. Of course.

So with that in mind, we decided to create a domain-specific language, called gintonic, that would enable us to quickly, and declaratively, express transformations of a GraphQL API. Transformations such as:

Filtering fields: Removing undesired fields from the target schema.

Locking arguments: Removing arguments and input fields from the target schema.

Updating documentation: Replacing outdated documentation on types, fields, arguments, and enum values.

Aliasing fields and types: Rename fields and types to make more sense in the target schema.

Disabling root operations: Disabling mutations or subscriptions entirely!

The language is heavily inspired by GraphQL, for a more intuitive programming experience, and can primarily be considered a function which takes a GraphQL Schema as an argument and producing a new Schema.

T is our transformation, which takes a GraphQL source schema (pink), and produces a target schema (green).

For instance, given the following very simple schema

Source schema.

we may apply the transformation

Transformation.

and produce the schema:

Target schema.

This transformation filters the secretField and aliases the field originally called field to f and to f2 . Field filtering is a powerful tool when it comes to controlling access to specific data for specific APIs. Field aliasing facilitates renaming fields to something that might make more sense in the concrete case and even replace the underlying field without breaking backward-compatibility. The same goes for type-aliasing.

Argument locking is what we call it when we provide the value for either a field-argument or an input object field. This feature is fundamentally different from specifying a default value since the argument will be removed from the target schema. The following example illustrates how argument locking works:

Source schema.

Transformation.

Target schema.

Here we perform two transformations; we alias the field fieldWithTwoArgs to fieldWithOneArg , and we lock the second argument, arg2 , in that field, effectively removing that argument from the target schema. Any queries on the new target field will have the user-specified first argument value and the transformation-locked second argument value.

Updating documentation: Since we are updating the signature of the schema, our documentation might become outdated. References to fields might break by aliasing or filtering, and the same goes for semantic documentation of arguments. Therefore it was essential for us to support updating the documentation. Supplying new documentation is done with string literals, just as in GraphQL, on the respective types, fields, enum-values, or arguments. The following example illustrates the two former cases:

Source schema.

Transformation.

Target schema.

Here we update the documentation on the Query object type and the field1 -field while preserving the documentation on the field2 -field.

Type aliasing is currently just type-renaming. When we specify a new type name in the target schema, all references to the original type are replaced with the new name, and the original type removed. All types, scalars, enums, input objects, interfaces, unions, and objects, can be renamed. The following example illustrates type aliasing on a scalar:

Source schema.

Transformation.

Target schema.

Notice that it is up to the developer to ensure that the transformations produce a type-correct schema.

Schema transformation; facilitates the disabling of root-operations by filtering mutations or subscriptions on our target API, entirely. Useful when we don’t want users altering data or streaming events.

Source schema.

Transformation.

Target schema.

Notice how the mutation type, Mutation , is removed entirely from the target schema, which is caused by applying tree-shaking as a transformation step, thus filtering all types not used implicitly or explicitly in the schema.

Besides the transformations mentioned above, gintonic performs three steps that ensure default values are consistent with the changed types; unused types are removed, providing a cleaner target schema, and finally the target schema type-checks according to the GraphQL specification:

Transform schema: The transformations are applied to the source schema generating the initial target schema. Values update: Default values and directive values are updated according to locked arguments in input objects. Tree-shaking: Unused types are removed cleaned from the schema. All types not used, explicitly or implicitly, will be removed from the resulting schema. Any definitions that are not type-system-definitions, such as type-system extension and executable definitions, are also removed. Type checking: The target schema is type-checked against the validation rules specified in the GraphQL specification.

The validity of the transformations is tightly coupled with the schema on which they are performed. If given a valid source schema, the transformations do not produce a valid target schema; the transformations are invalid.

Transforming operations

With the transformation successfully applied we now have a valid schema that we can host on a target server. A valid schema, however, means little to us without some way to resolve the incoming queries. GraphQL is a query language and as such bi-directional, and if we have to supply field-resolvers and data-fetchers for each transformation explicitly, there is little meaning to our efforts.

Therefor gintonic supports transforming executable operation definitions, such as operation and fragment definitions, in addition to type-system definitions (schema and type-definitions). Given a source schema and a transformation we can generate a target schema, and given an operation on that target schema, we may generate the corresponding operation on the source schema. Not only that, we may leverage the powerful field-aliasing feature of GraphQL to get output that is structurally valid for the target schema, thus making field resolving trivial. All except for internal fields and types, which should always be handled by the target API.

In the following example, we transform a schema by aliasing two fields and locking an argument.

Source schema.

Transformation.

Target schema.

When we receive an operation against the target schema above, e.g., the following mutation:

Operation on target schema.

We can automatically transform it to the following operation:

Operation on source schema.

Notice how aliasing is used to ensure that the resulting data structure is valid output for the original mutation.

Explore the language at our demo page: mito.ai/gintonic/demo.

Deployment

I’ve now hopefully convinced you that we can transform a schema, and have a decent method for handling operations on that transformed schema. We now need some way to host this endpoint and start accepting operations, since GraphQL is transport independent, gintonic, is as well, but lets for the remainder of this post assume that GraphQL APIs are served over HTTP accepting POST and GET requests, as outlined on the GraphQL website. For this case, we provide a koa-middleware which enables you to start up a target server and start accepting requests quickly.

This middleware is available on npm, and a simple target server may look something like this:

A simple Koa-server running gintonic.

Here all we need to specify is the URL of the source server, the upstream-URL, and the transformation that we want to perform. The middleware will then fetch the schema of the upstream server, by introspection, perform the transformation, and start accepting requests.

Typically you might want to provide some custom method for fetching from your upstream API, e.g., for authentication purposes. By providing a fetcher as in the argument to the middleware, you can do just that. A fetcher is here a function which takes query string, variables object, operationName , and koa-context as arguments, and produces a Promise containing the result. Which may look something like this:

Koa-server running gintonic with custom fetcher.

In this example, we provide an API key for the upstream server effectively making a subset of the source API public to anyone who has network access to our server. This pattern might be useful when you on your internal company network don’t want API-keys shared throughout the system or if you want to make a subset of your internal API available to the public.

Since the fetcher takes the koa-context as an argument, it has access to the current request and response objects, so implementing an authentication exchange becomes trivial. You may choose to provide an API-key against the source API if and only if the user of the target API has valid credentials, thus in effect propagating the authentication rules of the source server. At Mito.ai we use this pattern to ensure that a customer with their API, can't access the source API directly, or the API of another customer. Using JWT (powered by Auth0) and having a different audience claim for each customer, implementing this becomes trivial.

Deploying a target server might seem overwhelming, especially if you want to do it dynamically. We have automated this process on kubernetes with Helm, but if that seems too complicated for your use case, you can deploy it on the serverless infrastructure of your choice. The following example shows how you may convert the koa-middleware to an express middleware and then use it directly with Google Cloud Functions, deployed with serverless.

Serverless deployment of gintonic on Google Cloud Functions with serverless.

In this case, we have hard-coded the transformation and upstream URL. These could naturally be provided by serverless variables which will allow you to reuse the logic above, only requiring your team-members to supply their transformations to start deploying customer specific APIs. The same options that were presented in the koa example above are of course also available in the serverless environment. You may supply a custom fetcher and do authentication exchange just as in the previous examples.

Conclusion

Developing the gintonic language has been extremely exciting, and we have high hopes for how this can help us build a more secure and scalable platform in an intuitive and declarative manner. The language is naturally not ideal for every use-case, sometimes you need a single API, and you should, as always when designing your infrastructure, think long and hard about which technologies and levels of complexion that you introduce.

However, even if one-size does fit all in your case, I hope this project goes to show the awesome powers of having a strongly typed API and illustrates how we might use DSLs to reduce infrastructural complexity, heighten maintainability, and improve security. If you do decide to use gintonic, please let us know and feel free to contribute to the repository!

We are planning on adding new language features on demand, and will gladly consider any requests from the community.