The introduction of statically typed languages for client-side programming, such as TypeScript, has greatly boosted the quality of creation of large client-side applications such as commonly seen when building a modern SPA.

This increase in safety only extends up to the boundary of API calls. Results from a remote call are always returned as unstructured JSON, or any in TypeScript. Deserialisation is thus done manually, a tedious and error prone process. Moreover, deserialisation routines suffer from various forms of bit-rotting: as the underlying model evolves, its validation might become obsolete or, even worse, slightly off. This results in false negatives (the data is correct, but the deserializer thinks there is a mistake), or false positives (the wrong data is simply cast to the right model type, wreaking havoc in the rest of the application).

In short, client-side deserialisation is, mildly put, not a lot of fun.

The ever growing adoption of rich protocols such as OData (or its recent, hipster variant of GraphQL) make querying from the client side much more efficient: the client can reduce the number of received attributes ( $select ), rows ( $filter ), and even overall number of queries by requesting joined data ( $expand ). Unfortunately, this makes the received data even more complex to parse, adding a dynamic dimension to the problem that really does not help.

In this article, we will study the advanced typing mechanisms of TypeScript that make it possible to automatically generate a type-safe deserializer, based on a declarative description of the queried data constructed as an object.

Advanced types

TypeScript supports a very rich type language that allows the creation of types at compile time. The resulting types are created by assembling two (or more) existing types, by using operators resembling sum and product. This process is the abstract, type-level equivalent of creating numbers from other numbers ( 5+3 can indeed be seen as a way to create a number, 8 , by composing two numbers: 5 and 3 ).

Creating a new type from two existing types can be done with the | and & operators. a & b will create a type with all the fields of a , and all the fields of b . a | b will create a polymorphic type which contains either the fields of a , or the fields of b . Thus, for example:

{ name:string, surname:string } & { age:number }

is the same as type:

{ name:string, surname:string, age:number }

whereas:

type Car = { kind:"electric", num_batteries:number }

| { kind:"petrol", engine_size:number }

Will accept values matching either of the two shapes completely, and thus never tolerating a petrol car with a num_batteries .

Moreover, TypeScript also supports more special types: keyof t is the type of all field names of type t . Thus:

keyof { kind:"petrol", engine_size:number }

will be:

"kind" | "engine_size"

The final, special type that TypeScript supports is t[k] , provided that k extends keyof t . t[k] is the type of field k within type t . For example:

{ id:number, name:string, surname:string, age:number }["age"]

will, simply, be string .

There are other relevant types and type operators in TypeScript, but for the purposes of this article we do not need them all.

A query interface

By using these advanced types, we can build a simple query definition interface. For the limited scope of this article, we will only support a query definition language that only allows us to specify which fields we want from a given data source, plus some basic filtering. Management of references and joins can also be added as a relatively simple extension to the basis we setup in this article, but we will not see it here.

The query class will be generic in two types: the type of the data source, and the type of the data we would expect as a result from running the query. The query definition will also contain a deserialize function that turns an unstructured any into a result :

export class Query<result,source> {

deserialize : (res:any) => result ... constructor(deserialize : (res:any) => result) {

this.deserialize = deserialize

}

}

The starting point for any query can be handily encapsulated by a helper function which assumes that any entity we query will have, at the very least, an id field:

export let entity_query = <t extends { id:number }>() =>

new Query<{ id:t["id"] }, t>(

res => "id" in res && typeof res["id"] === "number" ?

({ id:res["id"] as number })

: fail(`Error: expected id in ${JSON.stringify(res)}`)

)

Notice that entity_query requires a single generic argument, t , which is expected to extend from { id:number } (that means that t must have a field id of type number , and not that t must necessarily explicitly inherit the extended type). The generated query will then produce a result of type { id:t[“id”] } . Deserialization in this case will simply check whether or not the received blob has the expected field id of type number . If this is the case, then we can produce the required object, otherwise we must fail with an exception (or a type safe alternative such as Option ).

Let us add some methods to our Query class. Since we want our queries to support looking up fields from a remote source, we could add a with_field method. This method will accept the field as a generic parameter which must be any of the keys of the source type: it would make no sense to lookup a field which is not present in the source!

readonly with_field = <k extends keyof source>(k:k) : ...

with_field will return a new query with the very same result as the original query, including field k as it was present in the source . The created query is, of course, still based on the original source . It type is thus:

Query<result & { [f in k]:source[k] }, source>

The type result & { [f in k]:source[k] } includes all fields that we found in result , plus a new field [f in k] with the type source[k] of k in source .

Deserialization in the new query is based on deserialization of the previous query, followed by addition of the new field k to the deserialized result:

(res:any) => {

let obj = this.deserialize(res) as any

if (!(k in res)) throw new TypeError(`Error: missing key ${k} in ${JSON.stringify(res)}`)

obj[k] = res[k]

return obj as result & { [f in k]:source[k] }

}

Thanks to this method, we can now create queries that include only the desired fields of a given type:

interface Person {

id:number,

name:string,

surname:string,

nationality:string,

age:number }

let name_age_query = entity_query<Person>()

.with_field("name")

.with_field("age") let res = name_age_query.deserialize({ "id":1, "name":"John", "age":23 })

The deserialized result will take type:

let res: {

id: number;

} & {

name: string;

} & {

age: number;

}

meaning that attempting something like res.surname will result in a compiler error, instead of being caught somewhere in the run-time. Moreover, if we attempt to create a query that includes a non-existing field, for example by adding .with_field(“favorite_food”) to our query, we will also get a compiler error right away, as favorite_food is not a valid field of Person . This has the potential to greatly reduce serialization-related bugs!

We could extend our query system to also support comparison queries. For example, we could include a type-safe filtering method that specifies that we want to filter based on equality of an attribute and a given value, thereby validating that the value always has the same type as the attribute it is compared to:

readonly filter_eq = <k extends keyof result>(k:k, v:result[k]) : Query<result, source> => {

return new Query(this.deserialize)

}

Notice that the only real constraints here are that k must be a valid property of the result , and that the comparison value v must have the same type of that property. Now we can write, for example:

let q = entity_query<Person>()

.with_field("name")

.with_field("age")

.filter_eq("name", "Johnny")

Attempting either .filter_eq(“name”, 3) or .filter_eq(“City”, “Rotterdam”) will both produce compiler errors: 3 is not compatible with string , whereas City is not a field of Person . Again, this will ensure, without even having to run the code, that our queries are well-formed.

Extensions

The code we have seen so far is relatively simple, and only covers the specification of queries and the generation of the deserialization function based on a query. It is possible to further extend this framework in order to include more complex query operations such as sorting, pagination, but more importantly lookup of relations (especially one-to-many). Moreover, we could even take this framework a step further, and generate the query itself (be it OData or GraphQL or something else) from the query specification. This might make for an interesting sequel to this article :)

Psst!

Are you an ambitious developer? Did you like the article? Are you looking for a company where other software engineers work at a high level, by applying functional programming and type theory concepts to build beautiful and reliable online software? Then look no further: at Hoppinger, in the awesome city of Rotterdam, we have multiple open positions! We accept candidates at all levels: from veterans to eager-to-learn juniors.

Code dump — for reference