In an attempt to create a more singular description of our data, we will move the specs into the Datascript schema. This won't actually work, but it'll define a goal:

( def schema { :person/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) } :person/name { :schema/spec string? } :person/age { :schema/spec ::number } :person/entity { :schema/spec ( s/keys :req [ :person/id :person/name :person/age ] ) } :movie/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) } :movie/title { :schema/spec string? } :movie/description { :schema/spec string? } :movie/release-date { :schema/spec ::number } :movie/people { :db/valueType :db.type/ref :db/cardinality :db.cardinality/many :schema/spec ( s/coll-of :person/entity ) } :movie/entity { :schema/spec ( s/keys :req [ :movie/id :movie/title :movie/description :movie/release-date :movie/people ] ) } } )

schema is a custom namespace that I just introduced. We'll make use of it shortly. Because the schema now has extraneous junk in it, Datascript will no longer eat it raw. We'll need a function that turns this back into a pure Datascript schema. Since we'll need a supporting function anyway, let's see if we can make some more improvements while we're at it.

When we reviewed our original specs, we concluded that they described everything about our data except for uniqueness constraints. It just so happens that spec has APIs to work with defined specs as data, allowing us to extract e.g. keys from a (s/keys) spec and more. That means we no longer need :db/valueType or :db/cardinality - the former can be induced from (s/keys) specs (those will be references) and the latter from (s/coll-of) specs (collection - :db.cardinality/many ).

This leaves us with this leaner representation:

( def schema { :person/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) } :person/name { :schema/spec string? } :person/age { :schema/spec ::number } :person/entity { :schema/spec ( s/keys :req [ :person/id :person/name :person/age ] ) } :movie/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) } :movie/title { :schema/spec string? } :movie/description { :schema/spec string? } :movie/release-date { :schema/spec ::number } :movie/people { :schema/spec ( s/coll-of :person/entity ) } :movie/entity { :schema/spec ( s/keys :req [ :movie/id :movie/title :movie/description :movie/release-date :movie/people ] ) } } )

Before we can extract the Datascript schema, we'll need to define the specs so we can mine them for data. cljs.spec.alpha/def is a macro, and in order to call it correctly on behalf of the schema definition, we need a macro to define the schema as well.

The macro goes into unified-schema/macros.cljc :

( ns unified-schema.macros # ? ( :cljs ( :require [ cljs.spec.alpha ] ) ) ) ( defmacro defschema [ name schema ] ( apply list ' do ( concat ( for [ [ attr attr-def ] schema ] ` ( cljs.spec.alpha/def ~ attr ~ ( :schema/spec attr-def ) ) ) ` [ ( def ~ name ~ schema ) ] ) ) )

...and can be used like so:

(ns unified-schema.example (:require [unified-schema.macros :refer-macros [defschema]]) (defschema example-schema {:person/id {:db/unique :db.unique/identity :schema/spec (s/conformer uuid)} :person/name {:schema/spec string?} :person/age {:schema/spec ::number} :person/entity {:schema/spec (s/keys :req [:person/id :person/name :person/age])} :movie/id {:db/unique :db.unique/identity :schema/spec (s/conformer uuid)} :movie/title {:schema/spec string?} :movie/description {:schema/spec string?} :movie/release-date {:schema/spec ::number} :movie/people {:schema/spec (s/coll-of :person/entity)} :movie/entity {:schema/spec (s/keys :req [:movie/id :movie/title :movie/description :movie/release-date :movie/people])}})

Now the specs will be defined, and example-schema will refer to our schema data.

To extract the schema, we'll start by simply preserving all the keys in the db namespace:

(defn select-namespaced-keys [m ns] (->> (keys m) (filter #(= (namespace %) ns)) (select-keys m))) (defn extract-schema [attributes] (->> attributes (map (fn [[k v]] [k (select-namespaced-keys v "db"])) (into {})))

We will now add :db.cardinality/many to any attribute that has a s/coll-of spec, and :db.type/ref to any attribute that has a s/keys spec. You can inspect the underlying data structure of a spec with s/form :

( s/form :person/entity )

To devise a generalized solution, we'd also like to support this spec:

( s/def :person/entity ( s/and ( s/keys :req [ :user/name ] ) ( s/or :email ( s/keys :req [ :user/email ] ) :phone ( s/keys :req [ :user/tel ] ) ) ) ) ( s/form :person/entity )

To work with this data, I wrote two helper functions (see all the way to the bottom if you're interested):

coll-of , which returns the first s/coll-of spec

, which returns the first spec specced-keys , which returns a set of all keys, required and optional, for a map spec ( #{:user/name :user/email :user/tel} in the above example).

Using these two functions, we can add the cardinality and ref types to the relevant attributes:

(defn- schema-attrs [attributes attr-key attr-def] (let [coll-type (coll-of attr-key)] (-> (select-namespaced-keys attributes "db") (assoc-non-nil :db/cardinality (when coll-type :db.cardinality/many)) (assoc-non-nil :db/valueType (when (or (seq (specced-keys attr-key)) (seq (specced-keys coll-type))) :db.type/ref))))) (defn extract-schema [attributes] (->> attributes (map (fn [[k v]] [k (schema-attrs attributes k v])) (into {})))

Finally, we'll evict all attribute definitions that don't have any descriptors in the db namespace:

( defn extract-schema [ attributes ] ( let [ schema ( ->> attributes ( map ( fn [ [ k v ] ] [ k ( schema-attrs attributes k v ) ] ) ) ( into { } ) ) ] ( ->> ( keys attributes ) ( filter # ( not ( seq ( select-namespaced-keys ( % attributes ) "db" ) ) ) ) ( apply dissoc schema ) ) ) )

We've unified the Datascript schema and specs. What about the mapping from API data to schema data? It would be neat if we could achieve that declaratively as well. In this particular case the mapping was quite straight forward: attributes have different names, and we want to pass values through our specs. The declarative bit of the solution could look like this:

( defschema example-schema { :person/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) :schema/source :id } :person/name { :schema/spec string? :schema/source :name } :person/age { :schema/spec ::number :schema/source :age } :person/entity { :schema/spec ( s/keys :req [ :person/id :person/name :person/age ] ) } :movie/id { :db/unique :db.unique/identity :schema/spec ( s/conformer uuid ) :schema/source :id } :movie/title { :schema/spec string? :schema/source :title } :movie/description { :schema/spec string? :schema/source :description } :movie/release-date { :schema/spec ::number :schema/source :release_date } :movie/people { :schema/spec ( s/coll-of :person/entity ) :schema/source :people } :movie/entity { :schema/spec ( s/keys :req [ :movie/id :movie/title :movie/description :movie/release-date :movie/people ] ) } } )

Frequently, your schema will contain namespaced versions of API data keys (e.g. :person/name vs "name" ). Because this particular mapping is so common, we'll just bolt it into the converter, and can leave them out of our schema.

The implementation of convert-data looks like this:

( defn convert-data [ attributes api-data key ] ( let [ { :keys [ schema/source ] } ( attributes key ) data ( if source ( get api-data source ) ( get api-data key ( get api-data ( keyword ( name key ) ) api-data ) ) ) collection-type ( coll-of key ) keys ( specced-keys key ) ] ( cond ( seq keys ) ( ->> keys ( map ( fn [ k ] [ k ( convert-data attributes api-data k ) ] ) ) ( into { } ) ) collection-type ( map # ( convert-data attributes % collection-type ) api-data ) :default ( if ( s/valid? key api-data ) ( s/conform key api-data ) ( s/assert key api-data ) ) ) ) )