Development and Structure Factors

12 Factor clearly states that configuration should live in environment variables:

“The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.”

Makes sense. But for me:

* it makes development harder

* it is simply not needed for certain apps I work on

* environment variables lack structure

But it does not mean nothing can be done to have both: 12 factor and the solution for these three points above. We are programmers, we create things.

The Golden Goose

I’ve done quite a few apps in Java over the last decade, and I know a thing or two on how to bridge the gap between applications and configurations.

In Clojure universe, the most popular library to manage configs is environ. Normally apps would have their properties live in lein profiles / .lein-env / .boot-env and could be overridden by environment variables and system properties. Currently this is the way to manage configs in Clojure.

Things I find lacking are:

1. Ultimately configuration is solely based on ENV variables (+ some system props) exported as individual properties: 100 properties? 100 env variables exported..

2. It allows only string values: no data structures, no numbers, etc.

3. It allows no structure / hierarchy, just one (top) level pile of properties

4. Ironically :) it keeps a single global internal config state, which makes it hard to have app (sub) modules with separate configs

This is not to say environ got it wrong, or it is no good, it is really useful and works great for a lot of people, but I feel I need an alternative, because of the 4 things above, and simply because I like to have alternatives. So I wrote one.

12 Factor + Some Love

My take on configuration and properties is called cprop. It is young and brave, and currently is used by me in several applications, a few others, and it is a default configuration approach in Luminus which is a micro-framework that is based on a set of lightweight libraries.

Instead of repeating cprop documentation, it would be important to notice that the factor number III out of 12 factors, which is called “Config“, is fully supported by cprop, while still allowing for more flexibility.

For example let’s take a concept of “structure” or “hierarchy”. Say you have a config in development (i.e. somewhere in dev-resources):

{ :datomic { :url "CHANGE ME" } , :aws { :access-key "AND ME" , :secret-key "ME TOO" , :region "FILL ME IN AS WELL" , :visiblity-timeout-sec 30 , :max-conn 50 , :queue "cprop-dev" } , :io { :http { :pool { :socket-timeout 600000 , :conn-timeout :I-SHOULD-BE-A-NUMBER , :conn-req-timeout 600000 , :max-total 200 , :max-per-route :ME-ALSO } } } , :other-things [ "I am a vector and also like to play the substitute game" ] } {:datomic {:url "CHANGE ME"}, :aws {:access-key "AND ME", :secret-key "ME TOO", :region "FILL ME IN AS WELL", :visiblity-timeout-sec 30, :max-conn 50, :queue "cprop-dev"}, :io {:http {:pool {:socket-timeout 600000, :conn-timeout :I-SHOULD-BE-A-NUMBER, :conn-req-timeout 600000, :max-total 200, :max-per-route :ME-ALSO}}}, :other-things ["I am a vector and also like to play the substitute game"]}

Looks like it passes the 12 factor litmus test which says that “the codebase could be made open source at any moment, without compromising any credentials” while preserving the structure.

There are several ways to provide real values for this config with cprop, such as other maps merged at runtime, system properties, runtime arguments, etc.. but the most interesting way, from the 12 factor point of view, is “environment variables”. Which can be simply done with cprop as:

export AWS__ACCESS_KEY =AKIAIOSFODNN7EXAMPLE export AWS__SECRET_KEY =wJalrXUtnFEMI / K7MDENG / bPxRfiCYEXAMPLEKEY export AWS__REGION =us-east- 1 export IO__HTTP__POOL__CONN_TIMEOUT = 60000 export IO__HTTP__POOL__MAX_PER_ROUTE = 10 export OTHER__THINGS = '[1 2 3 "42"]' export AWS__ACCESS_KEY=AKIAIOSFODNN7EXAMPLE export AWS__SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY export AWS__REGION=us-east-1 export IO__HTTP__POOL__CONN_TIMEOUT=60000 export IO__HTTP__POOL__MAX_PER_ROUTE=10 export OTHER__THINGS='[1 2 3 "42"]'

Check out the documentation, there are several more useful concepts that cprop brings to life.

I love bringing things to life.

Tags: clojure, cprop