I’d heard of functional programming of course; even tried it in a few languages. I’ve adopted many features such as map, curry, list comprehensions, but when I need to get stuff done — something that someone actually pays me to deliver — I would fall back to my imperative ways.

Then came my most recent project — a webhook delivery system that is supposed to be both scalable and transparent. It’s fun to look at the code base and see that everything is a function, except a few struct types. Go does not style itself as a functional language, yet when all is said and done, there are closures everywhere and only has one lonely global variable, an os.Stderr logger for fatal crashes.

So how did I get there? It started with pure laziness, the source of most good things in computer programming. I knew I would need unit tests everywhere as I’d be building it to handle heavily concurrent code. At best, it would be difficult to build, impossible at worst and no fun at all in any case. I started with abstraction. I realized most of the tedious work can be hidden behind two function definitions:

type splitFunc func(*Webhook) (bool, error) type couplingFunc func(*Webhook)

splitFunc receives a pointer to Webhook and returns true/false or an error if there was trouble (there is always trouble). It is used to direct the flow of hooks (uses include validation or de-duplication of webhooks).

//isResourceChanged implements splitFunc

func isResourceChanged(hook *Webhook) (bool, error){

//download resource from API and figure out if it has changed since the last time we downloaded it

//return true if yes; false otherwise

//return non nil error if there was trouble accessing the API

}

couplingFunc is even less ambitious, it receives a pointer to Webhook and presumably does something with it. It handles logging, rate limiting, etc.

//simpleLog implements couplingFunc

func simpleLog(hook *Webhook){

log.Printf("VALID HOOK %s %s", hook.Resource, Hook.ID)

}

From the point of unit tests I was able to mock splitFunc to randomly return true/false or an error, and a fake couplingFunc that does nothing at all.

This is great because it allows me to realistically model and test the flow of delivery, without having to write any of the tedious code. When the time does comes to write the actual tedious pieces they only have to fit the above definitions. It does not matter how complicated our CDN refresh process is; it just has to fit into our splitFunc function. For this to work, functions must be first class citizens in the language of choice. Fortunately this is the case in many languages these days (Go, Python, Javascript, etc). This makes it easier to write unit tests, provides a clear separation of concerns, and makes the code easier to understand and maintain.

However, trouble always rears its ugly head. The file logging function needs to share a file handle, the dispatcher needs an SQL connection and we need to know where to send hooks. It would be wasteful or impossible to make each function invocation create its own resource, so some things would need to be shared. First, I tried to create a shared resource such as SQL connection in the global scope for the functions to use. The drawback here became quickly apparent in the unit tests. Who initializes the SQL connection? Who and when will shut it down properly?

The next thing I tried were closures. Since functions can return functions, I created a factory that returns a splitFunc closure that shares the SQL connection (or pool) among all of its invocations. I still need to shut down the SQL connection but thankfully, Go allows multiple return values. From my factory function I returned the splitFunc and a cleanup closure. The caller does not actually need to know what’s being cleaned up, it just needs to know that something does.

I was able to hide the complex functionality behind this seemingly simple interface. In the whole code base there’s only three spots that know about the SQL connection: the factory function that created it, the close function that will close it when called, and the splitFunc closure that uses the SQL connection. The close function proved a godsend.

//closures to the rescue

type closeFunc func()

db, err := sql.Open(driverName, dataSourceName

splitFuncThatUsesSQL := func(hook *Webhook) (bool, error){

// do something here with the webhook and SQL

// e.g. see if anyone is subscribed to receive the hook

return true, nil

}

closeFuncThatClosesDBConn := func(){

db.Close()

} func pgLogFunctionFactory(pgUrl string) (splitFunc, closeFunc) {db, err := sql.Open(driverName, dataSourceName string splitFuncThatUsesSQL := func(hook *Webhook) (bool, error){// do something here with the webhook and SQL// e.g. see if anyone is subscribed to receive the hookreturn true, nilcloseFuncThatClosesDBConn := func(){db.Close() return splitFuncThatUsesSQL, closeFuncThatClosesDBConn

}

Say I want to shut down the SQL connection after our SQL aware splitFunc is finished. Having a unified type for these functions ensures that it’s trivial to ensure the shutdown happens in the correct order.

There are many other unexpected benefits too. I originally logged all the webhook delivery events to a file. Later I was able to add logging back-ends for both Postgres and InfluxDB for easier future lookups and metrics. The only thing the logging function needed to know was that there’s a factory function that, when given a name of event, returns a couplingFunc. All three of these back-ends, while differing internally, return the same function. Thanks to function composition it was easy to write a logging function factory that accepts an arbitrary number of logging functions and returns a logging function. This logging function will call all its constituents in turn.

//merge several logFuncs into one

//this allows us to have several logging backends

//while having only one logging function

func mergeLogFuncs(logFuncs ...couplingFunc) couplingFunc {

return func(hook *Webhook) {

for _, f := range logFuncs {

f(hook)

}

}

}

I found closures a much nicer alternative to more familiar approaches, such as globally shared resources, or or the classic OOP approaches. Having no shared state is more readable, modular, and testable than anything I’ve written in the past. To sum it up, it’s just more functional.