Our curvy road to Clojure

The Clojure language is spreading like wildfire at Unbounce. We just had three people attending Clojure West and being indoctrinated with the latest and greatest news about this language and its ecosystem. So we’re very serious about it! But this commitment didn’t happen overnight: it took us a while and some experiments (at times painful) to come to the conclusion that Clojure was the right tool for us.

Historically Unbounce started as a Ruby and Java shop, but it would be a mistake to reduce us to these two languages. We are all polyglot programmers at heart, and very curious ones on top that! Though there are some benefits in having two heterogeneous platforms and environments (like for example from a security standpoint), it also has drawbacks: when nothing can be shared between platforms, everything has to be done twice in two different ways, and this from coding to deployment.

So we’ve been looking for ways to reduce this divide and increase collaboration and sharing across our teams. To that end, we’ve explored using Vert.x as a common platform, since it is a polyglot platform on the JVM, with many language bindings (including Ruby). The module system of Vert.x 2 was particularly attracting to us, as it allows composing applications from multiple independent modules written in any languages and communicating with each other via an event bus (thus low coupling). Vert.x clean model for worker components was also very attractive to us, as it solves one of the common headaches with Ruby applications. Finally the fact that Vert.x has all its networking built on top of Netty was also a major decision point for us, since we had a great experience building our new Page Server on this solid library.

Unfortunately, after months of efforts (leading to one application in production and several in development), this approach didn’t work out for us and failed to deliver what we were expecting from it. Let me make it very clear that this was not a Vert.x issue per se, but more a problem with its ancillary tools:

Despite ruby-maven’s heroic efforts to give a Ruby flavour to the Java tool chain, the development process was slow and cumbersome. Even after hiding most of the build complexity in a Maven parent POM, our developers had to go through long build processes to bring all the necessary Gems, JARs and modules in order to prepare the application for startup, which was very frustrating.

Despite ruby-maven’s heroic efforts to give a Ruby flavour to the Java tool chain, the development process was slow and cumbersome. Even after hiding most of the build complexity in a Maven parent POM, our developers had to go through long build processes to bring all the necessary Gems, JARs and modules in order to prepare the application for startup, which was very frustrating. Some Gems were not thread safe and, thus, were causing issues when used in the multi-threaded context of the JVM. We’ve started to look at JAR equivalents of these Gems, since jRuby has a great interoperability with Java, and had some positive results but at the expense of making the development experience less familiar for Ruby developers.

jRuby performances were pretty bad, consistently one order of magnitude slower than MRI or the equivalent Java implementation. We’ve tried the recommended performance tweaks but to no avail. It’s very possible that with more efforts or maybe by using the upcoming version of jRuby, these problems could have been solved, but for us that was the last nail in the coffin for this attempt.

At this point, we were still very keen on using Vert.x so we’ve started to investigate other languages it supports, trying to find one that could appeal to both our Ruby and Java developers. Initially we thought that Scala was the most promising language because of its dual nature (object oriented and functional): our Ruby and Java developers would not have to give up on objects while gaining the benefits of functional programming.

But then something happened, something as beautiful as a celestial shower of rainbow-coloured parenthesis: our Rubyists expressed a strong interest in Clojure.

It was a surprise: letting go of objects was an option? But what a great surprise! Indeed, Clojure bindings for Vert.x are excellent. So we tried porting a Vert.x Ruby application to its Clojure equivalent and the results were great: much less code, a cleaner application structure and significantly improved performances.

But then something else happened. In the process of migrating to Clojure on Vert.x, we realized the following:

We were not using much of Vert.x core features,

We were actually fighting with the way Clojure verticles were loaded and isolated,

We were (again) suffering from using the Java toolchain for building a Clojure application as a Vert.x module.

Therefore, we ended up taking the drastic decision of letting go of Vert.x while keeping Clojure. For us, the wins with Clojure are plenty:

The language is concise but explicit (no AOP magic nor monkey patch madness), functional and immutable, and comes with perfectly crafted core libraries – simple made easy!

Access to a lot of excellent open source libraries and all the Java ecosystem as well, thanks to a great interoperability with Java,

We can re-use our common code library that is used extensively by our Java and Scala projects: having access to our own battle hardened code is invaluable,

Excellent tooling, like the Leiningen build tool, and editor support for everyone (Emacs, vim, IntelliJ and Eclipse).

So were does all this leave us? In a pretty great position, considering:

Clojure wrappers around our common Java code have been written for the major features we needed (configuration loading, logging, error reporting, AWS integration…),

We’ve open sourced a few Clojure projects,

One Clojure application has made its way to production and is already dealing with all our public API traffic. More applications are coming soon!

Our road to Clojure was curvy but, in hindsight, we couldn’t have come to this ending without having first given our best shot at a polyglot platform. The reality of unified polyglot development is that it’s hard because programming involves way more than just a language: build tools, libraries, application design and architecture are hard, if not impossible, to harmonize. Picking up a single language and its ecosystem is a more pragmatic approach, and the one that will lead to the best development experience.

We’ve recently set afoot our initiative towards a microservice architecture. Though it won’t be the only tool in our toolbox, Clojure play a prominent role at Unbounce in this effort to rebuild our infrastructure in a more modular and less coupled fashion.

David Dossot

Director of Software Architecture