Jon Pither is a development team leader by trade having led projects for JUXT at multiple Tier-1 Investment Banks, an online newspaper website, a major property portal, and an international public electric bikes scheme. Jon is a co-founder of JUXT.

We needed to get moving quickly

Corporate Chat

Yapster is a corporate chat product, along the lines of WhatsApp and Slack. It's different to either in that it ingests HR and payroll data and allows messaging based on all the entities extracted, including people, locations, areas, job functions and intersections of the entities.

Some of the biggest names in retail and hospitality, including Ann Summers, Caffe Nero, SuperDry, Ryman and Wyevale Garden Centres are working with Yapster to bring the technology to their workforces. Yapster’s customers benefit from being able to ask the development team for custom features, and for the ability to integrate with their existing systems, such as HR and finance databases.

Background

Jon: What's the background to Yapster?

Craig: The idea for the product that would eventually become Yapster was dreamt up in the pub back in 2015. We knew we wanted to do something in mobile and Rob had been thinking a lot about how we could help large companies to become less dysfunctional and more efficient.

This got us thinking about enterprise-level mobile messaging. The likes of Yammer, Slack and Facebook Workplace were already established, but there was a lightbulb moment when a senior contact in one of the world’s largest retailers pointed out that there are millions of employees in service industries that never use a desktop computer. At this point the target audience for Yapster became clear. We started out thinking of applications you could build on top of basic chat, such as time management and shift-swapping applications. We did some prototypes, but actually found that we got good traction with just our chat product. Now we're supporting 'apps layered on chat' such as time-management and help-desk integration, using our API for integration.

We started cutting code 2 years ago.

The Tech

Jon: What's your tech stack?

Craig: I set out to build something that would scale well. We started with Cassandra and Kafka for persistent pub/sub. The application itself is written in Clojure and ClojureScript.

The backend API is stream processing and uses a web framework called Yada and Onyx for stream processing.

For native apps we deploy HTML/CSS/JavaScript via Cordova.

Jon: What about React Native?

Craig: If we were starting again, then yes, but it was a beta for Android back then and it didn't support web.

Cordova has been good; there aren't many areas where it's obvious that it's a webapp vs a native app. Since we use a ClojureScript stack consisting of re-frame and Reagent, there is the real possibility of a relatively sane migration path to React native. We may even do it.

Jon: How about re-frame?

Craig: It answers the problem of state management, with which Reagent gives you little help. With Reagent it's possible to get yourself into a tangle, with lots of atoms spread around your views. The re-frame model is very simple and we've had a very easy time with it.

Jon: Onyx?

Craig: We don't use many of the more advanced features such as streaming aggregations. We mainly use it for pulling data out of Kafka and feeding our message routing and notification pipelines (for example pushing to mobile devices via websockets). The flow-conditions concept makes it very easy to build tasks which generate effect data structures. Unlike Storm it doesn't tie you to a particular Clojure version. With Onyx it's trivial to add more tasks, and it manages error flows and reprocessing in the case of temporary failures.

Jon: Pretend I'm not from JUXT, what's your opinion of Yada?

Craig: Great actually. We wanted a fully asynchronous non-blocking back end. If you can climb the hill of handling async requests then it becomes simpler after, where back-pressure and streams are taken care of. All of our back end is non-blocking, we also use Manifold because of Aleph (which Yada is built on).

Also, Cats has worked out brilliantly for structuring our async code. Cats code reads largely like sequential code. So far it hasn't been hard to get new devs to work on it. Though if you stick them in front of the Wikipedia page on what is a monad, then it might not work so well!

A while ago we excised all of the core.async code in the backend in favour of Manifold and Cats. We use Alia as our fundamental Cassandra interaction library which wraps the Datastax Java client. It's well put together, handling the async side very well.

We have a library er-cassandra that returns high level Clojure maps to us from Cassandra, and looks after the denormalisation that Cassandra models generally require.

Clojure

Jon: Why choose Clojure in the first place?

Craig: Familiarity; we needed to get moving quickly. I had spent the previous two years working with Clojure and enjoyed the experience.

I've probably written a few hundred thousand lines of Clojure in the last few years, I've never got to the point when I needed to start again with a code-base.

Jon: The Lisp bottom-up programming experience..?

Craig: Yes, Clojure is great for growing a system from a small prototype to a full production system. It's never a problem to incrementally add functions.

I like Lisp. Things that require compiler mods in other languages, you can do with macros in Lisp. The profusion of monadic libs such as Cats, is great.

Jon: Any reservations?

Craig: I'm not sure Clojure will be the last language I'll ever use. Lux is exciting me currently; it's a Haskell in Lisp form that runs on the JVM. It has static type checking where all the type information is just a data-structure. The macros are monadic and have access to the full compiler state. It's really quite beautiful.

We also get a number of dynamic type errors which would be avoidable in a world of static type checking. The dynamic approaches of spec and schema aren't really going to cut it; helpful sure, and generative testing support is nice, but there are still lots of silly errors such as not catching the missing branch of a case statement when the data model has changed.

On the other hand, it's great with Clojure that code is shared between front end and back end - this avoids a whole class of bugs around sharing messages on the wire.

Hiring?

Jon: How have you found hiring Clojure devs?

Craig: It's just good enough. I've hired mostly through the Clojurians slack, a little bit through LinkedIn and my network. We haven't had huge numbers of respondees but we've had good-quality candidates to fit the role. We avoid recruitment consultants.

Jon: Are you hiring now?

Craig: Currently we've two devs and a UI person. The devs are full stack which is so far working out really well. I am always on the lookout for excellent Clojure developers to help me and my dev team keep up with rising customer demand and make every new version of Yapster the best yet!

Jon: Full stack is a lot fun.

Craig: Yeah I did some swipe gesture handling in the application - the instantaneous nature of development and playing with it was great fun.

Jon: How much of your work is local vs remote?

Craig: We are mostly remote, but we do have office days. If people are interested and intrinsically motivated, then it's good to work from home, as there can be fewer interruptions.

And of course, we have our own messaging product for communication.

State of Clojure

Jon: What's your view on the state of Clojure in London?

Craig: Hmm, not sure. It's not sufficiently popular that there's so much noise, but there is a significant number of significant companies doing significant things.

Jon: Sounds significant. Any last words?

Craig: I'm maybe a bit jaded with Clojure now, but I wouldn't have done things any differently.