TL;DR: Copy and paste the gists below to add application monitoring to your Phoenix app. It will look something like this:

Edited July 11, 2015

Updated the plug gist (fixed typo) and repo gist (adjusted to the latest ecto version).

Prologue

As lots of engineers in the Elixir community I have a ruby background. Rails used to be my go-to MVC framework.

About half a year ago I started using the amazing Phoenix Elixir web framework. It allows you to build super fast web applications. There is support for web sockets out of the box, including native libraries. And it is so well designed that you are super productive using it.

Yet there is one big drawback when compared to rails: No SaaS monitoring tools. All my rails apps have either New Relic or skylight.io metrics. Both of these don’t work with Erlang or Elixir (yet).

This is a big problem: Metrics are extremely important. In case something goes wrong they are my first stop for finding out where the problem may lie. Additionally they let me measure how fast I respond to requests. That is important to maintain a high quality. The most important application is that they give me an idea of my apps health. Good metrics can prevent problems or even downtimes by being an early warning system. Without metrics you are flying blind.

Custom monitoring for Phoenix

Thankfully I found the ideal replacement for the SaaS tools I was using with rails. The powerful combination of Exometer, StatsD and DataDog. Let me explain each part of the system.

Exometer

Last December I was in Berlin for Erlang Factory 2014. I was lucky enough to attend a talk by Brian Troutwine from AdRoll. It convinced me that Exometer would be the perfect library to do this. You can find his talk here: http://www.erlang-factory.com/berlin2014/brian-troutwine

Exometer is an Erlang package that allows you to collect different kinds of stats. There is two ways to record metrics: (1) Tell Exometer to update a value. (2) Have it call functions regularly itself. It uses ETFs to store the metrics in the Erlang VM. Before exporting them it can aggregate values. That is crucial if you are recording thousands of values per second (like hit counts).

Exometer knows a couple of different metric types. Most of these types are similar in other metric engines and services. You can read more about them in the Exometer read me. I’ll introduce the ones we will be using:

Gauge: This is by far the easiest one. It holds one numeric value. You can update it repeatedly but when exporting it uses the last value. This works well for metrics like memory usage or disk space. We do not care about down-to-the-nanosecond values here. As long as we aggregate regularly enough the latest value is all we are interested in.

Histogram: Histograms are a collection of time-series data points. A good example for this is response times. Histogram values are all recorded and then aggregated. Common aggregations are minimum values, maximum values, averages and percentiles.

Spiral: This type is Exometer-specific. It enables you to increase a value over time. When reporting the value it returns the total and the metric reset. This metric type is great for metrics like hits per minute. You can increase the value by 1 every time until it’s exported.

You have to define Exometer metrics before you can use them. There are two types of creating metrics. (1) Dynamic definition (by using functions during runtime). (2) Static definition (by using a config file). We will be using the second way as we know what we want to collect beforehand.

Exometer consists of three parts: The metric types, the storing service and the exporters. I will explain later how to tie them together.

The true power of Exometer is that it is super robust and lives within the Erlang VM. Thus you can record as many metrics as you like. You will not slow down your application down. You don’t want monitoring to take down your application.

StatsD

StatsD is an agent that receives metric values using UDP and forwards them to other services. It is comparable to an intermediary. The metrics have to be formatted in a way that makes sense to StatsD. It also does aggregation, too.

There are many agents out there and some SaaS monitoring tools offer custom agents. It is possible to have one StatsD agent forward stats to another agent. That way you can use different monitoring tools at the same time.