It’s September, and in universities, that means tons of new people. New staff, new faculty, new students. Lots and lots of new people.

Here at The College of Computer and Information Science at Northeastern University, we’ve got a banner crop of incoming CS students. So many, in fact, that we bumped up against one of those things that we don’t think about a lot. Email licenses.

Every year, we pay for a lot of licenses. We’ve never monitored the number used vs the number bought, but we buy many thousand seats. Well, we ran out last week. Oops.

After calling our reseller, who hooked us up with a temporary emergency bump, we made it through the day until we could buy more. I decided that it was time to start monitoring that sort of thing, so I started working on learning the Zimbra back-end.

Before you follow along with anything in this article, you should know — my version of Zimbra is old. Like, antique:

Zimbra was very cool about this and issued us some emergency licenses so that we could do what we needed until our new license block purchase went through. Thanks Zimbra!

In light of the whole “running out of licenses” surprise, I decided that the first thing I should start monitoring is license usage. In fact, I instrumented it so well that I can pinpoint the exact moment that we went over the number of emergency licenses we got:

Cool, right?

Well, except for the whole “now we’re out of licenses” again thing. Sigh.

I mentioned a while back that I was going to be concentrating on instrumenting my infrastructure this year, and although I got a late start, it’s going reasonably well. In that blog entry, I linked to a GitHub repo where I built a Vagrant-based Graphite installation. I used that work as the basis for the work I did when creating a production Graphite installation, using the echocat graphite module.

After getting Graphite up and running, I started gathering metrics in an automated fashion from the rest of the puppetized infrastructure using the pdxcat CollectD puppet module, and I wrote a little bit about how similar that was with my Kerbal Space Administration blog entry.

But my Zimbra install is old. Really old, and the server it’s on isn’t puppetized, and I don’t even want to think about compiling collectd on the version of Ubuntu this machine runs. So I was going to need something else.

As it turns out, I’ve been working in Python for a little while, and I’d written a relatively short program that serves both as a standalone command that can send a single metric to Carbon or can function as a library, if you need to send a lot of metrics at a time. I’m sure there’s probably a dozen tools to do this, but it was relatively easy, so I just figured I’d make my own. You can check it out on GitHub if you’re interested.

So that’s the script I’m using, but a script needs data. If you log in to the Zimbra admin interface (which I try not to do, because it requires Firefox in the old version we’re using), you can actually see most of the stats you’re interested in. It’s possible to scrape that page and get the information, but it’s much nicer to get to the source data itself. Fortunately, Zimbra makes that (relatively) easy:

In the Zimbra home directory (/opt/zimbra in my case), there is a “zmstats/” subdirectory, and in there you’ll find a BUNCH of directories with dates as names, and some CSV files:

… snip …

drwxr-x — — 2 zimbra zimbra 4096 2014–09–04 00:00 2014–09–03/

drwxr-x — — 2 zimbra zimbra 4096 2014–09–05 00:00 2014–09–04/

drwxr-x — — 2 zimbra zimbra 4096 2014–09–06 00:00 2014–09–05/

-rw-r — — — 1 zimbra zimbra 499471 2014–09–06 20:11 cpu.csv

-rw-r — — — 1 zimbra zimbra 63018 2014–09–06 20:11 fd.csv

-rw-r — — — 1 zimbra zimbra 726108 2014–09–06 20:12 imap.csv

-rw-r — — — 1 zimbra zimbra 142226 2014–09–06 20:11 io.csv

-rw-r — — — 1 zimbra zimbra 278966 2014–09–06 20:11 io-x.csv

-rw-r — — — 1 zimbra zimbra 406240 2014–09–06 20:12 mailboxd.csv

-rw-r — — — 1 zimbra zimbra 72780 2014–09–06 20:12 mtaqueue.csv

-rw-r — — — 1 zimbra zimbra 2559697 2014–09–06 20:12 mysql.csv

drwxr-x — — 2 zimbra zimbra 4096 2014–06–15 22:13 pid/

-rw-r — — — 1 zimbra zimbra 259389 2014–09–06 20:12 pop3.csv

-rw-r — — — 1 zimbra zimbra 893333 2014–09–06 20:12 proc.csv

-rw-r — — — 1 zimbra zimbra 291123 2014–09–06 20:12 soap.csv

-rw-r — — — 1 zimbra zimbra 64545 2014–09–06 20:12 threads.csv

-rw-r — — — 1 zimbra zimbra 691469 2014–09–06 20:11 vm.csv

-rw-r — — — 1 zimbra zimbra 105 2014–09–06 19:08 zmstat.out

-rw-r — — — 1 zimbra zimbra 151 2014–09–06 06:28 zmstat.out.1.gz

-rw-r — — — 1 zimbra zimbra 89 2014–09–04 21:15 zmstat.out.2.gz

-rw-r — — — 1 zimbra zimbra 98 2014–09–04 01:41 zmstat.out.3.gz

Each of those CSV files contains the information you want, in one of a couple of formats. Most are really easy.

sudo head mtaqueue.csv

Password:

timestamp, KBytes, requests

09/06/2014 00:00:00, 4215, 17

09/06/2014 00:00:30, 4257, 17

09/06/2014 00:01:00, 4254, 17

09/06/2014 00:01:30, 4210, 16

… snip …

In this case, there are three columns, which include the timestamp, the number of kilobytes in queue, and the number of requests. Most CSV files have (many) more columns, but this works pretty simply. That file is updated every minute, so if you have a cronjob run, grab the last line of that file, parse it, and send it into Graphite, then your work is basically done:

zimbra$ crontab -l

… snip …

* * * * * /opt/zimbra/zimbra-stats/zimbraMTAqueue.py

And looking at that file, it’s super-easy: