Monitoring Java applications can be a painful operation that often require lots of onfiguration, with technologies like byte code instrumentation and JMX, you can literally have thousands of Metrics to choose from just from a single Java application. This post will talk about JMole which has good sensible defaults that makes monitoring java applications much less painful.

JMole is an excellent monitoring client framework for gathering data dynamically from various Java MBeans (Java 6 and higher). It consists of a single JAR file that only need to be bootstrapped or if deployed in an Java Enterprise Application Server this is done automatically by a built-in Singleton Bean. The MBeans data is then exposed as using different monitoring-systems/protocols like:

JMX

Munin

NRPE (Nagios, Icinga)

Zabbix

StatsD

Standard Java Logging (java.util.logging)

Jolokia

Lets do a simple demo of the capabilities of what JMole is able to do for us! Apache tomcat is very easy to get up and running so we use it to test JMole. And present it to Munin to make some nice graphs for us.

For this we need Java (6 or higher) and Maven installed (for building) or grab the Mole artifacts directly from sonartype: here

$ mkdir /tmp/demo ; cd /tmp/demo $ wget http://www-eu.apache.org/dist/tomcat/tomcat-8/v8.5.6/bin/apache-tomcat-8.5.6.tar.gz $ tar xvf apache-tomcat-8.5.6.tar.gz $ git clone https://bitbucket.org/awelen/jmole.git $ cd jmole $ git checkout tags/1.2.1 -b 1.2.1 $ mvn package .... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] JMole .............................................. SUCCESS [ 0.145 s] [INFO] JMole Core ......................................... SUCCESS [ 5.239 s] [INFO] JMole Web .......................................... SUCCESS [ 0.349 s] [INFO] JMole Mule ESB ..................................... SUCCESS [ 2.646 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 9.741 s [INFO] Finished at: 2016-11-09T13:49:53+01:00 [INFO] Final Memory: 37M/973M [INFO] ------------------------------------------------------------------------

Total build time of just under 10 seconds is not bad. Build is done. Now choose the artifact for JMole Web jmole-web/target/jmole-web-1.2.1.war which is used for Web Containers (Servlet 2.3) like our Tomcat - put it in Tomcat’s webapps directory.

We also need to copy the standard JMole configurations for Tomcat and for the JVM.

$ cp /tmp/demo/jmole-web/target/jmole-web-1.2.1.war /tmp/demo/apache-tomcat-8.5.6/webapps/ $ cp /tmp/demo/jmole/jmole-core/configs/JMole_Tomcat_8.xml /tmp/demo/apache-tomcat-8.5.6/conf/ $ cp /tmp/demo/jmole/jmole-core/configs/JMole_JVM.xml /tmp/demo/apache-tomcat-8.5.6/conf/

That’s all! Now we are ready to start tomcat. First we need to specify some JMole configuration parameters. This is done via the JAVA_OPTS environment variable, we add the JMole configuration for both Tomcat and for the JVM.

$ cd /tmp/demo/apache-tomcat-8.5.6/ $ JAVA_OPTS = "-Djmole.config.filenames=/tmp/demo/apache-tomcat-8.5.6/conf/JMole_Tomcat_8.xml -Djmole.protocol.munin.enabled=true" bin/catalina.sh run

If you check the logfile you should see something like:

25-Dec-2016 15:00:00.031 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive /tmp/demo/apache-tomcat-8.5.6/webapps/jmole-web-1.2.1.war ... 25-Dec-2016 15:00:00.356 INFO [localhost-startStop-1] net.welen.jmole.Lifecycle.setup Starting JMole 25-Dec-2016 15:00:00.360 INFO [localhost-startStop-1] net.welen.jmole.JMole.configure Configuring JMole with config file: /tmp/demo/apache-tomcat-8.5.6/conf/JMole_Tomcat_8.xml ...

With tomcat started we now have a munin-node (via JMole) listening on port 4949, lets try it!

$ echo "fetch jmole" | nc localhost 4949 # munin node at 127.0.0.1:4949 multigraph Requests___ajp_nio_8009_ requestCount.value 0.0 errorCount.value 0.0 multigraph Requests___http_nio_8080_ requestCount.value 967.7655856692489 errorCount.value 0.733155746719128 multigraph Requests____localhost_examples_async3 requestCount.value 302.06016764828075 errorCount.value 0.0 ... multigraph Connection_pool___http_nio_8080_ keepAliveCount.value 2 maxKeepAliveRequests.value 100 connectionCount.value 2 maxConnections.value 10000 ...

JMole is alive and sends us metrics acting as a Munin node. To start using it you just need to add JMole munin-node to munin (I’ll leave the installation and configuration of Munin as an exercise to the reader), example config:

[JMole] address localhost port 4949

Which will get picked up by munin and will generate quite a lot of graphs for us:

Graph list

And after about a day or so we will get some nice graphs:

Generated JMole graphs

But wait! JMole also supports NRPE native, so we can use Nagios/Icinga to monitor our JVM/Web Container Runtime. To start the NRPE “client” we have to add -Djmole.protocol.nrpe.enabled=true to JAVA_OPTS, so starting tomcat now looks like:

$ JAVA_OPTS = "-Djmole.config.filenames=/tmp/demo/apache-tomcat-8.5.6/conf/JMole_Tomcat_8.xml -Djmole.protocol.munin.enabled=true -Djmole.protocol.nrpe.enabled=true" bin/catalina.sh run

Here are some example uses:

# Check warnings $ ./check_nrpe -n -H 127.0.0.1 -c checkWarnings OK - No problems detected. # Check criticals $ ./check_nrpe -n -H 127.0.0.1 -c checkCriticals OK - No problems detected. # Lets check JVM threads, warn if over 200 and critical over 500 $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM Threads ThreadCount 200: 500: OK - JVM/Threads/ThreadCount=121 # Test for warning 100 and critical 130 $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM Threads ThreadCount 100: 130: WARNING - JVM/Threads/ThreadCount=110 # Test for critical ( 100 ) $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM Threads ThreadCount 50: 100: CRITICAL - JVM/Threads/ThreadCount=110 # To check without thresholds $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM Threads ThreadCount 0:0 0:0 OK - JVM/Threads/ThreadCount=121 # Get JVM uptime $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM Uptime Uptime 0:0 0:0 OK - JVM/Uptime/Uptime=6232566 # Get Time spent in last Garbage Collection: PS MarkSweep $ ./check_nrpe -n -H 127.0.0.1 -c getValue -a JVM "Time spent in last Garbage Collection: PS MarkSweep" duration 0:0 0:0 OK - JVM/Time spent in last Garbage Collection: PS MarkSweep/duration=50

So with just one tool we are able to monitor our Java applications using different monitoring systems, very handy!

There are a lot of things that is possible to monitor, you can find more by the MBean net.welen.jmole:service=jmole, if you want more information or read up on how to configure JMole to monitor exactly what you need, read more at JMole’s Wiki