monitoring, nginx, web, web server, proxy, reverse proxy, load balancing

What is NGINX?

NGINX is a popular high performance HTTP and reverse proxy server, with the ability to handle a high number of concurrent connections with a low CPU and memory footprint.

Monitoring NGINX

As with most services, there are two main aspects of monitoring NGINX, both when used as web server to serve dynamic or static content, and when used as a reverse proxy or load balancer. The first aspect to consider is the underlying server where NGINX is being run, and the other being what metrics NGINX itself can expose about its internal operations and health.

The free and open source distribution of NGINX does not expose a great selection of internal monitoring metrics, as the more advanced monitoring capabilites of NGINX are reserved for commerical customers with a license for NGINX Plus. All hope is not lost though, as the free version of NGINX comes with a module called ngx_http_stub_status_module or stub_status for short, that can expose a few key status metrics from the running server.

By utilizing a monitoring solution that can turn some of the NGINX status metrics into aggregated counters, in combination with key process and network metrics from the underlying OS, you can build a powerful monitoring setup for your NGINX instances.

Useful metrics

The stub_status module will expose some important real-time metrics that can be useful to keep an eye on, such as the number of active connections, the number of connections in reading, writing and waiting states, as well as counters for the number of accepted, handled and total number of recieved requests duing NGINX's running time.

In addition to the metrics exposed by NGINX, it will also make sense to monitor the NGINX processes on the OS level, particularly the CPU and memory usage, as well as network interface traffic coming in and going out, and the number of associated TCP/IP sockets in different states.

NGINX status - stub_status metrics

Here is a overview of all the metrics you can obtain from the stub_status module of a running NGINX instance: Number of active connections

Active connections in reading state

state Active connections in writing state

state Active connections in waiting state In addition there are three counters representing the total number of its kind since the start of the NGINX instance: Requests accepted

Requests handled

Requests in total monometric.io chart monitoring active NGINX connections in different states

Normally the accepted and handled counters will be the same, unless some requests have hit a resource limit, such as the number of available workers. The number of requests in total will generally be higher than the number of accepted and handled requests, as this is an HTTP procotol request counter, where one accepted TCP/IP connection can handle multiple HTTP requests over the same session.

Enabling the stub_status module

First you should make sure that NGINX has been compiled with the status module enabled. Running the command below will tell you if NGINX has been compiled with the necessary with-http_stub_status_module flag. If the module is missing from your installation, you will have to obtain a release of NGINX with the module enabled, or compile NGINX with this flag yourself.

# nginx -V 2>&1 | grep -o with-http_stub_status_module

Enabling the stub_status module is straight forward, and is usually done by adding a location block within an existing server block in your NGINX configuration file. Remember to restart or reload the NGINX service after updating the configuration file.

location /nginx_status { allow 127.0.0.1; deny all; stub_status; } } server { listen 80; server_name example.com; root /srv/example.com/www/; index index.html; location / { try_files $uri $uri/ /index.php$is_args$args; }

By adding the status location block as shown above, you should now be able to access /nginx_status on your web server from localhost. It's important to note that the access list rules specified will only allow access from localhost, and should you need to access the status module remotely, you will need to specify the correct IP address in the allow rule. It's never recommended to leave the status location accessible to everyone.

After reloading the NGINX service, we can now test the status location URL to make sure everything works:

# curl -q -s http://example.com/nginx_status

Active connections: 59 server accepts handled requests 116160 116160 2033070 Reading: 2 Writing: 45 Waiting: 12

If your seeing output like the example above, congratulations — the NGINX status module is working correctly.

Making use of the NGINX status counters

Making use of the request counter metrics directly can be hard, as they tell you very little about the rate of which requests and accepts are happening. By using a monitoring tool that understands and aggregate the NGINX status counters, such as the accepted and total requests, you can monitor the number of requests per second which can be much more useful than just the counter value alone. monometric.io chart monitoring NGINX requests per seconds

Additional metrics to monitor

In addition to the metrics exposed by the NGINX status module, there are other important metrics to keep an eye on. Most important are the underlying server metrics that details the performance of the server on the operating system level, such as CPU and memory utilization. A proper monitoring solution will be able to tell you the exact CPU and memory usage of your NGINX processes, as well as information about network traffic and much more. CPU and memory usage A web server running NGINX will never perform better than what the underlying hardware or virtual machine will allow. Therefore it makes sense to keep an eye on the CPU and memory utilization of your server, to make sure you have enough capacity to handle the NGINX workload. It can be very useful to be able to correlate NGINX's own metrics with those of the server and operating system, to identify potential bottlenecks and to fine tune your setup. If you are seeing a growth or spike in NGINX's connections waiting metric, you should check if this correlates with a high load on your server. Likewise, if you are running low on RAM, this will impact the operations of NGINX, especially in relation to in memory caching and buffering amongst other things. Network interface traffic and TCP/IP sockets NGINX's performance is, naturally, also dependent on your underlying network capacity. If you are hitting your limits in terms of available bandwidth, this will have a negative impact on your web servers ability to handle requests in a timely fashion. It could lead to requests filling up in a waiting state, or cause connections to be dropped all together. Similarly, the performance of NGINX depends on having enough TCP/IP sockets available to handle all incoming and processing requests. Running out of available sockets on the operating system level will have severe impacts on your werb server performance, and could lead to high processing times, large wait queues and dropped requests. It is always recommended that you monitor these metrics in addition to the metrics exposed by NGINX itself, to get the best idea of your web server performance and where to do any necessary tuning. monometric.io chart monitoring NGINX CPU and memory usage monometric.io chart monitoring network interface traffic monometric.io chart monitoring the load of a server running NGINX monometric.io chart monitoring the RAM utilization of a server running NGINX

Monitoring NGINX with monometric.io

With monometric.io you get access to hundreds of underlying server and operating system metrics, as well as an easy to install NGINX plugin that enables you to make the most out of NGINX's metrics.

The NGINX plugin for monometric.io will expose all stub_status metrics, as well as useful aggregates of the status counters, enabling you to monitor such metrics as request per second, accepts per second and requests handled per second.

The NGINX plugin will also give you metrics for CPU and memory utilization by the NGINX processes.

NGINX dashboard

Build a customized dashboard to monitor the metrics most relevant to you. Correlate NGINX specific metrics with operating system, server and network interface metrics to get the perfect overview of your web server health and performance.