Setup Node Monitoring

After successfully setting and and running your node, the last thing you can do is setup services to monitor your node.

We use multiple services: updown.io, Grafana, P-Rep Monitor, Tracker

updown.io

With updown you can perform regular health checks for any of your services. In our case we are going to use updown to check if our node is online and fully synced.

updown.io Admin Dashboard

To create a new health check we are going to add the API endpoint URL of our node so it checks if the response was successful (HTTP 200), but it also checks if a certain string is contained in the response (we use that to check if the blockchain is synced to the latest state).

Create a new health check

In our case we use our node’s API endpoint. Then we tell the service to check the response for the string “Service is online”. We can use that to check if the node is fully synced. Finally we will do this health check every 15 seconds.

We recommend to setup Telegram notifications, so updown.io will immediately send you a notification as soon as your node has issues and you can fix any issues very fast.

After all you also get a nice dashboard where you can see your uptime statistics and latency from various locations all over the world:

updown.io Stats Dashboard

Grafana

The second monitoring tool is Grafana and it’s installed directly on the node. There is an awesome GitHub repository called “dockprom” which already comes with pre-configured dashboards for both the host machine and the docker containers.

The installation is very simple, you only need to clone the repository and run the service via docker-compose:



(prep)$ cd grafana

(prep)$ docker-compose up -d (prep)$ git clone https://github.com/stefanprodan/dockprom grafana(prep)$ cd grafana(prep)$ docker-compose up -d

dockprom docker containers

This will start up a few more docker containers. Most of them are responsible for collecting data of both the host machine and any running containers.

Make sure port 3000 is open on your P-Rep machine as that’s where the Grafana dashboard can be accessed:

Grafana Login

The default login credentials are:

Username : admin

: admin Password: admin

After signing in for the first time, you immediately get prompted to change your password — which you should definitely do if you plan to enable access for all IP Addresses to your dashboard!

Grafana Home Dashboard

Click on “Home” in the top left corner and then you should see the available, pre-configured dashboards. As the name suggests “Docker Containers” is the dashboard where you can see the metrics of your P-Rep node, “Docker Host” are the metrics for your host machine.

Docker Containers dashboard

Setting up a dashboard to track your nodes metrics this way is very convenient as you don’t have to be familiar with Grafana. If you are more experienced, you can change the dashboard, track other metrics and do all kind of fancy analytics stuff. 🤓

By default, your password always gets reset to the default one (admin:admin) via environment variables in the dockprom’s docker-compose.yml file. So the last step is to disable those environment variables in order to persist the changed password:

(prep)$ cd grafana

(prep)$ vi docker-compose.yml

dockprom docker-compose.yml

Scroll down to the “grafana” section and comment out all the environment variables:

#environment:

#- GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}

#- GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}

#- GF_USERS_ALLOW_SIGN_UP=false

Now just restart and you are good to go:

(prep)$ docker-compose down && docker-compose up -d

Done. The changed password will be persisted and you can enjoy your fancy dashboard! 👌

P-Rep Monitor

The P-Rep Monitor is an official tool from the ICON team. We use it to monitor the overall network state, other P-Reps, etc.

You can find the TestNet P-Rep Monitor at: https://zicon.mon.solidwallet.io

P-Rep Monitor

Tracker

The tracker is also a standard tool by the ICON team itself. It’s a normal block explorer with an integrated governance tracker. We use it to monitor our P-Rep position as well as our productivity.

You can find the TestNet tracker at: https://zicon.tracker.solidwallet.io/governance