As of 2.5.0, promtool has a feature to allow you to test your recording rules.

PromQL is a powerful language, and it's possible to write expressions that don't quite do what you expect. As with programming generally, unit tests can help you detect such problems and reduce the chances of inadvertent breakage in the future. The unit test feature of promtool is based on the unit tests used by the PromQL implementation internally.

Let's take a simple example:

wget https://github.com/prometheus/prometheus/releases/download/v2.5.0/prometheus-2.5.0.linux-amd64.tar.gz tar -xzf prometheus-*.tar.gz cd prometheus-* cat >rules.yml <<EOF groups: - name: example rules: - record: job:up:sum expr: sum without(instance)(up) EOF

To test this rule you can create test.yml with the following content:

rule_files: - rules.yml evaluation_interval: 1m tests: - interval: 1m input_series: - series: 'up{job="node",instance="foo"}' values: '1+0x10' - series: 'up{job="node",instance="bar"}' values: '1+0x5 0+0x5' - series: 'up{job="prometheus",instance="foo"}' values: '1+0x10' promql_expr_test: - expr: job:up:sum eval_time: 1m exp_samples: - labels: 'job:up:sum{job="node"}' value: 2 - labels: 'job:up:sum{job="prometheus"}' value: 1 - expr: job:up:sum eval_time: 6m exp_samples: - labels: 'job:up:sum{job="node"}' value: 1 - labels: 'job:up:sum{job="prometheus"}' value: 1

You can then run these tests with:

./promtool test rules test.yml

which should return:

Unit Testing: test.yml SUCCESS

Let's break down the test file:

rule_files: - rules.yml evaluation_interval: 1m

This says that we want to load the rules file rules.yml and evaluate the rules in it every minute.

tests: - interval: 1m input_series: - series: 'up{job="node",instance="foo"}' values: '1+0x10' - series: 'up{job="node",instance="bar"}' values: '1+0x5 0+0x5' - series: 'up{job="prometheus",instance="foo"}' values: '1+0x10'

This defines the first set of tests, and provides input data. The series will have samples every minute. The first series is 1 1 1 1 1 1 1 1 1 1 , the second 1 1 1 1 1 0 0 0 0 0 , and the third is 1 1 1 1 1 1 1 1 1 1 .

promql_expr_test: - expr: job:up:sum eval_time: 1m exp_samples: - labels: 'job:up:sum{job="node"}' value: 2 - labels: 'job:up:sum{job="prometheus"}' value: 1 - expr: job:up:sum eval_time: 6m exp_samples: - labels: 'job:up:sum{job="node"}' value: 1 - labels: 'job:up:sum{job="prometheus"}' value: 1

Finally there's the actual tests. The first evaluates job:up:sum at one minute, which returns 2 and 1. The second evaulates it at six minutes, by which time up{job="node",instance="bar"} has become 0 and the result is now 1 and 1.

While this is a trivial example, it shows how you can go about testing more complex PromQL expressions.

Have a question about PromQL? Contact us.