Adding rate limiting to the load balancer

In order to add rate limiting support to our load balancer, we need to modify the configuration file that the HAProxy instance uses. We have to make sure that the loadbalancer container picks up the haproxy-ratelimiter.cfg configuration file.

Simply modify the Dockerfile to use this one instead.

Rate limiting directives

The configuration file haproxy-ratelimiter.cfg is what this article is all about.

Let’s take a closer look

HAProxy offers a very low level set of primitives that offer great flexibility and can be used for a variety of use cases. The generic counters it exposes, often remind me of a CPU’s accumulator register. They store intermediate results, can take various semantics, but at the end of the day they are just numbers. To get a good understanding it makes sense to start from the very end of the configuration file.

The Abuse stick table

Here, we define a dummy backend called Abuse . Dummy, since it’s only used to define a stick-table that the rest of the configuration can refer to by the name Abuse . The stick-table is nothing but a storage space, or better, a lookup table for request data. Our stick-table has the following characteristics:

type ip : Requests stored in the stick table will have their IP as key. So, requests from the same IP will refer to the same record. Essentially this means that we keep track of IPs and data related to them.

: Requests stored in the stick table will have their IP as key. So, requests from the same IP will refer to the same record. Essentially this means that we keep track of IPs and data related to them. size 100K : The table has a maximum of 100K entries.

: The table has a maximum of 100K entries. expire 30m : The table entries expire after 30 minutes of inactivity.

: The table entries expire after 30 minutes of inactivity. store gpc0,http_req_rate(10s) : The table records store the general purpose counter gpc0 and the IP’s request rate for the last 10 second interval. We’ll be using the gpc0 to keep track of the amount of times an IP has been marked as abusive. Essentially, a positive value implies that the IP has been marked as abusive. Let’s call this counter the abuse indicator .

All in all, what the Abuse table does is keep track of whether an IP is abusive as well as its current request rate. We therefore have their historical track record, as well as real-time behavior.

Now let’s go to the frontend proxy section and see what’s new there.

ACL functions and rules

An ACL(Access Control List) is a function declaration. The function is only invoked when used by a rule. In and of itself an ACL is nothing more than a declaration.

Let’s see all 3 of them in detail. Keep in mind that since all explicitly refer to the Abuse table which uses the IP as key, the functions are applied on the request’s IP.

acl is_abuse src_http_req_rate(Abuse) ge 10 : Function is_abuse returns True if the current request rate is greater than or equal to 10.

: Function returns if the current request rate is greater than or equal to 10. acl inc_abuse_cnt src_inc_gpc0(Abuse) gt 0 : Function inc_abuse_cnt returns True if the incremented value of gpc0 is greater than 0. The initial value of gpc0 is 0, and therefore the function always returns True . In other works, it increments the value of the abuse indicator , essentially marking the IP as abusive.

: Function returns if the incremented value of is greater than 0. The initial value of is 0, and therefore the function always returns . In other works, it increments the value of the , essentially marking the IP as abusive. acl abuse_cnt src_get_gpc0(Abuse) gt 0 : Function abuse_cnt returns True if the value of gpc0 is greater than 0. In other words it tells whether the IP has already been marked as abusive.

As mentioned earlier, the ACLs are simple declarations. They are not applied on incoming requests unless invoked by some rule.

It makes sense to take a look at the rules defined in the same frontend section. The rules are applied in turn on every incoming request and make use of the ACLs that we just defined. Let’s see what each one does.

tcp-request connection track-sc0 src table Abuse : Adds the request to the table Abuse . Since the table has defined the IP as its key, this rule basically adds the request IP to the table.

: Adds the request to the table . Since the table has defined the IP as its key, this rule basically adds the request IP to the table. tcp-request connection reject if abuse_cnt : Rejects new TCP connections if the IP has already been marked as abusive. In essense, it forbids new TCP connections from an abusive IP.

: Rejects new TCP connections if the IP has already been marked as abusive. In essense, it forbids new TCP connections from an abusive IP. http-request deny if abuse_cnt : Denies access to request if the IP has already been marked as abusive. This applies to already established connections that are still open, but correspond to an IP that has just been marked as abusive.

: Denies access to request if the IP has already been marked as abusive. This applies to already established connections that are still open, but correspond to an IP that has just been marked as abusive. http-request deny if is_abuse inc_abuse_cnt : Denies access to request if is_abuse and inc_abuse_cnt both return True . In other words, it denies access to the request if the IP currently has a high request rate, and then proceeds to annotate it as abusive.

Essentially we place real-time checks as well as historical. The second rule rejects all new TCP connections if the IP has been marked as abusive. The third rule denies serving HTTP requests if the IP has already been marked as abusive, regardless of its current request rate. The fourth rule ensures that HTTP requests from an IP are denied at the very moment its request rate threshold is crossed. So basically the second rule operates upon new TCP connections, whereas the third and fourth on established connections with the former being a historical check and the latter being a real-time check.

Let’s play!

We can now build and run our containers again.

$ sudo docker-compose down

$ sudo docker-compose build

$ sudo docker-compose up

Now, the loadbalancer should be running in front of the 2 api servers.