TL;DR — Erlang VM is very efficient as it crunches small amounts of messages with no effort at all. Letting message queue to build up decreases performance 2–5 times. Avoid writing your own message rate control code — look into GenStage or Flow . Also see power tip below for extra memory management tips.

Subscribe to this publication, so that you’ll get instant updates when I post new benchmarks. If you interested in specific benchmarks — write a comment and I’ll try to write another article on that.

Crawling to stop

I write an application, which in some extreme cases has to produce big data structure, iterate over it and send to another process over 500k elements for serializing and network delivery. And on big requests this application was really getting very slow, so I decided to investigate and benchmark.

Basically there are 2 processes:

data producer, which iterates map and it is quite fast

data consumer, which does serialization and it is slow

While doing benchmarks I noticed that consumer message queue was quickly building up. You can check it in observer application or by checking

Process.info(pid)[:message_queue_len]

Consumer process was processing messages slowly and falling much behind. This was leading to queue build-up, extra memory consumption and it was bringing consumer to crawl.

To show this behaviour I decided to write some tests.

Flood message test exhibits exactly this behaviour and sync message test stops every 1000 messages and waits acknowledgment from consumer that it processed all messages so far.

Seems that removing messages from queue is quite slow, even if it is just first message in queue and not a selective receive.