Update: This article is part of a series on load testing. I will try to regularly publish related topics in this series. I won’t repeat this table of contents on every single article, so please check this very first one to get new updates. Also I will try to link articles together, so that you can easily follow the next one in the series. Ever since, these has been published:

Let’s start with some questions:

Have you ever thought about ways to find out if your infrastructure setup could stand under high loads of users?

Have you ever been able to run a load test without dealing with so many configurations and steep learning curve of current testing tools?

Have you ever thought about ways to script your way through testing, either for load/performance or otherwise?

Have you ever thought that performance testing can be simplified and be included as part of your CI/CD process?

You are not alone in that. I remember days when I was the lead developer of a team working on developing an API to serve our ever-increasing set of Apps with many micro-services like content- and user-management, billing, analytics and the like. At that time, we used to use the latest best practices in software engineering in source control and CI/CD processes. We have set up our CI/CD process using docker and hosted GitLab and used GitLab CI to deal with CI process. After a while of testing and code-push, we have decided to go with an alpha-stage API that can answer our needs with a new App we used to develop.

After a while of development, testing and improvement, we decided to test our API under load to see if it can hold up to the promises. Since we have seen that after 200,000 users subscribing and 60,000 of them using one of our other Apps, there is an slow-down on the request/response flow to the other API, which roughly had the same setup.

So, in order to not miss the opportunity of (load) testing before customers coming in, we decided to investigate ways to load test our API. After our investigations into Locust, Apache JMeter and the like, we decided to go with Apache JMeter, which was (and still is) the tool of the trade. We thought it would be a good candidate for this purpose. It took us almost three days to be able to learn and execute simple load tests with the GUI version. We were able to produce load test from one server and analyze the results. The results were good, because it gave us a bit more insight into what we should expect from our API, in terms of number of concurrent users and our API’s response time under load.

But later we didn’t use the tool, since there was no clear way to integrate it into our CI/CD process at that time, and we didn’t have enough time to learn and use it. We wanted something more developer-friendly, more integrable to our workflow, and easier to learn. Nowadays, it has been improved very much and it integrates with CI/CD pipelines easily.