Recently we had an issue with one of our services which became very slow. As we tried to find the problem, I came across the idea to start a performance test series. In this blog post, I want to tell you something about Gnuplot and how we used it to find the issue with our very slow service.

Introduction to Gnuplot

Gnuplot is a very old tool which was published in 1986.

It is based on C and can be used to generate diagrams of any kind. The current version is 5.2.4 released in June 2018. Gnuplot does not provide a graphical interface but has a command line interface.

To provide an example I created a simple script to generate some random data and save it as PNG image with Gnuplot. See the table below the script for explainations.

#!/bin/bash DATA_FILE="$(mktemp)" # 1. PLOT_OUTPUT="plot.png" # 2. for ITERATION in $(seq 0 50) # 3. do echo -e "${ITERATION}\t${RANDOM}" | tee -a ${DATA_FILE} # 4. done gnuplot <<-EOF > "${PLOT_OUTPUT}" # 5. set terminal png # 6. set xlabel "Iterations" # 7. set ylabel "Random Number" # 7. set title "Random Number Chart" # 8. plot '${DATA_FILE}' using 1:2 with linespoints title "" # 9. EOF rm ${DATA_FILE} # 10.

mktemp creates a temporary file under / tmp Static path to the output image I use seq to iterate from 0 to 50 This echo simply prints the current iteration (0 to 50) and a random number up to 32767 (${RANDOM} is a bash builtin function which returns a 16 bit signed integer) I simply send a here-document into gnuplot and redirects the output into ${PLOT_OUTPUT} With set terminal, you are able to set the output format. As example , I set it to png to get an image afterwards The xlabel and ylabel option may be used to name both axes title can be used to name the whole diagramm plot is quite interesting. This is the actual plotting command for gnuplot . I command plot to use ${DATA_FILE} as data source. To define which columns of the data source should be used to plot you can use 1:2 which means use the first and second column to print the diagram. The with argument tells plot which type of lines will be used. I took linespoints which result in a diagram with straight lines and a point on each data point. This deletes the temporary datafile for cleanup purposes

I executed the script and received this image. Your image may look different.

Service Issue

Now, back to the issue with one of our services. It’s a service in front of a database with a paging interface which allows us to get all objects as defineable chunks. At some point, it wasn’t possible to iterate through all objects because we reached TCP timeouts.

Here comes Gnuplot to the rescue. I wrote a small script which executes curl with GNU Parallel.

#!/bin/bash DATA_FILE="$(mktemp)" STARTED_AT="$(date +%s)" LIMIT=50 # How many objects per request OFFSET=0 # At which index position it should start ITERATIONS=100 # How many requests should be started in total JOBS=20 # How many jobs should run parallel BASE_URL="https://url.tld" # Base url PNG="plot_${STARTED_AT}_LIMIT-${LIMIT}_OFFSET-${OFFSET}_ITERATIONS-${ITERATIONS}_JOBS-${JOBS}.png" # Output filename echo "Will create ${PNG}" performanceParallel() { CURRENT_PROCESS=${1} LIMIT=${2} OFFSET=${3} URL="${4}?limit=${LIMIT}&offset=$(( OFFSET + CURRENT_PROCESS * LIMIT ))" curl -o /dev/null -s -w "${CURRENT_PROCESS} %{time_total}

" -X GET "${URL}" } export -f performanceParallel parallel -j${JOBS} performanceParallel "{}" "${LIMIT}" "${OFFSET}" "${BASE_URL}" ::: $(seq 0 ${ITERATIONS}) | tee -a ${DATA_FILE} gnuplot <<-EOF > "${PNG}" set terminal png set title "${STARTED_AT} - ${BASE_URL} ::: $(( ITERATIONS * LIMIT )) Objects received" plot '${DATA_FILE}' smooth csplines title "", '${DATA_FILE}' with points title "" EOF rm ${DATA_FILE}

If you don’t know GNU Parallel you should take a look at their man Page to get an idea of how it works. In this example, I exported the function performanceParallel() and executed it with parallel. Due to restrictions, it’s only possible to execute one command with parallel but I wanted to do some calculation (because of the pagination).

As you can see I altered the Gnuplot command plot a little bit. It plots two graphs. The first plot is smooth csplines which produces kinda smoother lines than linespoints . The second plot is points which only prints points to visualize the actual data points.

Interpreting Diagrams

I executed several combination of tests against the service and received the following two images.

This is a test with JOBS=1 which means the execution is serial

This is a test with JOBS=5 which means the execution is running with 5 jobs in parallel

The x-axis in both images means the iteration number (${ITERATION}) which is unique within a test run. The y-axis stands for the response time measured by curl with these parameters: -w “${CURRENT_PROCESS} %{time_total}

”

In total, we received 2500 objects for each test but there is a huge difference. As you can see the serial test responses were between ~2,1 seconds and ~4,2 seconds. By contrast, the parallel test responses were between ~3 seconds and ~15 seconds. I noticed the slowest response in the parallel test is nearly 5 times slower response than the fastest response. This finding leads to the theory that the service is single threaded at some point.

After some further tests with nearly same results I forwarded my knowledge to the developers which results in a bug fix. Now the service is performing very well.

Conclusion

To be honest Gnuplot saved my day. It’s fairly easy to use and runs on nearly all Operating Systems. You are also able to set the terminal to svg to receive a scalable image.

Do you have any successful uses of Gnuplot? Let me know in the comments or send me an E-Mail.

Thanks for reading! 🙂