Noisemark Benchmarking Godot's programming alternatives

Each benchmark focuses on something and depending on this focus it may tell a different story. For the different programming languages usable with the Godot Game Engine, the most popular benchmark is Carter Anderson’s Godot Bunnymark, which gives a relatively strong emphasis to rendering. There’s nothing wrong with that, but it doesn’t tell the story I want to hear.

I am not particularly interested in using any language other than GDScript for the sake of rendering more things on the string. But I can see myself using other languages for doing more computation-intensive tasks, like procedural generation or AI for certain genres. So, where’s the benchmark telling this story?

Hopefully, here it is! Ladies and gentlemen: noisemark.

What’s Noisemark?

In short, it is just a measurement of the times taken by different implementations of OpenSimplexNoise – all of them straightforward conversions from the reference Java code, without any significant optimizations.

The benchmark itself consists of calling the noise function a large number of times: once for each pixel in a 500×500 pixels image. That’s an important point: the results shown below would be different if I had made just one single call to a function that takes a much longer time to execute. But that would be a different story.

Spoiler alert! There are results here!

In addition to D/GDNative, C# and GDScript, I added two other measurements. The first, Constant,is not a real implementation. It just uses a constant value as if it was the noise. This is meant to measure the overall overhead of looping through each image pixel and setting it.

The second one, Built-in, is just the built-in implementation of OpenSimplexNoise that arrived with Godot 3.1. It’s meant to be a lower bound: since it is compiled right into the engine executable, it doesn’t have any overhead the other implementations do. Basically, we cannot hope to be faster than that.

Also note that I didn’t include multiple languages for GDNative. That could be interesting, and I might add them someday, but I don’t think there would be a qualitative difference in the results if instead of D I had used Nim, Rust or C++.

Results

I ran each of the implementations five times and took the average (I didn’t took note of the standard deviation, but it was really low). The Time − Constant line is just this average subtracted of the Constant time (therefore it should be the most meaningful value: just the computation time, without the overhead of putting pixels into an image). The final column, Relative to Built-in, takes this latest value and scales it in proportion to the built-in implementation time, so that we have numbers that are easier to compare.

Intel Core i5-4460 (4th Gen) @ 3.20GHz

What Time (ms) Time − Constant (ms) Relative to Built-in Constant 55.0 0.0 − Built-in 139.6 84.6 1.00 GDNative (D) 181.4 126.4 1.49 C# 352.8 297.8 3.52 GDScript 2152.4 2097.4 24.79

Intel Core i3-8130U (8th Gen) @ 2.20GHz

What Time (ms) Time − Constant (ms) Relative to Built-in Constant 70.6 0.0 − Built-in 146.8 76.2 1.00 GDNative (D) 177.4 106.8 1.40 C# 355.6 285.0 3.74 GDScript 1946.4 1875.8 24.62

Source Code

Noisemark’s source code is available, under the MIT license. Have fun!