.Net Core 2.0 Performance Notes Revisited

This post is part of the F# Advent Calendar 2017 series. Many thanks to Sergey Tihon for organizing these.

Over the past few weeks I've been submitting improvements to some of the F# programs in the Benchmarks Game. In a previous post I did this for the C# programs.

Since that post things have moved on and C# is currently faster than Java for 8 out of 10 of the programs. Java is faster for regex-redux as .Net Core doesn't yet have a compiled regex implementation. For k-nucleotide Java makes use of a dictionary well suited to the program not available to C#.

Most of the submissions to the F# programs were ports of the C# code that had recently been optimised. For fasta and k-nucleotide further optimisations were discovered. ArrayPool is very useful in the case of fasta . For k-nucleotide the largest dictionary can be constructed more efficiently in four parallel parts.

Another tempting optimisation was there being a one to one replacement in F# to use native pointers for arrays e.g. Array.get a i becomes NativePtr.get a i . This only actually provided a small improvement in most cases and wasn't always done.

I feel I must plug Expecto's Expect.isFasterThan. It's a quick way of checking that one implementation is truly faster than another and has proven invaluable.

C# vs Java, F# vs C#, F# vs Java, F# vs Haskell, F# vs OCaml, F# vs Python

Program C# F# Java Haskell OCaml Python pidigits 3.03 3.05 3.12 Error Error 3.43 reverse-complement 0.78 0.82 1.03 1.40 0.79 3.26 fannkuch-redux 14.44 16.65 17.26 15.40 16.12 565.97 binary-trees 8.26 8.54 8.34 23.66 10.03 93.55 n-body 21.37 22.86 22.10 21.43 21.67 838.39 mandelbrot 5.83 6.66 6.04 11.69 55.18 225.24 fasta 2.09 1.67 2.33 9.36 6.00 59.47 k-nucleotide 11.47 10.43 8.70 35.01 21.63 77.65 regex-redux 30.74 31.02 10.34 Error 24.66 15.22 spectral-norm 4.07 4.22 4.23 4.04 4.31 180.97

As mentioned in the previous post there are some caveats to these results. They represent the current state of a set of programs on a specific test machine. However, there is enough evidence for some general conclusions.

The overall results for .Net Core 2.0 are very impressive compared to other managed platforms.

F# performance in the worst case is only 15% behind C#. F# is a high-level language that results in simpler and shorter code. It's good that even in the extreme of a low-level performance benchmark it is not too far behind C#.

F# in fact shows very good performance against Java resulting in a five all draw. This means F# would be expected to perform better than Scala or Kotlin if they were to participate in the benchmarks.

F# looks to have the best performance among the functional languages. This is due to the performance of .Net Core 2.0 and being able to write F# in a functional-first style.

Hopefully 2018 will see continued adoption of both .Net Core 2.0 and F#.

Happy New Year!