Perl 6 speed improvements over the years

2018/09/25

Perl 6.c (aka Perl 6.0) was officially released at christmas 2015, december 25 (https://perl6advent.wordpress.com/2015/12/25/christmas-is-here/). A year before that, Merijn ‘Tux’ Brand made a Perl 6 version of the widely used module Text::CSV ( https://github.com/Tux/CSV ), he started working on it September 3, 2014, “My first start” on October 3, 2014), and he made a test suite for this, which became the speed canary for Perl 6:

https://tux.nl/Talks/CSV6/speed.log

The first test results came in October at a whopping 256,98 seconds (yes, more than 4 minutes!). A week later, the speed increased, reducing the test time to 201,62 seconds and again three weeks later, on November 5, 2014, over 2 minutes were taken off when the test suite took a mere 71,85 seconds. By the time Perl 6.c was released, test.t took 12,97 seconds.

Over the years Perl 6 got faster and faster (some people would say “less slow”). Several people now consider Perl 6 fast enough to use it in production. Other people say that Perl 6 needs to be at least 10 times faster before they will even begin to consider it fast enough. With the current rate of speed increases, some 16% per quarter, it will take another 4 years before we will hit that mark.

Tux has made a page where he compares the speed of execution of his tests using a lot of different programming languages (with or without the use of some modules):

https://tux.nl/Talks/CSV6/speed5.html

With the fastest method, Perl 5 (Text::CSV::Easy_XS, which uses XS-code, and does not have the full feature set)) takes 0,008 seconds and is therefore 221 times faster than the 1,770 seconds that Perl 6 uses now (and that number used to be several thousands!). The full-featured Perl 5 version of Text::CSV that uses XS-code takes 0.033 seconds, and that is 53,6 times faster than in Perl 6.

The fastest full-featured test of the Perl 5 version of Text::CSV without XS-code (“Pure Perl”) takes 0,848 seconds. That is the most honest comparison: the Perl 6 version is also full-featured and does not use C or other code. Compared this way, Perl 5 is merely 2,09 times faster than Perl 6 (and beginning of this year it was 3,48 times faster). So, Perl 6 is catching up fast. And when the current speed increase would continue, it would take less than 1,5 year to overtake this.

To make this blog sound a bit triumphant: the test also checks the speed for Text::CSV with the option –race, and the fastest sofar (today at 12:08, 0,791 seconds) is faster than the 0,848 seconds of the Pure Perl version. When I noticed that, I did a little happy dance. Also, this feature shows that concurrency actually can be useful (not in all cases, real-life CSV data cannot be guaranteed to be “spotless” enough to be handled nicely).

Now the overview of progress. What I have done in the table below, is look in the speed.log file for the fastest test in the days before/after the mentioned date. The speed varies often, even on one day (the test will use the bleed version of Perl 6, taking in account the latest core and other developments of Perl 6), so I took some liberties, and results are of course debatable, and I am not a statistician. But the overall progress is clearly visible. And several Perl 6 core developers claim more significant speed increases are coming later this year and of course next year. Some notes: the hardware that is used for test.t has not changed, and the test suite has not changed significantly, making the test results comparable.

Date Seconds Speed change Change over 1 year

. compared to

. previous period

2014-10 256,98

2014-11 71,85 72,04% (since 2014-10)



2015-01 59,04 17,83% 77,03% (since 2014-10)

2015-04 40,94 30,66% (since 2015-01)

2015-07 42,54 -3,91% (since 2015-04)

2015-10 41,00 3,62%



2016-01 12,50 69,51% 78,83% (since 2015-01)

2016-04 12,16 2,72%

2016-07 9,39 22,78%

2016-10 7,22 23,11%



2017-01 5,56 23,00% 55,52% (since 2016-01)

2017-04 4,89 12,05%

2017-07 4,09 16,36%

2017-10 3,25 20,55%



2018-01 2,95 9,23% 46,94% (since 2017-01)

2018-04 2,55 13,56%

2018-07 2,23 12,55%

2018-09 2,015 9,64%

2018-09-25 1,770 12,16% (since 2018-09)

. 40,00% (since 2018-01)

. 68,17% (since 2017-01)

. 85.84% (since 2016-01)

. 97,00% (since 2015-01)

. 97,54% (since 2014-11)



That means that the current version of Perl 6 is 145 times faster than the version in October 2014. In a period less than 4 years, that is a nice achievement.

Of course, there are other speed canaries for Perl 6. I didn’t find one that shows lots of numbers like Text::CSV does. An example to show the increase of the speed of Perl 6: https://perl6advent.wordpress.com/2017/12/16/day-16-%F0%9F%8E%B6-deck-the-halls-with-perf-improvements-%F0%9F%8E%B6/

Please feel free to tell us in the comments about your own favorite speed canary for Perl 6.

I wish everybody success who is working on improving Perl 6 usability, features, and speed. I wonder where Perl 6 will be in 12 months.