Perl6's newline translation behavior has been fixed A. Sinan Unur December 23, 2015

I am very happy to report that Bug #126881 for perl6: Newline handling is broken on Windows was recently resolved:

Now: We do translation of \r

graphemes to

on all input read as text except sockets, independent of platform.

graphemes to on all input read as text except sockets, independent of platform. We do translation of all

graphemes to \r

on text output to handles except sockets, on Windows only.

graphemes to on text output to handles except sockets, on Windows only.

is now, unless use newline is in force, always \x0A

is now, unless is in force, always We don't do any such translation when using .encode / .decode , and of course when reading/writing Buf s to files, providing an escape hatch from translation if needed.

If I understand the changes correctly, this is the logical way to handle to handle newline conventions for text streams:

In text mode, carriage return–linefeed combinations are translated into single linefeeds on input, and linefeed characters are translated to carriage return–linefeed combinations on output. … Open in binary (untranslated) mode; translations involving carriage-return and linefeed characters are suppressed.

With this change in place, let's try the tests that previously failed when the source file used LF for line endings:

D:\Src\rakudo> perl t\harness --moar t\spec\S02-literals\heredocs.t t\spec\S02-literals\heredocs.t .. ok All tests successful. Files=1, Tests=22, 1 wallclock secs ( 0.05 usr + 0.03 sys = 0.08 CPU) Result: PASS D:\Src\rakudo> perl t\harness --moar t\spec\S02-literals\quoting.t t\spec\S02-literals\quoting.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/184 subtests Test Summary Report ------------------- t\spec\S02-literals\quoting.t (Wstat: 256 Tests: 184 Failed: 1) Failed test: 126 Non-zero exit status: 1 Files=1, Tests=184, 4 wallclock secs ( 0.11 usr + 0.03 sys = 0.14 CPU) Result: FAIL

What happens if I do:

chcp 65001?

Everything works:

D:\Src\rakudo> perl t\harness --moar t\spec\S02-literals\quoting.t t\spec\S02-literals\quoting.t .. ok All tests successful. Files=1, Tests=184, 5 wallclock secs ( 0.06 usr + 0.03 sys = 0.09 CPU) Result: PASS

Since the test involves q:x and UTF-8, that counts as a full pass.

In fact, most of the tests I listed as failing now pass. And, the remaining failures seems to be unrelated to EOL conversions.

For example:

D:\Src\rakudo>perl t\harness t\spec\S32-io\IO-Socket-Async.t t\spec\S32-io\IO-Socket-Async.t.. Dubious, test returned 255 (wstat 65280, 0xff00) Failed2/6 subtests TestSummary Report ------------------- t\spec\S32-io\IO-Socket-Async.t(Wstat: 65280 Tests: 4 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 6 tests but ran 4. Files=1,Tests=4, 3 wallclock secs ( 0.08 usr + 0.02 sys = 0.09 CPU) Result:FAIL D:\Src\rakudo>perl t\harness t\spec\S32-io\IO-Socket-Async.t --verbosity=3 t\spec\S32-io\IO-Socket-Async.t.. 1..6 ok1 - Async listen on bogus hostname ok2 - Async connect to unavailable server breaks promise ok3 - Async connect to available server keeps promise ok4 - Echo server ok5 - Discard server ok6 - bytes-supply ok Alltests successful. Files=1,Tests=6, 3 wallclock secs ( 0.08 usr + 0.01 sys = 0.09 CPU) Result:PASS D:\Src\rakudo> perl t\harness t\spec\S32-io\IO-Socket-Async.t t\spec\S32-io\IO-Socket-Async.t .. ok All tests successful. Files=1, Tests=6, 2 wallclock secs ( 0.08 usr + 0.01 sys = 0.09 CPU) Result: PASS

Interesting and possibly related to RT#126992.

I am very happy at this point. Any other problems that remain seem to be of a practical, implementation related nature, and they don't all have to be addressed for an exciting Perl 6 launch.

The behind-the-scenes conversion of

in string literals to \r

sequences was a serious philosophical issue of making what has always been a property of streams a mandatory property of literal strings, and needed to be addressed. I appreciate the speed with which the issue was resolved.

I am looking forward to trying out the release version soon after Christmas. Perl6 developers are busy, working hard on putting various finishing touches. Good luck!

And, Merry Christmas!

PS: You can discuss this post on /r/perl.

PPS: For reference, see Perl6's newline conversion behavior seems to be by design and Newline translation in Perl6 is broken for a brief background on my realization of how newlines were being handled in Perl6 on Windows prior to this change.