This document is the April, 2017 progress report for TPF Standardization, Test Coverage, and Documentation of Perl 6 I/O Routines grant

Timing

As proposed to and approved by the Grant Manager, I've extended the due date for this grant by 1 extra month, in exchange for doing some extra optimization work on IO routines at no extra cost. The new completion date is May 22nd; right after the next Rakudo compiler release.

Communications

I've created and published three notices as part of this grant, informing the users on what is changing and how to best upgrade their code, where needed:

IO Action Plan Progress

Most of the IO Action Plan has been implemented and got shipped in Rakudo's 2017.04.2 release. The remaining items are:

Implement better way to signal closed handle status (was omited from release due to original suggestion to do this not being ideal; likely better to do this on the VM end)

Implement IO::CatHandle as a generalized IO::ArgFiles (was omited from release because it was decided to make it mostly-usable wherever IO::Handle can be used, and IO::ArgFiles is far from that, having only a handful of methods implemented)

Optimization of the way we perform stat calls for multiple file tests (entirely internal that requires no changes to users' code)

Documentation and Coverage Progress

In my spreadsheet with all the IO routines and their candidates, the totals show that 40% have been documented and tested. Some of the remaining 60% may already have tests/docs added when implementing IO Action Plan or ealier and merely need checking and verification.

Optimizations

Some of the optimizations I promised to deliver in exchange for grant deadline extension were already done on IO::Spec::Unix and IO::Path routines and have made it into the 2017.04.2 release. Most of the optimizations that will be done in the upcoming month will be done in IO::Spec::Win32 and will largely affect Windows users.

IO Optimizations in 2017.04.2 Done by Other Core Members:

Elizabeth Mattijsen made .slurp 2x faster rakudo/b4d80c0

Samantha McVey made nqp::index—which is used in path operations—2x faster rakudo/f1fc879

IO::Pipe.lines was made 3.2x faster by relying on work done by Elizabeth Mattijsen rakudo/0c62815

Tickets Resolved

The following tickets have been resolved as part of the grant:

Possibly more tickets were addressed by the IO Action Plan implementation, but they still need further review.

Bugs Fixed

Fixed a bug in IO::Path.resolve with combiners tucked on the path separator. Fix in rakudo/9d8e391f3b; tests in roast/92217f75ce. The bug was identified by Timo Paulssen while testing secure implementation of IO::Path.child

IO Bug Fixes in 2017.04.2 Done by Other Core Members:

Timo Paulssen fixed a bug with IO::Path types not being accepted by is native NativeCall trait rakudo/99840804

NativeCall trait rakudo/99840804 Elizabeth Mattijsen fixed an issue in assignment to dynamics. This made it possible to temp $*TMPDIR variable rakudo/1b9d53

variable rakudo/1b9d53 Jonathan Worthington fixed a crash when slurping large files in binary mode with &slurp or IO::Path.slurp rakudo/d0924f1a2

Jonathan Worthington fixed a bug with binary slurp reading zero bytes when another thread is causing a lot of GC rakudo/756877e

Commits

So far, I've commited 192 IO grant commits to rakudo/roast/doc repos.

Rakudo

69 IO grant commits:

c6fd736 Make IO::Spec::Win32.is-absolute about 63x faster

7112a08 Add :D on invocant for file tests

8bacad8 Implement IO::Path.sibling

a98b285 Remove IO::Path.child-secure

0b5a41b Rename IO::Path.concat-with to .add

9d8e391 Fix IO::Path.resolve with combiners; timotimo++

1887114 Implement IO::Path.child-secure

b8458d3 Reword method child for cleaner code

51e4629 Amend rules for last part in IO::Path.resolve

9a2446c Move Bool return value to signature

214198b Implement proper args for IO::Handle.lock

c95c4a7 Make IO::Path/IO::Special do IO role

fd503f8 grant] Remove role IO and its .umask method"

0e36bb2 Make IO::Spec::Win32!canon-cat 2.3x faster

40217ed Swap .child to .concat-with in all the guts

490ffd1 Do not use self.Str in IO::Path errors

1f689a9 Fix up IO::Handle.Str

c01ebea Make IO::Path.mkdir return invocant on success

d46e8df Add IO::Pipe .path and .IO methods

6ee71c2 Coerce mode in IO::Path.mkdir to Int

0d9ecae Remove multi-dir &mkdir

ff97083 Straighten up rename, move, and copy

7f73f92 Make IO::Path.new-from-absolute-path private

da1dea2 Fix &symlink and &link

8c09c84 Fix symlink and link routines

90da80f Rework read methods in IO::Path/IO::Handle

c13480c IO::Path.slurp: make 12%-35% faster; propagate Failures

f1b4af7 Implement IO::Handle.slurp

184d499 Make IO::Handle.Supply respect handle's mode

b6838ee Remove .f check in .z

6a8d63d Implement :completely param in IO::Path.resolve

e681498 Make IO::Path throw when path contains NUL byte

b4358af Delete code for IO::Spec::Win32.catfile

0a442ce Remove type constraint in IO::Spec::Cygwin.canonpath

0c8bef5 Implement :parent in IO::Spec::Cygwin.canonpath

a0b82ed Make IO::Path::* actually instantiate a subclass

67f06b2 Run S32-io/io-special.t test file

954e69e Fix return value of IO::Special methods

a432b3d Remove IO::Path.abspath (part 2)

94a6909 Clean up IO::Spec::Unix.abs2rel a bit

966a7e3 Implement IO::Path.concat-with

50aea2b Restore IO::Handle.IO

15a25da Fix ambiguity in empty extension vs no extension

b1e7a01 Implement IO::Path.extension 2.0

b62d1a7 Give $*TMPDIR a container

099512b Clean up and improve all spurt routines

cb27bce Clean up &open and IO::Path.open

4c31903 Add S32-io/chdir-process.t to list of test files to run

5464b82 Improve &*chdir

2483d68 Fix regression in &chdir's failure mode

ca1acb7 Fix race in &indir(IO::Path …)

a0ef2ed Improve &chdir, &indir, and IO::Path.chdir

aa62cd5 Remove &tmpdir and &homedir

a5800a1 Implement IO::Handle.spurt

36ad92a Remove 15 methods from IO::Handle

87987c2 Remove role IO and its .umask method

9d8d7b2 Log all changes to plan made during review period

0c7e4a0 Do not capture args in .IO method

c360ac2 Fix smartmatch of Cool ~~ IO::Path

fa9aa47 Make R::I::SET_LINE_ENDING_ON_HANDLE 4.1x Faster

0111f10 Make IO::Spec::Unix.catdir 3.9x Faster

4fdebc9 Make IO::Spec::Unix.split 36x Faster

dcf1bb2 Make IO::Spec::Unix.rel2abs 35% faster

55abc6d Improve IO::Path.child perf on *nix

4032953 Make IO::Handle.open 75% faster

a01d679 Remove IO::Path.pipe

212cc8a Remove IO::Path.Bridge

76f7187 Do not cache IO::Path.e results

dd4dfb1 Fix crash in IO::Special .WHICH/.Str

Perl 6 Specification

47 IO grant commits:

3b36d4d Test IO::Path.sibling

7a063b5 Fudge .child-secure tests

39677c4 IO::Path.concat-with got renamed to .add

92217f7 Test IO::Path.child-secure with combiners

f3c5dae Test IO::Path.child-secure

a716962 Amend rules for last part in IO::Path.resolve

4194755 Test IO::Handle.lock/.unlock

64ff572 Cover IO::Path/IO::Pipe's .Str/.path/.IO

637500d Spec IO::Pipe.path/.IO returns IO::Path type object

8fa49e1 Test link routines

416b746 Test symlink routines

d4353b6 Rewrite .l on broken symlinks test

7e4a2ae Swap .slurp-rest to .slurp

a4c53b0 Use bin IO::Handle to test its .Supply

feecaf0 Expand file tests

a809f0f Expand IO::Path.resolve tests

ee7f05b Move is-path sub to top so it can be reused

b16fbd3 Add tests to check nul byte is rejected

8f73ad8 Change \0 roundtrip test to \t roundtrip test

7c7fbb4 Cover :parent arg in IO::Spec::Cygwin.canonpath

896033a Cover IO::Spec::QNX.canonpath

c3c51ed Cover IO::Spec::Win32.basename

d8707e7 Cover IO::Spec::Unix.basename

bd8d167 Test IO::Path::* instantiate a subclass

43ec543 Cover methods of IO::Special

e5dc376 Expand IO::Path.accessed tests

0e47f25 Test IO::Path.concat-with

305f206 Test empty-string extensions in IO::Path.extension

2f09f18 Fix incorrect test

b23e53e Test IO::Path.extension

1d4e881 Test $*TMPDIR can be temped

79ff022 Expand &spurt and IO::Path.spurt tests

ba3e7be Merge S32-io/path.t and S32-io/io-path.t

3c4e81b Test IO::Path.Str works as advertised

86c5f9c Delete qp{} tests

430ab89 Test &*chdir

86f79ce Expand &chdir tests

73a5448 Remove two fudged &chdir tests

04333b3 Test &indir fails with non-existent paths by default

bd46836 Amend &indir race tests

f48198f Test &indir

5a7a365 Expand IO::Spec::*.tmpdir tests

14b6844 Use Numeric instead of IO role in dispatch test

8d6ca7a Cover IO::Path.ACCEPTS

091931a Expand &open tests

465795c Test IO::Path.lines(*) does not crash

63370fe Test IO::Special .WHICH/.Str do not crash

Documentation

76 IO grant commits: