In July, when I replaced my dated MacBook Air with a shiny new 2017 MacBook Pro, it was my first workstation equipped with a blazing-fast NVMe SSD. At that time Apple was releasing developer and beta versions of their new macOS, High Sierra, which includes APFS – filesystem that is going to succeed an almost 2 decades old Mac OS Extended (HFS+). I did some benchmarks, that while being maybe not that well-thought, showed some disadvanteges of using Apple’s new filesystem on Apple’s new laptop, especially when it came to encrypted volumes.

This July post with HFS+ and APFS benchmarks gained some popularity in the last few days, when Apple released a stable/final version of High Sierra, so I decided to redo these tests on the same laptop and include some additional ones.

Testing environment:

Hardware: 2017 MacBook Pro A1706 with i5-7267U, 16 GB 2133 RAM and AP0256J NVMe disk AKA “MacBook Pro (13-inch, 2017, Four Thunderbolt 3 Ports)”

OS: macOS 10.13 High Sierra (release version, 17A365)

Software used: dd, ioping, Blackmagic Disk Speed Test, sysbench, blogbench, Bonnie++

$ ./ioping -v ioping 1.0.7.gec17b76 $ ./sysbench --version sysbench 1.0.9 $ ./blogbench --help|head -n2 blogbench 1.1 - Sep 26 2017 $ ./bonnie++ (...) Version: 1.97

All tests were performed in a “real” and not “laboratory” environment, meaning that I wasn’t creating a filesystem from flash-drive-booted Recovery Mode, running commands that did operations on it, only to repartition the disk with another filesystem and re-run the tests.

Each filesystem was tested on a clean, running macOS High Sierra and each test was performed a couple of minutes after rebooting the system (battery not disconnected) to avoid caching to take place. After each filesystem, before partition table wipe, all created files were removed and a file containing random data was written in their place.

During all the tests, no network was connected to minimize I/O of other processes and AC power was connected. Additionally, one Terminal.app window was opened for shell access.

Keeping HFS+…

The macOS High Sierra installer automatically converts HFS+ partitions to AFPS (at least in case of this NVMe SSD disk), so both HFS+ installs were started via the following command:

/Volumes/Install\ macOS\ High\ Sierra/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --converttoapfs NO --volume /Volumes/HFS

In this case “/Volumes/Install macOS High Sierra” is a mountpoint of a flash drive with the system installer and “/Volumes/HFS” is a mountpoint of a previously created, clean HFS+ partition, and “HFS” is the name of the partition. The command was run in a Terminal started in flash-booted installer. My post about keeping HFS+ while upgrading to macOS High Sierra has a more detailed info on this.

Benchmarks

dd

Yes, I did dd tests again. Are they reliable? Don’t think so, no but I’m putting them here if only for the comparison with beta ones. These tests were the weirdest part of benchmarking, as even on encrypted HFS+ I was sometimes hitting close to 10GB/s while reading a random-filled 10GB file after starting powered down laptop.

Seek rate

HFS+

$ ioping -c 1000000000 -R /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 2.63 M requests completed in 2.84 s, 10.0 GiB read, 924.0 k iops, 3.52 GiB/s generated 2.63 M requests in 3 s, 10.0 GiB, 875.4 k iops, 3.34 GiB/s min/avg/max/mdev = 0 ns / 1.08 us / 1.07 ms / 863 ns

HFS+ encrypted

$ ioping -c 1000000000 -R /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 2.92 M requests completed in 2.83 s, 11.1 GiB read, 1.03 M iops, 3.93 GiB/s generated 2.92 M requests in 3 s, 11.1 GiB, 972.9 k iops, 3.71 GiB/s min/avg/max/mdev = 0 ns / 970 ns / 1.25 ms / 911 ns

APFS

$ ioping -c 1000000000 -R /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 2.43 M requests completed in 2.86 s, 9.27 GiB read, 849.8 k iops, 3.24 GiB/s generated 2.43 M requests in 3.00 s, 9.27 GiB, 809.8 k iops, 3.09 GiB/s min/avg/max/mdev = 0 ns / 1.18 us / 75 us / 489 ns

APFS encrypted

$ ioping -c 1000000000 -R /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 2.47 M requests completed in 2.86 s, 9.42 GiB read, 864.3 k iops, 3.30 GiB/s generated 2.47 M requests in 3.00 s, 9.42 GiB, 823.3 k iops, 3.14 GiB/s min/avg/max/mdev = 0 ns / 1.16 us / 87 us / 502 ns

Filesystem latency

This is the average time in which a filesystem reacted to a 4K ‘ping’ during a 10 minutes long test. 20000 nanoseconds = 0.00002 seconds.

HFS+

$ ioping -c 600 -q /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 599 requests completed in 11.6 ms, 2.34 MiB read, 51.5 k iops, 201.4 MiB/s generated 600 requests in 9.98 min, 2.34 MiB, 1 iops, 4.01 KiB/s min/avg/max/mdev = 10 us / 19.4 us / 98 us / 5.88 us

HFS+ encrypted

$ ioping -c 600 -q /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 599 requests completed in 11.8 ms, 2.34 MiB read, 50.9 k iops, 198.8 MiB/s generated 600 requests in 9.98 min, 2.34 MiB, 1 iops, 4.01 KiB/s min/avg/max/mdev = 12 us / 19.7 us / 76 us / 5.34 us

APFS

$ ioping -c 600 -q /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 599 requests completed in 14.0 ms, 2.34 MiB read, 42.7 k iops, 167.0 MiB/s generated 600 requests in 9.98 min, 2.34 MiB, 1 iops, 4.01 KiB/s min/avg/max/mdev = 14 us / 23.4 us / 99 us / 4.63 us

APFS encrypted

$ ioping -RL /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 93.4 k requests completed in 2.99 s, 22.8 GiB read, 31.2 k iops, 7.61 GiB/s generated 93.4 k requests in 3.00 s, 22.8 GiB, 31.1 k iops, 7.60 GiB/s min/avg/max/mdev = 27 us / 32.1 us / 1.37 ms / 6.59 us

Sequential read

HFS+

$ ioping -RL /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 96.5 k requests completed in 2.99 s, 23.6 GiB read, 32.2 k iops, 7.87 GiB/s generated 96.5 k requests in 3.00 s, 23.6 GiB, 32.2 k iops, 7.85 GiB/s min/avg/max/mdev = 25 us / 31.0 us / 2.95 ms / 11.3 us

HFS+ encrypted

$ ioping -RL /tmp --- /tmp (hfs /dev/disk1) ioping statistics --- 118.6 k requests completed in 2.99 s, 29.0 GiB read, 39.6 k iops, 9.67 GiB/s generated 118.6 k requests in 3.00 s, 29.0 GiB, 39.5 k iops, 9.65 GiB/s min/avg/max/mdev = 21 us / 25.2 us / 1.30 ms / 5.92 us

APFS

$ ioping -RL /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 102.5 k requests completed in 2.99 s, 25.0 GiB read, 34.2 k iops, 8.36 GiB/s generated 102.5 k requests in 3.00 s, 25.0 GiB, 34.2 k iops, 8.34 GiB/s min/avg/max/mdev = 24 us / 29.2 us / 1.57 ms / 8.05 us

APFS encrypted

ioping -RL /tmp --- /tmp (apfs /dev/disk1s1) ioping statistics --- 93.4 k requests completed in 2.99 s, 22.8 GiB read, 31.2 k iops, 7.61 GiB/s generated 93.4 k requests in 3.00 s, 22.8 GiB, 31.1 k iops, 7.60 GiB/s min/avg/max/mdev = 27 us / 32.1 us / 1.37 ms / 6.59 us

sysbench

On each filesystem, 128 files (800 MB each, 100GB total) were created for sysbench tests.

Below are 5 minutes long tests on the files created in the preparation phase.