About a year ago I noticed various benchmarks such as running specs were running abnormally slow on macOS. Noah Gibbs started some basic investigation but did not come up with anything definitive.

Since then I crafted a version of Discourse Bench (used at: https://rubybench.org/) that is there to profile “performance” of local development machines.

It is a simple benchmark that measures performance of various tasks you are likely to run into in development:

Booting Rails

Running a spec

Visiting a very common page

Creating a topic

Converting text to HTML via our markdown library

It also contains some micro benches to help isolate culprits including:

PBKDF2 calculation

Filesystem random read access

The benchmark lives here, anyone can run it.

git clone https://github.com/SamSaffron/discourse-dev-benchmark.git bundle ./bench

We have run the benchmark on many of our development machines and noticed a trend. The same CPU tends to perform a lot better (especially boot) on either Linux native or WSL2 (virtualized Linux on Windows) than it does on MacOS

Additionally, I took the time to run the same benchmark on a dual booting Mac that I have. Here are some examples of the benchmarks we got (abbreviated, cause we have many benches):

Booting Rails

In this benchmark we simply run rails r Post.first .

Desktop - 9900KF - nvme - Arch | 1.375 Desktop - 9900KF - nvme - WSL2 | 1.407 Desktop - Ryzen 3900X - NVME - Ubuntu | 1.417 Lenovo P1 - Gen 2 - 9850H - WSL2 - Ubuntu | 1.433 Laptop - XPS 15 - 9880HK - Linux 5.5.8 | 1.439 Lenovo P1 - Gen 2 - Arch Native - 9850H | 1.539 MacBook Pro (16-inch, 2019) - Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz | 2.51 MBP i7-3615QM, Windows WSL 2 (2.3 quad mid 2012) | 2.52 MBP i7-3615QM, MacOS Catalina (2.3 quad mid 2012) | 3.953

desc seconds variance percent Desktop - i9-9900K Linux Mint 1.356 (±1.7%) 100.0% Desktop - 9900KF - Asus Prime Z370 - nvme - Arch 1.375 (±1.0%) 101.4% Desktop - 9900KF - Asus Prime Z370 - nvme - WSL2 1.407 (±2.7%) 103.76% Desktop - Ryzen 3900X - NVME - Ubuntu 1.417 (±0.4%) 104.5% Lenovo P1 - Gen 2 - 9850H - WSL2 - Ubuntu 1.433 (±1.2%) 105.68% Laptop - XPS 15 - 9880HK - Linux 5.5.8 1.439 (±0.6%) 106.12% Lenovo P1 - Gen 2 - Arch Native - 9850H 1.539 (±1.0%) 113.5% Desktop: AMD Ryzen 3950X, Arch, NVMe, ruby 2.6.5 (rvm) 1.728 (±3.2%) 127.43% Oryx Pro - i7-9750H CPU @ 2.60GHz, 32GB, NVMe. 1.787 (±4.5%) 131.78% MacBook Pro (16-inch, 2019) - Intel® Core™ i9-9980HK CPU @ 2.40GHz 2.51 (±2.4%) 185.1% MBP i7-3615QM, Windows WSL 2 (2.3 quad mid 2012) 2.52 (±0.3%) 185.84% MBP i7-9750H @ 2.6GHz (15", 2019, macOS Catalina) 3.046 (±1.7%) 224.63% Lenovo P1 - Gen 2 - 9850H - WSL1 - Ubuntu 3.06 (±0.9%) 225.66% MBP i7-3615QM, MacOS Catalina (2.3 quad mid 2012) 3.953 (±0.1%) 291.52%

Observations

An ancient 2012 MacBook Pro with WSL2 boots Discourse almost as fast as the state of the art MacBook Pro!

9880HK CPU boots about 80% faster on Linux than it does in macOS

WSL1 is slower on boot, but can have adequate performance if you enable bootsnap (which we do)

9900K by intel is the single thread king, beating both the 3900X and 3950X by AMD

3950X though on paper faster than the 3900X failed to beat any of our 3900X based computers on single thread benches despite the 2x price tag.

Top of the range laptops perform Rails boot very close to decked up desktops

We stopped benching on rbenv based systems, everyone moved to chruby or rvm cause the shims rbenv adds introduce significant delays on boot.

Running the Discourse Post spec

In this benchmark we run: bin/rspec spec/models/post_spec.rb --seed 1 , it is one of our larger spec files and is pretty representative.

Desktop - i9-9900K Linux Mint | 9.907 Desktop - 9900KF - nvme - Arch | 9.994 Desktop - Ryzen 3900X - NVME - Ubuntu | 10.524 Laptop - XPS 15 - 9880HK - Linux 5.5.8 | 10.74 Lenovo P1 - Gen 2 - Arch Native - 9850H | 11.346 Desktop - 9900KF - Asus Prime Z370 - nvme - WSL2 | 11.748 Desktop: AMD Ryzen 3950X, Arch, NVMe, ruby 2.6.5 (asdf) | 11.809 Lenovo P1 - Gen 2 - 9850H - WSL2 - Ubuntu | 12.308 Oryx Pro - i7-9750H CPU @ 2.60GHz, 32GB, NVMe. | 12.557 MacBook Pro (16-inch, 2019) - Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz | 12.963 MBP i7-9750H @ 2.6GHz (15", 2019, macOS Catalina) | 16.173 Lenovo P1 - Gen 2 - 9850H - WSL1 - Ubuntu | 17.598 MBP i7-3615QM, MacOS Catalina (2.3 quad mid 2012) | 23.396 MacBook Air (13-inch, 2018) - i5 8210Y | 23.569 MBP i7-3615QM, Windows WSL 2 (2.3 quad mid 2012) | 23.738

desc seconds variance percent Desktop - i9-9900K Linux Mint 9.907 (±0.6%) 100.0% Desktop - 9900KF - Asus Prime Z370 - nvme - Arch 9.994 (±0.5%) 100.88% Desktop - Ryzen 3900X - NVME - Ubuntu 10.524 (±0.4%) 106.23% Laptop - XPS 15 - 9880HK - Linux 5.5.8 10.74 (±2.4%) 108.41% Lenovo P1 - Gen 2 - Arch Native - 9850H 11.346 (±0.0%) 114.53% Desktop - 9900KF - Asus Prime Z370 - nvme - WSL2 11.748 (±0.5%) 118.58% Desktop: AMD Ryzen 3950X, Arch, NVMe, ruby 2.6.5 (asdf) 11.809 (±0.5%) 119.2% Lenovo P1 - Gen 2 - 9850H - WSL2 - Ubuntu 12.308 (±2.0%) 124.24% Oryx Pro - i7-9750H CPU @ 2.60GHz, 32GB, NVMe. 12.557 (±2.5%) 126.75% MacBook Pro (16-inch, 2019) - Intel® Core™ i9-9980HK CPU @ 2.40GHz 12.963 (±1.7%) 130.85% Mac Mini i7-8700B @ 3.2 GHz (2018, Ubuntu 19.10, USB3 SSD) 13.971 (±11.3%) 141.02% MBP i7-9750H @ 2.6GHz (15", 2019, macOS Catalina) 16.173 (±1.0%) 163.25% Lenovo P1 - Gen 2 - 9850H - WSL1 - Ubuntu 17.598 (±0.6%) 177.63% MBP i7 2.6 GHz, 15’’ 2018, Catalina 16Gb RAM 20.922 (±2.1%) 211.18% MBP i7-3615QM, MacOS Catalina (2.3 quad mid 2012) 23.396 (±1.7%) 236.16% MBP i7-3615QM, Windows WSL 2 (2.3 quad mid 2012) 23.738 (±0.6%) 239.61% (Docker) MBP i7-8559U (13-inch, 2018) 74.232 (±4.0%) 749.29%

Observation

Even if we compensate for the 1 second slower boot, performance of the 9980HK under macOS is still 10% or so slower than Linux using the same CPU

Again, to of the range laptops are very very close to top of the range desktops for running a single spec

Docker on Mac is a world of pain when it comes to developing Rails

WSL2 and Catalina on the same machine have similar performance running our spec file. It seems that in a steady state after the slow boot, Mac can beat virtualized Linux performance but remains slower than non virtualized Linux.

Random Read

In this benchmark we read 10 random files that are 10_000 bytes long from a directory containing 5000 files.

This benchmark really highlights the slowness of WSL1 and MacOS

Lenovo P1 - Gen 2 - Arch Native - 9850H | 15745.077 Lenovo P1 - Gen 2 - 9850H - WSL2 - Ubuntu | 14991.689 Lenovo P1 - Gen 2 - 9850H - WSL1 - Ubuntu | 1270.476

Laptop - XPS 15 - 9880HK - Linux 5.5.8 | 16956.501 Laptop XPS15, Ubuntu 18.04.4 w/KDE Plasma (2.4GHz i9-9980HK, 32GB RAM) - V2 | 15997.256 MacBook Pro (16-inch, 2019) - Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz | 3776.437

Raw data omitted cause graphs tell the story. It appears across the board, reading lots of smallish files randomly is significantly slower on macOS which may be a big reason for the much slower boot times.

Does anyone have any ideas how we could get boot and performance on MacOS improved?

Any other thoughts here?