Unexpected benefit with Ryzen - reducing power for home server

I received an unexpected benefit today when I checked for BIOS updates and found that my ASRock AB350M Pro4 Ryzen motherboard now supports ECC. I decided that it is finally time to upgrade my home server from the venerable E3-1270 (Xeon, 3.4GHz, 4-core/8-thread) to the 2700X (Ryzen, ~4.1GHz, 8-core/16-thread). In doing so, I am using ECC memory and the only EUDIMMs I have are 2133. At those memory frequencies, overclocking the cpu itself doesn't really add much benefit so I decided to test just how LOW a power envelope I could set and still retain full multi-threaded performance for compile jobs. That is, the system is limited by relatively low memory bandwidth so there's no need to overclock the cpu itself. It turns out... really low! Ryzen CPUs are very sensitive to memory bandwidth. To make these changes I tell the BIOS to use the AMD CBS settings (usually in the OC Tweaker menu), then go into the CBS settings at Advanced->CBS->NBIO->XFR 2.0 Configuration. I Enable XFR2, then set the PPT, TDC, and EDC limits. Set TDC and EDC high enough so they don't get in the way, then limit power consumption by adjusting PPT. By using the PPT limit instead of manually setting the CPU frequency, the motherboard gets the best of both worlds... it will idle just as low as it did before, it will still run one or two cores at full speed (~4.1 GHz), and it will ratchet down the frequency when all cores are loaded. Using a PPT limit with XFR2 is far, far superior to using manual OC frequency settings for the CPU. With standard XFR enabled in auto mode the 2700X will pull around 180W at the wall at full load. This might be useful if I had DDR4 3000 memory in it, but I don't, so there's no real need to pull that much power at full load. I was able to reduce this all the way down to 85W at full load without really impacting a concurrent -j 32 nativekernel NO_MODULES=TRUE test compile. Now, obviously, more compute-bound workloads will suffer a lot more, but I mostly do compiles and with 16 cpu threads compiles tend to be limited by memory bandwidth and not CPU frequency. NOTE! Some BIOSes these are in mW, others they are in W, double check! Incorrect settings can crowbar your PSU and/or blow up the CPU! The below tests are with XFR2 enabled with a PPT power envelope limit set. XFR2 controlls the CPU frequency (rather than using manual OC settings for the CPU frequency). time make -j 32 nativekernel NO_MODULES=TRUE >& /tmp/bk.out PPT 100000 (160W) - 1:06 (3.6 GHz at full load, 1-core 4.2 GHz) PPT 75000 (135W) - 1:07 (3.4 GHz at full load, 1-core 4.2 GHz) PPT 65000 (115W) - 1:08 (3.2 GHz at full load, 1-core 4.1 GHz) PPT 55000 (90W) - 1:09 (3.0 GHz at full load, 1-core 4.1 GHz) PPT 50000 (85W) - 1:10 (2.8 GHz at full load, 1-core 4.0 GHz approx) (TDC and EDC set to 300000 to get them out of the way) I settled on running the server with a PPT of 65000. That seems to be a reasonable trade-off and its a huge benefit to power costs. I've noticed similar behavior on the Threadripper 2990WX (the 32-core/64-thread beast)... there's just no need to run the TR2990WX at 330W. 225W or 250W is just fine for multi-threaded workloads. -- This is *very* impressive efficiency. Whod a thought that one would be able to run an 8-core/16-thread CPU at full load at only 85W and still reap most of the benefit in a memory-heavy workload! Of course, in the server space, we've known for a long time that maximum efficiency occurs with a high number of cores running at lower frequencies, and that efficiency trumps performance on machines with high core counts. But I never considered that the consumer Ryzen CPUs could also benefit from the same thing until now. Now that consumer CPUs are starting to get up there in core-count, it does make sense. AMD has a real winner with their XFR2 feature not only for overclocking, but also for reducing the power envelope. Not to mention supporting ECC on all of their CPUs, consumer and server (which many Ryzen consumer mobos now support as demand has increased for the feature). -Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20180902/865c5489/attachment.html>