DragonFlyBSD Project Update - colo upgrade, future trends

For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone. The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system. Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while. Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get. The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time. -- Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going. John Marino (marino) pioneered the 'synth' system for dports and continues to help out, but his focus is on RavenPorts for now. Rimvydas Jasinskas (zrj) and Antonio Huete Jimenez (tuxillo) have done a huge amount of work bringing dports back into operational form and getting the FreeBSD ports sync stuff working better. DragonFlyBSD is still using synth, and will probably remain on synth for the foreseeable future, though there is always some discussion about how best to move dports forwards. It's an excellent build platform for us. Francois Tigeot (ftigeot) has done a ton of work taking DRM up to Linux-4.7.10 and this has worked very well for Intel iGPUs. We are now finally starting to dive into Linux's 'amdgpu' subsystem which is much older, in order to modernize our AMD support (which is still deficient). Numerous other people have spent a considerable amount of time helping test GPU support and tracking down bugs. The work is ongoing. I apologize for only writing everyone's names in plain ascii :-) Right now HAMMER2 makes for an excellent single-device filesystem (extremely well given that it supports writable snapshots and compression out of the box), but it remains deficient when it comes to expandable storage, multiple devices, and clustering. This will be active work for me but honestly the amdgpu support has to come first so it's still going to be a long-haul for HAMMER2. The mailing lists are not seeing much if any activity any more. This is more a generational issue... people kinda prefer web-based forums these days and younger generations do not use mailing lists at all for group stuff (not really). Even the devs almost universally use IRC and not mailing lists for discussions now (its kinda too bad that we don't have a permanent irc log stored on DFly servers for posterity). So we are looking into potentially shifting user interaction to a web-based forum, perhaps this year, and retiring the mailing lists, leaving just an archive for the mailing list. Possibly sometime this year, so look for action on that upcoming. Our other main developers: Sascha Wildner (swildner), who keeps the tree in tip-top shape and catches numerous issues, Justin Sherrill (JustinS) who handles our distribution rolls and the DragonFly and general BSD blog on our home page, Sepherosa Ziehau (sephe) who focuses on the network subsystem, Tomohiro Kusumi who helps with HAMMER and HAMMER2, Peter Avalos, and more. I will list more down below. -- It's hard to say how the future will develop. There are only three open-source operating systems in the entire world that really pull it together on having a complete, modern, SMP kernel: Linux, DragonFlyBSD, and FreeBSD. And that's it. We also have NetBSD and OpenBSD and I'd kinda like to know what their plans are, because the future is clearly going not only multi-core, but many-core. For everything. But as I like to say, for SMP there are only three at the moment. One can't dispute that Linux has nearly all the eyeballs, and DragonFly has very few. But OpenSource tends to live on forever and algorithms never die... I think there is a place for all of these projects and there really aren't any alternatives if you want a sparkling clean system that doesn't have too many layers of abstraction. At the current Juncture DragonFlyBSD is doing well and there are no plans to slow down or stop. There are many other developers who help out with DragonFlyBSD on a regular basis, or drop in from time to time, as well as past developers who did an awful lot of work. For this I am going to run the names out of the git log in alphabetical order, so I don't miss anyone (hopefully). And to 'User' and 'Charlie Root'... we will never know who you were, but the party is still going! -Matt Aaron LI Adam Hoka Adam Sakareassen Adrian Chadd Aggelos Economopoulos Alex Hornung Alexander Kuleshov Alexander Polakov Alexandre Perrin Antonio Huete Antonio Huete Jimenez Antonio Nikishaev Aycan iRiCAN Ben Woolley Bill Yuan Brad Hoffman Brills Peng Charlie Root Chris Pressey Chris Turner Chris Turner Chris Wilson Christian Groessler Constantine A. Murenin Daniel Bilik Dave Hayes David P. Reese David Rhodus David Shao David Xu Diederik de Groot Dimitris Papastamos Dylan Reinhold Ed Schouten Eirik Nygaard Eitan Adler Francis GUDIN Franco Fichtner Fran\xc3\xa7ois Tigeot Gregory Neil Shapiro Gwenio Hasso Tepper Hidetoshi Shimokawa Hiroki Sato Hiten Pandya Ilya Dryomov Imre Vadasz Imre Vad\xc3\xa1sz Imre Vad\xc3\xa1sz Jan Lentfer Jan Sucan Javier Alc\xc3\xa1zar Jean-S\xc3\xa9bastien P\xc3\xa9dron Jeffrey Hsu Jeremy C. Reed Jeroen Ruigrok/asmodai Joe Talbott Joerg Sonnenberger Johannes Hofmann John Marino Jordan Gordeev Joris Giovannangeli Justin C. Sherrill Levente Kurusa Liam J. Foy Lubos Boucek Magliano Andrea Markus Pfeiffer Matt Dillon Matteo Cypriani Matthew Dillon Matthias Rampke Matthias Schmidt Maurizio Lombardi Max Herrgard Max Herrg\xc3\xa5rd Max Okumoto Maxim Ag Michael Neumann Michael Neumann Michael Neumann Mihai Carabas Nicolas Thery Nicolas Thery Nolan Lum Noritoshi Demizu Nuno Antunes Nuno Antunes Peeter Peeter Must Peter Avalos Pierre-Alain TORET Robert Garrett Robin Hahling Rui Paulo Rumko Samuel J. Greear Sascha Wildner Scott Ullrich Sepherosa Ziehau Simon 'corecode' Schubert Simon Arlott Simon Schubert Simon Schubert Stathis Kamperis Sylvestre Gallon Thomas E. Spanjaard Thomas Nikolajsen Tim Tim Bisson Tobias Heilig Tomasz Konojacki Tomohiro Kusumi Ulrich Sp\xc3\xb6rlein User Venkatesh Srinivas Victor Balada Diaz Vishesh Yadav Walter Sheets YONETANI Tomokazu Yellow Rabbit Yonghong Yan Zach Crownover b86 dumbbell glebius hrs jkim minux rnoland sinetek zrj \xc3\x81kos Kov\xc3\xa1cs -Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20190722/780565e5/attachment.html>