Summary of the Debian CD BoF at DC15

To: debian-devel-announce@lists.debian.org

Cc: debian-cd@lists.debian.org, debian-boot@lists.debian.org

Subject: Summary of the Debian CD BoF at DC15

From: Steve McIntyre <steve@einval.com>

Date: Tue, 15 Sep 2015 13:52:20 +0100

Message-id: <[🔎] 20150915125220.GA31487@einval.com>

Reply-to: debian-cd@lists.debian.org

[ Please note the cross-post and Reply-To ] Hi folks, As promised, here's a quick summary of what was discussed at the BoF session in Heidelberg. Apologies for the delay - it takes a while to write these up, especially when struck down with the traditional post-DebConf plague. :-( Thanks to the awesome efforts of our video team, the session is already online [1]. I've taken a copy of the Gobby notes too, alongside my small set of slides for the session. [2] Current state ============= We have a massive number of different images created using debian-cd: * CDs * DVDs * non-free netinsts (including non-free firmware packages) * non-free firmware bundles Live images created using live-build etc. from the debian-live team * including non-free images with firmware Openstack images using openstack-debian-images All of these are built on pettersson, our big central CD builder machine hosted by the nice folks at umu.se. Plans for images - no new CDs! ============================== I'm planning to basically stop making *CD* sets for new builds (testing/stretch onwards). What do I mean by that? We'll stop building full CD sets of all of Debian (up to 90 per architecture) for new releases, and testing. And we'll also kill the differing CD1 options for different desktops. We'll continue building DVD and BD images. DVD#1 will continue to be configured to fir on a 4GB USB stick. We'll *keep* building netinst images as a small handy image type that many people use, and we'll keep building CD sets and CD1 options for older releases (Wheezy and Jessie) - we're not going to change tack on those half-way through. Why are we doing this? Two main reasons: * Essentially, approximately nobody is using these images any more. The chances of anybody actually using (for example) mips CD43 is zero. There's no point in making these any more and wasting the time, effort and disk space. For those people who are still stuck on machines that for some reason can't use bigger media, the netinst will suffice for boot and then use the network or similar. For anybody actually writing CDs, blank DVD media would be so much cheaper today that it would be cheaper to use DVDs and buy a drive! * It's become less and less useful to install from CD in recent years. We've had an ever-present problem with the different desktop environments growing in size as they add more features. For the last couple of releases, the small subset of Gnome or KDE that would fit on a single CD is, at best, not a good representation of the desktop capabilities. At worst, it's been simply non-functional without adding an extra package source (more discs, or a network repository). If users are stuck booting off smaller media, then I'd recommend using the netinst to boot and then add more package sources later. Stretch d-i alpha 3 (just released this week) is the last build I expect to include CDs with. Live images have mostly been too large to fit on CD for a while anyway - no plans to change the set there. I'm open to making more different-sized image types so long as they make sense - ask! get.debian.org ============== With the change to not make CDs, using cdimage.debian.org as the name for our central distribution site is a little silly! After some consultation on the mailing lists for a better name, I've picked get.debian.org as a new name. For the sake of old links etc., the old name is not going to go away. But for new links, documentation etc. we should start linking to get.debian.org instead. For now, the main landing page at get.debian.org is still the same as that for cdimage.debian.org. We'll work on this shortly. Help would be lovely! Non-free images =============== As mentioned in the firmware talk[3], We make a number of *unofficial* non-free images alongside our normal installer and live images today. They're essentially the same as the normal images, but with non-free firmware included for those people who need it to make their hardware function correctly. We've deliberately not advertised these very much in the past, so much so that many people (including lots of DDs!) don't even know these exist. We're planning to advertise them some more, and make some detail changes in how we deal with firmware which will make things easier for people, but we will *not* be including non-free firmware on official Debian media any time soon. See the firmware discussion for more details. New image types =============== We've recently added official Debian Openstack images. We'd like to add more types of image in the future. We have the capacity to make more images available on get.debian.org, and we are currently specifying a new more powerful server which will make that better again. Currently I'm thinking of: * More cloud images (Google, Microsoft, Amazon, Profitbricks, ...), or using cloud-init or something similar to make single images work on all the different clouds. Several of the vendors already have "Debian" images, but we're not 100% sure of their provenance, If possible, it would be lovely to make official images on Debian-controlled machines so that people can trust them fully. * More live images, starting with some for Blends. Iain Learmonth already has been working on a Debian Hams image, and there are more coming, I'm sure. * Live images for non-x86 targets, where they make sense. Many won't make sense, due to difficulty in booting them, lack of memory, etc. But some will, and we're thinking of arm64 and (maybe) armhf as the first targets here. * Neil Williams has been putting a lot of effort into the vmdebootstrap tool as a generic way of making a wide variety of live-style images. A major new addition is UEFI support. \o/ Suggestions and comments from the BoF: * Add a simple live netboot image (kernel and initramfs) which will grab and extract a tarball into a tmpfs for general usage * In Riku's bootable images talk[4] we heard that there are more than 10(!) different image creation tools in common use, just within Debian. Most of the job is simple, but lots of the tools only do a small subset of the simple things! If you're interested in any of this (helping, or ideas for more images) please talk to us! Misc questions and discussion ============================= Q. Multi-architecture CDs - possible? A. Yes, but it depends massively on the architectures involved. We already do multi-arch amd64/i386 CDs/DVDs, and we used to have others: amd64/i386/powerpc, alpha/hppa/ia64 (the HP special!). There's scope to make more like this, so long as the architectures don't clash in terms of how their boot support is configured. Sector 0 is special on lots of different architectures, and some of them clash in terms of what they want in various locations in this boot sector. More important is how useful the resulting images are for users... :-) The m-a x86 netinst is nice as that way we can ship a single image/disc which will work on any PC. Didier Raboud has been hacking on getting auto-detect working again on that image. As/when/if we get another architecture (arm64?) with enough users, we may add that to the m-a netinst too. Q. What should we do to generate and test different cloud images? A. We need to get some testing infrastructure in place to do some basic QA on these. At the moment, the Openstack images seem to work OK (no bad feedback so far!), but we should be doing more. In terms of reporting issues, at the moment we don't have a specific place for that. Bug reports should probably go to the debian-cloud mailing list. Aside: we have a cdimage.debian.org BTS pseudo-package, and we clearly should add get.debian.org too. We should use that and redirect reports appropriately. We need to get some tests defined and running at build-time. Make sure that what we produce looks sane and is fit to be released. Maybe define standard tests for what's allowed to be called "Debian" in this area? Some discussion apparently happened in the "What should be allowed to call itself Debian" talk earlier", but there wasn't a clear agreement there. First test for cloud images - can we boot one of these images and ssh in? We have a framework for testing Debian installer CDs, but it's never been intergrated fully. :-( Similar story for live images - initial test implementation has happened, but it's not in use? "Packer" (I think) was mentioned software written in Go that can automate creation/testing(?) of various virtualisation image formats. Would need packaging for us to use it, needs volunteer(s). This whole area needs more help if it's going to happen in time for Stretch... Q. Can we tweak debian-cd to install a full 64-bit CD/system with *some* 32-bit support included, but not full? A. People have asked about this kind of thing multiple times in the past, but no code yet. debian-cd m-a support is currently designed to include exactly the same level of packages for all the architectures on a m-a CD. This could be hacked fairly easily to fix that - just include the base system and some libraries for secondary architectures. No time to look at this yet, but a wishlist bug would help! :-) UEFI ==== We just started a new UEFI team a few weeks back[6]. We're explicitly looking for bug reports from people. If UEFI does not just work, please report issues. There's a cross-distro effort now going on to track these so that we can share information and complain to manufacturers. Secure Boot is in progress still. We were hoping to get it working last year, but for various reasons it didn't happen yet. Hoping to get something soon, maybe by the end of the year. We'll be adding support for boot, validating kernel and modules etc.; at least that's the plan. Still open for debate as to exactly what we're going to do. We'd love to do SB on x86 and arm64 at the very least. We aim to have working SB support in d-i, debian-cd, kernel and bootloaders. Will derivatives be able to hook in? Honestly not sure - the devil is in the details here. How far does the trust chain extend? Will arm64 shim be signed by Microsoft? No idea - it's under discussion. kFreeBSD ======== UEFI (and SB) on kFreeBSD is an unknown quantity at this time. I've no idea how well it might work (or not). Help needed, happy to plumb it in in Debian if somebody can tell us how! We may get an unofficial kFreeBSD jessie CD release done - waiting on information at this point. Please help! ============ We're always looking for more help, and we're about to add even more stuff to our TODO list. Also, always looking for more help when testing images for release etc. Join us on the debian-cd list, or on #debian-cd. [1] http://meetings-archive.debian.net/pub/debian-meetings/2015/debconf15/Debian_CD_discussion.webm [2] http://www.einval.com/~steve/talks/Debconf15-CDs-are-dead/ [3] https://lists.debian.org/debian-devel/2015/08/msg00622.html [4] https://summit.debconf.org/debconf15/meeting/246/creating-bootable-debian-images/ [5] https://summit.debconf.org/debconf15/meeting/205/what-should-be-allowed-to-call-itself-debian/ [6] http://blog.einval.com/2015/08/02#new_debian_uefi_team -- Steve McIntyre, Cambridge, UK. steve@einval.com The two hard things in computing: * naming things * cache invalidation * off-by-one errors -- Stig Sandbeck Mathisen