Status update from the Reproducible Builds project

To: debian-devel-announce@lists.debian.org

Cc: reproducible-builds@lists.alioth.debian.org

Subject: Status update from the Reproducible Builds project

From: Mattia Rizzolo <mattia@debian.org>

Date: Sun, 23 Jul 2017 13:54:32 +0200

Message-id: <[🔎] 20170723115426.cvsxjnvf2bro6ytb@mapreri.org>

Mail-followup-to: reproducible-builds@lists.alioth.debian.org

Reply-to: reproducible-builds@lists.alioth.debian.org

Dear fellow developers, It has been a while[1] since we last wrote here to reach you all. Since then, we have made considerable progress which has been reported during DebConf 15 and 16 talks as well as other conferences around the world. However, for the sake of information preservation and clear communication we felt the need to write a newer report here. Scope ===== At the start of 2015 it was safe to say that Debian was fairly alone in the quest for reproducible builds, and a relevant number of developers were unconvinced by the effort's goals. Thankfully, this is not true anymore. Since then we managed to pull onboard several other projects[2], including noteworthy GNU/Linux distributions such as Arch Linux, Fedora, OpenSUSE, LEDE and Tails, as well as FreeBSD and NetBSD. In turn, a lot of upstream projects, including GNU itself, have started to better see the effort as a worthy one and work toward it. To improve cross-project communication and support we first set up a project-agnostic namespace at reproducible-builds.org, coupling it with a project-agnostic mailing list[3] that everybody wanting to contribute anything to the project is free to join. We have further plans for tests.reproducible-builds.org as a collaboration space for various projects. Progress ======== On our website[4] there are nice colourful graphs showing our progress in numerical terms. In particular, let us point to the stretch/amd64 graph[5]: since our slow start ~3 years ago we have been steadily improving the reproducibility of our archive, reaching a staggering 94% at the time of writing! [6] Accordingly, in buster there are currently ~1300 unreproducible packages, many of which already have a bug filed with a patch. We are asking you, package maintainers, to review the bugs we have filed in the previous 3 years, and upload said patches if you're happy with them. We are often sad to read that many of our bugs have gone unacknowledged for years. [7][8] Buildinfo files =============== We have been playing with .buildinfo files [9] for more than two years, and dpkg finally started producing them with version 1.18.11 (Nov 2016). Some weeks later dak started to store those files, and they are accessible to all DDs in mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo. There are 214791 unique .buildifo files at the time of writing. Our dreams for those files do not end there, however; we want those files be published and much more. You can see in [10] more details on these files as we thought of them, and the current format as implemented by dpkg is available in deb-buildinfo(5) [11]. We are currently blocked awaiting feedback from ftp-masters on this (see #763822 [12]). Please consider helping out if you can! We also have a proof of concept website reachable at https://buildinfo.debian.net that is aiming to collect .buildinfo files coming from everywhere; currently only our CI is systematically pushing .buildinfo files there, but there is another open bug for ftp-master to push the .buildinfo of officially built packages there as well (see #862073 [13]). Also note that the service is open to anyone without authentication (by design), so everybody could push their own .buildinfo file with a simple HTTP POST. There will also be a talk about .buildinfo at DebConf 17 [14]. Tools ===== As part of the effort, several tools have been and continue to be developed. The previous report mentioned diffoscope[15]: a lot of work went into it, and it's now being widely used also outside our project to improve quality assurance and not just as a "better" diff. In typical "Debian" style, a multitude of tools now exists for testing packages for reproducibility issues. Worth mentioning are debrepro(1) (from devscripts) and reprotest(1) (in its own package). `reprotest` in particular aims to be as agnostic as possible about how a build is performed, and aims to be used to test all kinds of builds, not only Debian packages. Debian Policy ============= We are in the process of making reproducibility of packages something properly documented in policy. Writing patches for policy is not easy, so we welcome input from everyone to be able to better consider all the needed facets. See bug #844431 [16] for it. Also, we wish to remind everyone that Debian Policy aims at documenting current practices, it's not a "stick" to impose new rules. That said, we believe reproducible builds to be among the best practices today. Contribute ========== As usual, we welcome all kinds of contributions! If you are a package maintainer, please check your package for reproducibility issues [17]. Several packages already have patches filed in the bug tracker; please review, apply and upload them. If there are no patches filed already, consider investigating the issue yourself; you'll discover most are really easy to fix. Additionally, talk to us and we'll help find a fix for you. Contacts ======== For Debian-related queries, we are reachable via mail at reproducible-builds@lists.alioth.debian.org and we hang out in IRC in #debian-reproducible on the OFTC network. For general purpose queries (not strictly related to Debian) our channels are rb-general@lists.reproducible-builds.org [3] and #reproducible-builds on the OFTC network. Thanks ====== We want to thank all the entities that helped our endeavour. In no particular order: * Profitbricks — https://profitbricks.com — for sponsoring the x86 VMs our CI runs on, not only for Debian, but also for other projects * Codethink — https://codethink.co.uk — for sponsoring the arm64 nodes * Core Infrastructure Initiative — https://coreinfrastructure.org — for sponsoring some us to work on the Reproducible Builds * Linux Foundation — https://linuxfoundation.org — for sponsoring our meetups and some more * Debian — https://debian.org — for sponsoring some of the armhf equipment and partially covering some expenses for our physical meetups ... and many others that helped us along the path. Also many thanks to all those of you that contributed to the project, and also those that simply merged our patches: all of you helped a lot! Best wishes, for the Reproducible Builds team, Mattia Rizzolo Chris Lamb Vagrant Cascadian Holger Levsen Jenkins [1] https://lists.debian.org/debian-devel-announce/2015/02/msg00007.html [2] https://reproducible-builds.org/who/ [3] https://lists.reproducible-builds.org/listinfo/rb-general [4] https://tests.reproducible-builds.org/debian/ [5] https://tests.reproducible-builds.org/debian/stretch/ [6] at a footnote level, that's a cheat, as the stretch builds happen without varying the build path :P That doesn't change that it's an impressive result considering where we started 3 years ago [7] https://tests.reproducible-builds.org/debian/stats_bugs_sin_ftbfs_state.png [8] https://tests.reproducible-builds.org/debian/stats_bugs_sin_ftbfs.png [9] https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles [10] https://wiki.debian.org/ReproducibleBuilds/BuildinfoInfrastructure [11] https://manpages.debian.org/unstable/dpkg-dev/deb-buildinfo.5.en.html [12] https://bugs.debian.org/763822 [13] https://bugs.debian.org/862073 [14] https://debconf17.debconf.org/talks/91/ [15] Renamed from "debbindiff" to remove the "deb" part, as it was made Debian-agnostic. [16] https://bugs.debian.org/844431 [17] https://tests.reproducible-builds.org/debian/unstable/index_dd-list.html -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `-