Debian discusses how to handle 2038

Did you know...? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

At this point, most of the kernel work to avoid the year-2038 apocalypse has been completed. Said apocalypse could occur when time counted in seconds since 1970 overflows a 32-bit signed value (i.e. time_t ). Work in the GNU C Library (glibc) and other C libraries is well underway as well. But the "fun" is just beginning for distributions, especially those that support 32-bit architectures, as a recent Debian discussion reveals. One of the questions is: how much effort should be made to support 32-bit architectures as they fade from use and 2038 draws nearer?

Steve McIntyre started the conversation with a post to the debian-devel mailing list. In it, he noted that Arnd Bergmann, who was copied on the email, had been doing a lot of the work on the kernel side of the problem, but that it is mostly a solved problem for the kernel at this point. McIntyre and Bergmann (not to mention Debian as a whole) are now interested in what is needed to update a complete Linux system, such as Debian, to work with a 64-bit time_t .

McIntyre said that glibc has been working on an approach that splits the problem up based on the architecture targeted. Those that already have a 64-bit time_t will simply have a glibc that works with that ABI. Others that are transitioning from a 32-bit time_t to the new ABI will continue to use the 32-bit version by default in glibc. Applications on the latter architectures can request the 64-bit time_t support from glibc, but then they (and any other libraries they use) will only get the 64-bit versions of the ABI.

One thing that glibc will not be doing is bumping its SONAME (major version, essentially); doing so would make it easier to distinguish versions with and without the 64-bit support for 32-bit architectures. The glibc developers do not consider the change to be an ABI break, because applications have to opt into the change. It would be difficult and messy for Debian to change the SONAME for glibc on its own.

Moving forward with the 32-bit time values will, obviously, run into year-2038 problems, so it makes sense for a system like Debian to request 64-bit time support—except that there is a lot of code out there that will not work at all with the newer ABI. Some of it is in binary form, proprietary applications of various sorts (e.g. games), but there are plenty of problems for the open-source code as well. Bergmann scanned the libraries in Debian and "identified that about one third of our library packages would need rebuilding (and tracking) to make a (recursive) transition", McIntyre said. He outlined two ways forward that they had come up with.

The first is to rename libraries that need to be fixed to support 64-bit time values so that there could be two versions of them that could both be installed on a single system. That entails fixing a bunch of packages and rebuilding lots of code. "This effort will be *needed* only for the sake of our 32-bit ports, but would affect *everybody*."

The second is to decide which of the 32-bit architectures Debian supports will actually be viable in 2038 and to create new versions of those architectures with different names. There would be two versions of those architecture ports active for, probably, one release and users would not be able to simply upgrade from one to the other. But it would reduce the impact:

This would allow most of our developers to ignore the problem here (as 64-bit arches are not affected) and let a smaller number of people re-bootstrap with new ABIs with 64-bit time_t embedded.

He and Bergmann think the second option is the right way to go, but McIntyre was soliciting input from others in the project. Ansgar Burchardt replied that the i386 port, at least, probably should not even take the second path. It should simply continue using 32-bit time values.

So maybe just recommend people to move to 64-bit architectures and put 32-bit applications in a time namespace so they believe they are still in 2001 ;-) 32-bit architectures will probably still be useful in embedded contexts for a long time and there it might be easier to just change the ABI, but for a general-purpose distribution we start seeing more and more problems and I don't really see us supporting them as a full architecture in 10+ years.

If that is the chosen direction for i386, Russ Allbery suggested that the "cross-grading" feature be fully supported. Cross-grading would allow a Debian system to be upgraded to a new architecture, but is currently not for the faint of heart:

I'm sure I'm not the only one who is stuck with continuously-upgraded i386 hosts who has been wanting to switch but has been waiting until cross-grading is a little bit less scary.

But Burchardt is not convinced there will be enough i386 Debian systems to matter in even ten years. He showed some numbers from Debian's popularity contest (popcon) on i386 versus amd64 that suggest i386 will only make up around 0.2% of the total in ten years. "For just avoiding the Y2038 problem, i386 might sort itself out already without additional intervention." McIntyre thought that cross-grading support might make an interesting project for an intern, however.

There is work going on in glibc to provide both 32- and 64-bit interfaces simultaneously, Guillem Jover said, which could be used to provide a smoother transition without needing a SONAME change. Bergmann said that the glibc work is proceeding, but he was not sure that it would actually help with the transition that much; the problem is that on the scale of a whole distribution it "adds so much extra work and time before completion that there won't be many people left to use it by the time the work is done.;-)"

Jover pointed to the large-file support (LFS) transition as something of a model, though he cautioned that "the LFS 'transition' is not one we should be very proud of, as it's been dragging on for a very long time, and it's not even close to be finished". LFS allows applications to handle files with sizes larger than a 32-bit quantity can hold, which parallels the time_t situation. Jover suggested that fully enabling LFS as part of the time_t transition might make sense. Bergmann said that LFS support is "a done deal", as both glibc and musl libc require using 64-bit file sizes ( off_t ) when 64-bit time values are used.

Lennart Sorensen also saw parallels to the LFS transition, but Ben Hutchings thought otherwise:

LFS is a great example of how *not* to do it. 23 years on, we still have open bugs for programs that should opt in but didn't. Not every program needs to handle > 2 GiB files, but there are now filesystems with 64-bit inode numbers and they break every non-LFS program that calls stat(). Similarly, every program that uses wall-clock time will fail as we approach 2038, and the failure mode is likely to be even worse than with LFS as few programs will check for errors from time APIs.

YunQiang Su suggested a third option to consider, which would define 64-bit versions of all of the affected interfaces throughout the distribution, by adding a "64" for the new function and data structure names, then modify packages to use those interfaces over time. A deadline could be set of, say, 2030; after that, anything using the older interfaces would be considered to have a release-critical bug. McIntyre said that it would be technically possible to do so, but there are some major downsides:

The problem here is that we have many thousands of packages to work on, glibc up through other libs to all the applications that use them. It's invasive work, and likely to take a very long time. Since it's work that would *not* be needed for 64-bit architectures, we're also likely to see push-back from some upstreams. 2030 is also too late to fix the problem - people are already starting to see issues in some applications. We want to make a difference soon: the earlier that we have fixes available, the fewer broken systems will have to be fixed / removed / replaced later.

Most who commented in the thread seem to see i386 as a dying architecture, at least for Debian on the time scale being considered here (18 years). Florian Weimer summed it up this way:

My opinion (professional in this case, even) is that i386 users want compatibility with their binaries from 1998. Otherwise they would have rebuilt them for x86-64 by now. Under this worldview, i386 is for backwards compatibility with existing software. Users will want to run these old programs in a time namespace with shifted time, too.

Bergmann generally agreed with that assessment, but thought it made sense to see how things go for a different 32-bit architecture that probably needs to have a user space that supports 64-bit time_t : armhf. Once that work is done, it may be straightforward to apply it to i386 if that is deemed useful—or a decision could be made to phase out i386 sometime before 2038.

But Marco d'Itri wondered why the picture for armhf was different than that of i386. Bergmann pointed out that armhf is being used for a lot of embedded systems and that is likely to continue. McIntyre also noted that the Civil Infrastructure Platform is based on Debian and is often used with armhf. Bergmann gave a summary of some research he did on the use of the various Arm architecture versions, some of which he expects to still be in use, in 32-bit form, in 2038 and beyond:

In some deeply embedded systems, you'd be looking at installing a current version of Debian (because why not) and then running it for decades beyond the end of support without updates. While this is often no problem in the absence of attack vectors, the time32 problem means that a piece of industrial equipment may be created for a 40 year lifetime today and work flawlessly for the first 18 years before suddenly breaking. The sooner time64 gets supported in Debian, the more of them have a chance of surviving.

The consensus seems to be that the second option, a new architecture name that indicates 64-bit time support, is the right way to go and that armhf makes the most sense as a starting point. In his reply to Jover, Bergmann summed it up this way:

So far, armhf is the only Debian target architecture that I know needs this, so it seems best to focus the work on that one. Once the porting work is done and enough bugs are fixed, the other architectures can decide if they still care. If any new 32-bit architectures (rv32, arc?) get added, it would probably be sensible to start out with a time64 port.

McIntyre is hoping to get started on work soon, so that an armhf port for 64-bit time might perhaps be released with Debian 11 ("bullseye"), which is presumably coming in mid-2021. Bergmann also noted that Adélie Linux has been working on porting its user space for 64-bit times using musl libc; it has a list of open issues that were found:

Most of these are for packages that use low-level system calls directly rather than going through glibc, either for syscalls that don't have an abstraction (seccomp, futex, ...) or for implementing a runtime environment for a language other than C.