Linux distributions and Python 2

Did you know...? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

Python 2.7 will reach its end of life in less than two years—at least for the core development team. Linux distributions need to figure out how to handle the transition given that many of their users are still using that version of the language—and may still be well beyond the end-of-life date. Petr Viktorin and Matthias Klose led a session at the 2018 Python Language Summit to discuss distributions' approaches to deprecating Python 2.

Viktorin works for Red Hat and focused on the Fedora distribution. He wants to help figure out how to help the Python downstreams so that Python 2 can be fully discontinued. There are two different ways to do that; either make sure that everyone switches to Python 3 or simply deprecate Python 2 and "wash our hands" of the problem. He would prefer the first alternative. He will be working on this transition for Red Hat as part of his day job and would like to do it in the community as well; that will minimize the need to maintain Python 2 going forward.

Fedora now has about two-thirds of its packages that are Python 3 compatible. What is left is mostly stuff that no one cares about, he said, but there are still some important packages that need Python 2. There are also packages where the Python 2 version could be dropped in favor of one that works with Python 3, but the distribution does not know if that will upset users or not. In theory, the package maintainer should have some idea, but in many cases, they do not.

The list of things that have not been ported to Python 3 includes the Mercurial and Bazaar distributed version-control systems, plugins for tools like GIMP and Inkscape, and bindings for Samba and various database-management systems. In addition, OpenStack is not yet Python 3 ready.

Fedora is a community distribution and it can at times suffer from a lack of interested maintainers. Some think the model of the interested distribution package maintainer is outdated and that package repositories like the Python Package Index (PyPI) should be used instead. But distribution packagers still have their place, Viktorin said. For example, the licensing information on PyPI is sometimes wrong, which is often caught by the distribution maintainers. One other barrier for community distributions is that there is no real authority to direct the volunteer efforts; so the distribution can't really deprecate Python 2 directly.

In companies it is difficult to get funds allocated for refactoring working Python 2 code to Python 3, so that change is usually pushed by developers from below. On the other hand, advertising the use of Python 3 is useful for recruiting developers, or so he has heard from several sources.

One of the goals for Fedora in this process is to make sure that it doesn't make things harder for those users who have already switched to Python 3. The first step in the transition is to move leaf packages (i.e. those that have no other packages dependent on them) to Python 3. Those changes have to be tested to ensure there are no lingering Python 2 dependencies. If there are problematic packages, where it is hard to switch them over to Python 3, he wants to help get them working.

Fedora still needs to support Python 2.7 as long as it is needed, he said. It also supports Python 3.3, 3.4, and 3.5, but only for testing that packages and applications work with them. Łukasz Langa asked what "as long as needed" means for Python 2.7; Viktorin said that it is as long as someone is depending on it. That didn't sit entirely well; Langa said that people will be reporting bugs in 2.7 to the Python community even though it is past its end of life. But Klose pointed out that the situation is no different than today; "end of life" Python versions are still supported, especially by the enterprise distributions.

Nick Coghlan noted that RHEL 7 shipped with Python 2.7 and it will be supported until 2024. Viktorin said the problem is going to keep happening, no matter who is maintaining old Python versions; what he wants is for whoever does the maintenance to care about it.

There are some "little annoyances" with gradually shifting over to Python 3, however. PEP 394 says that executing " python " should currently invoke Python 2. It also says that " python " in the shebang (" #! ") line should mean that the script is compatible with both 2 and 3. However, maintainers of Python scripts may either be unaware of the PEP or just have not gotten around to switching, so the two "requirements" are at odds a bit. As detailed in a pull request to change the PEP, it would make sense to have no python symbolic link at all in some environments, especially in places where removing dependence on Python 2 is desired.

Langa suggested that perhaps switching the python symbolic link to Python 3 could be aligned with a long-term support (LTS) version of Python as was discussed in an earlier session. If, say, the 3.9 release were made in mid-2020 as an LTS, that might make a good point to change the recommendation in the PEP.

Klose took over at that point to give his perspective as a Debian and Ubuntu Python maintainer. For Ubuntu 16.04 LTS and Debian 9 ("stretch"), the python symbolic link points to Python 2. For Ubuntu 18.04 LTS, Python 2 is not part of the installation image. Invoking python will either remind users how to install Python 2 or will note the existence of Python 3 if it is installed.

There is ongoing work to get the distributions Python 3 ready. There are bug reports for things that do not work with Python 3, but they may be ignored if the bugs do not show up for Python 2, he said. What will not be so easily ignored is Lintian, which is a tool that finds problems in Debian packages. Adding Python 3 checks to Lintian should help reduce the number of problems.

Debian has made the decision to ship Python 2 with Debian 10 ("buster"), which will be released in mid-2019; that means Python 2 will be supported in Debian well past its 2020 end of life. For Ubuntu, the plan for the 20.04 LTS release has Python 2 removed from the main repository. That means it will not be supported by Canonical, so the community will need to pick it up if it is to continue at that point.