A Gilectomy update

Please consider subscribing to LWN Subscriptions are the lifeblood of LWN.net. If you appreciate this content and would like to see more of it, your subscription will help to ensure that LWN continues to thrive. Please visit this page to join up and keep LWN on the net.

In a rather short session at the 2018 Python Language Summit, Larry Hastings updated attendees on the status of his Gilectomy project. The aim of that effort is to remove the global interpreter lock (GIL) from CPython. Since his status report at last year's summit, little has happened, which is part of why the session was so short. He hasn't given up on the overall idea, but it needs a new approach.

Gilectomy has been "untouched for a year", Hastings said. He worked on it at the PyCon sprints after last year's summit, but got tired of it at that point. He is "out of bullets" at least with that approach. With his complicated buffered-reference-count approach he was able to get his "gilectomized" interpreter to reach performance parity with CPython—except that his interpreter was running on around seven cores to keep up with CPython on one.

The old adage of "fast, cheap, good; pick any two" has a parallel in the world of the Gilectomy (and other similar projects). In that case the three items are: "high performance", "doesn't break the C API", and "uses multiple cores". CPython doesn't use multiple cores and Gilectomy 1.0 is not high performance, which leads him to consider breaking the C API.

For "Gilectomy 2.0", Hastings will be looking at using a tracing garbage collector (GC), rather than the CPython GC that is based on reference counts. Tracing GCs are more multicore friendly, but he doesn't know anything about them. He also would rather not write his own GC.

This new version of Gilectomy would also have "cheap local object locking". It would distinguish between objects that are only visible in one thread versus those that are visible to two or more. Objects can transition from local to non-local in various ways, some of which are not particularly obvious. It will be difficult to identify all of those, so it is not a change he makes lightly.

His next step will be to get rid of the existing code. He hopes to be able to gain access to the Instagram runtime discussed in the previous summit session and to gilectomize that. Though the code has not been released, he may be able to arrange an agreement with Facebook/Instagram to gain access to the code, he said. Since that version of the interpreter did not break the C API (at least for extensions that Instagram uses), it may mean that Gilectomy 2.0 will actually not have to break that API.

But Thomas Wouters pointed out that there are things that the Instagram experiment is doing that will immediately break big and important Python extensions like NumPy and SciPy. Hastings is a bit sanguine about that: if Gilectomy ever ships in CPython, the core developers can go on vacation for a month and the extension developers will have it all fixed by the time they return. In truth, it would be a great problem to have, but he is so far away from being successful with his Gilectomy efforts that he can't even see it from where he is, he said.

One audience member said that it would be nice to be able to see the Instagram experimental code to determine just what the C API compatibility issues are. Hastings agreed but noted that there was no way to make that happen until the company is ready to do so. There was also a question of how this work fits with the subinterpreter effort. Hastings said that he saw no reason that the Gilectomy and subinterpreters would not play well together.