PostgreSQL reconsiders CommitFests

Did you know...? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

Patch review is a perennial problem in free-software projects. There have been various development processes adopted to try to deal with that problem, but most projects still struggle with it. For PostgreSQL, regularly scheduled CommitFests are used to manage patch review, but recently that process has not been working that well. So the project is discussing alternatives.

Bruce Momjian summarized the problems on December 11, but the conversation had already started in another thread. The CommitFest process is supposed to allow PostgreSQL core contributors to have time to work on their own features in between focused bouts of patch review. That patch review phase is the core of the CommitFest and it typically runs for a month. But, instead of that month-long effort, there has effectively been a continuous CommitFest going since August. Momjian posits various ideas about how things have changed such that the process is no longer working:

1 more new developers involved in contributing small patches

2 more full-time developers creating big patches

3 increased time demands on experienced Postgres developers These changes are all driven by increased Postgres popularity. While the commitfest process hasn't changed much and was very successful in the first few years, a few things have changed externally:These changes are all driven by increased Postgres popularity.

There is some quibbling about whether the number of patches is actually increasing, but there seems to be a clear consensus that the overall complexity of the set of patches per CommitFest does increase over time. It's a common "complaint" that we see in other projects as well. Typically, the easier things have already been done, so it is the "hard stuff" that is left. Those kinds of patches require the attention of core team members, but the numbers of those kinds of reviewers has not kept pace. As Momjian put it: "We have always known that we can create novice developers faster than experienced ones, but it seems this difference is accelerating."

Part of the problem with CommitFests that never end is that the CommitFest managers (CFMs) are not able to conclusively declare them over and defer any remaining patches to the next. According to Josh Berkus, CFMs are burning out, at least partly because they aren't able to finalize CommitFests:

Every CFM who has taken forceful measures to close out the CF on time has gotten a large amount of flack for it, and absolutely zero back-up from the committers or the core team. This is why David, Robert and I will no longer manage CFs (and probably others as well). The CFM is expected to miraculously end the CF on time, without bouncing or forwarding unreviewed patches, cutting off patches which already had one round of review, bouncing out or holding patches from contributors who don't participate in the review process, nagging anyone too much, or taking any other steps which anyone might take exception to.

Robert Haas piled on, calling the CFM job, especially for the last CommitFest before a release, "roughly equivalent to having your arm boiled in hot oil". He concluded that the PostgreSQL community is not truly interested in enforcing any discipline on the CommitFests after he was unable to get any support for doing so when he was a CFM. In the end, it doesn't really matter why the patches aren't getting reviewed, he said:

I think the problem we need to solve here is that there are just not enough senior people with an adequate amount of time to review. Whether it's because the patches are more complex or that there are more of them or that those senior people have become less available due to other commitments, we still need more senior people involved to be able to handle the patches we've got in a timely fashion without unduly compromising stability. And we also need to do a better job recruiting and retaining mid-level reviewers, both because that's where senior people eventually come from, and because it reduces the load on the senior people we've already got.

But Tom Lane didn't think reviews should be limited to senior developers: "Also, one part of the point of the review mechanism is that it's supposed to provide an opportunity for less-senior reviewers to look at parts of the code that they maybe don't know so well, and thereby help grow them into senior people." On the other hand, Berkus claimed that "inexpert reviews" were strongly discouraged by many of the senior contributors. But others in the thread saw things a little differently.

It turns out that there is a question about whether (and how) to credit reviewers in the release notes for a particular PostgreSQL release. The project being unwilling to credit them led Berkus to conclude that new reviewers aren't welcome but, again, others disagreed somewhat. Regardless, there certainly seems to be a consensus that there aren't many new reviewers coming along—certainly not as many as are needed.

That's a problem that many projects have, once again. No universal solution—or even one that works for some projects—has been found, seemingly. It can be "resolved" in several different ways (e.g. less review of patches, a general slowdown in the pace of development), but none are particularly satisfactory.

One interesting proposal was made by Peter Eisentraut, who essentially suggested doing away with CommitFests entirely. There would effectively be a single CommitFest that ran for the length of the development cycle, more or less:

As far as I'm concerned, we might as well just have one commitfest per major release. Call it a patch list. Make the list sortable by created date and last-updated date, and let the system police itself. At least that's honest.

The reaction was generally favorable. In some ways, it would be a return to how things were done before the CommitFest era, but with better tracking of patches. Patches getting lost was one of the original reasons for the CommitFest process. There are some alternate suggestions (such as David Johnston's commit manager + commit reviewer idea) that could also be incorporated. There were also ideas floated about using Git in a more "Git-like" way in the development process, which were met with a mixed reaction.

One of the nice things about free-software development is that it is done in the open. That means projects can easily observe and learn from the struggles of others. While each project has its own idiosyncrasies, there is often enough commonality that ideas from other projects can be fully or partially adopted. Some of the problems that PostgreSQL is struggling with here sound much like those that the Linux kernel also wrestles with. Whether ideas can flow in one direction or the other remains to be seen.