Last week, we looked at some of the numbers that define the OpenStack community and the projects that they have built. But how do these numbers stack up against other large open source projects, and more importantly, what do they tell us about OpenStack? What can be learned from the successes and failures of other projects of a similar scale?

At the OpenStack Summit in Atlanta last month, I attended a talk given by Parallels's CTO of Server Virtualization James Bottomley that attempted to give some meaning to these numbers. Bottomley has experience with both projects, as the maintainer of the SCSI subsystem in the Linux kernel along with his dayjob working with OpenStack and related technologies at Parallels.

The Linux kernel and OpenStack are surprisingly close in some measurements; in February, OpenStack surpassed the kernel in terms of number of commits per month for the first time, and although variable, these numbers track fairly closely. So given their similar size, how do the projects differ? Here are four of the examples Bottomley gives.

Process

The process for submitting code and getting those changes accepted is somewhat different. Both projects have formal documentation, but in the kernel, unlike OpenStack, it is up to the individual maintainers of various portions of the code to enforce these rules. The result is an uneven application of the rules, for better or for worse. To some extent, a more lengthy formal process might dissuade new contributors or make submitting minor fixes more tedious, but it comes with obvious benefits in the form of predictability and fairness.

Testing

OpenStack's testing program offer an example that the Linux kernel has never had previously, but the kernel has recently begun efforts to emulate OpenStack's model under a project supported by the Linux Foundation. The extensive testing infrastructure of OpenStack relies on Jenkins, tied to the review process, which can't be exactly copied because of differences in the review process. But steps to emulate the concepts, if not the tools, are being made.

Reviews

Like other areas, OpenStack has a more formal process. But formality or lack thereof is not the major issue. Rather, we see that both projects suffer from a common issue: there are not enough reviewers in place to keep commits from piling up. Attracting and maintaining qualified and dedicated reviewers has been an issue for both projects. Related, both project have difficulties with bikesheeding, or the creep of arbitrary code refactor requests into the review process. It's not a tool issue, but a social one, and lessons learned in this area would benefit both projects.

Release cycles

The kernel has roughly twice as fast of a release cycle as OpenStack. In the kernel's case, there are roughly 2-3 month release cycles containing a two week merge period with six to ten week of stabilization work. OpenStack's cycle is six months, made up by a four week planning window, 14 weeks of code merger, and six weeks dedicated to stabilization. The result? Faster releases for the kernel, but perhaps less significant changes per release.

Want to learn more more? The complete video is 37 minutes and well worth the watch.