A proposed Subversion vision and roadmap

From: "C. Michael Pilato" <cmpilato-AT-collab.net> To: Subversion Development <dev-AT-subversion.apache.org> Subject: Subversion Vision and Roadmap Proposal Date: Fri, 02 Apr 2010 11:28:30 -0400 Archive-link: Article, Thread

Last week, five of Subversion's developers -- myself, Hyrum Wright (who helped me author this lengthy mail), Greg Stein, Stefan Sperling, and Karl Fogel ("we", henceforth) -- met in New York City to evaluate Subversion's current trajectory as a project and to jointly develop suggestions for any course correction deemed necessary. The unfortunate impetus for the meeting was feedback from representatives of some very large installations of Subversion that, from where they sat, the Subversion project was stagnating. This is a somewhat fair assertion. It's not that the community is or has ever been in danger of doing nothing. But even when we march in step towards a goal, we don't do a good job of communicating outwardly about that fact, leading to the appearance of inactivity. And lately, our ongoing efforts have been less like a team of folks working in concert towards a common vision, and more like the individual piddlings common to mature open source software. Make no mistake: Subversion is mature open source software. It works well for open source projects and -- in one of the biggest software coups of the last decade -- we've found that it's also good enough to supplant other well-established version control systems, open-source or otherwise, inside the highly structured walls of the enterprise. Unfortunately, "good enough" still leaves the enterprise user base frustrated at the most inopportune times (like, say, when managing branches near releases). And if you judge success by mindshare, it's now clear the DVCS tools have rendered our "good enough" somewhat obsolete when it comes to serving many open source projects, even. VISION Subversion has no future as a DVCS tool. Let's just get that out there. At least two very successful such tools exist already, and to squeeze another horse into that race would be a poor investment of energy and talent. What's more, huge classes of users remain categorically opposed to the very tenets on which the DVCS systems are based. They need centralization. They need control. They need meaningful path-based authorization. They need simplicity. In short, they desperately need Subversion. It's this class of user -- the corporate developer -- that stands to benefit hugely from what Subversion brings to the party. And it's that class of user which we believe Subversion should cater to, ideally without ostracizing the open source volunteers who remain our largest source of development investment. Corporate developers come in all shapes and sizes, from self-administering 10-person teams to geographically-distributed organizations of thousands of developers on hundreds of teams serviced by dedicated IT departments. Subversion can't be all things to all people, but it can be a well-defined subset of things to most people. It shouldn't sacrifice its trademark simplicity when growing the features most likely to be used in large-scale deployments; it shouldn't optimize for the large enterprise in a way that will alienate the long tail composed of much smaller deployments. By defining the subset of things Subversion is and those it is not, we recognize our own boundaries and prevent years of wandering through the proverbial wilderness of feature creep. Someone wiser than I once said, "Where there is no vision, the people perish." So recognizing the benefits that Subversion already offers, and projecting a bit into the future what we'd like to see Subversion become, we offer the following vision statement for your review: Subversion exists to be universally recognized and adopted as an open-source, centralized version control system characterized by its reliability as a safe haven for valuable data; the simplicity of its model and usage; and its ability to support the needs of a wide variety of users and projects, from individuals to large-scale enterprise operations. A shorter, business-card-sized motto (offered as a replacement to the obsolete "A compelling replacement for CVS") might be: "Enterprise-class centralized version control for the masses". ROADMAP With that vision in mind, we identified a number of high-value improvements which Subversion should gain in coming releases. Then we took a casual pass at assigning some technical "plumbing" dependencies for each of these. Here's what we came up with, in the form "FEATURE: [DEPENDENCY ...]", where "FS-NG" = major FS backend overhaul, "WC-NG" = WC-NG, and "Ev2" = svn_editor_t: * Obliterate: FS-NG * Shelve/Checkpoint: WC-NG * Repository-dictated Configuration: WC-NG (?) * Rename Tracking: WC-NG, Ev2, FS-NG (?) * Improved Merging: WC-NG * Improved Tree Conflict Handling: WC-NG, Ev2, Rename Tracking * Enterprise Authentication Mechanisms: * Forward History Searching: FS-NG (?) * Log Message Templates: Repository-dictated Configuration By examining this dependency chain, factoring in the development momentum which will exist around WC-NG, and admitting that some of the major plumbing overhauls not currently underway may prove to be just as costly as the WC-NG work (if not more so), we believe that the following feature roadmap is one which will serve Subversion users well: 1.7: WC-NG; HTTPv2; 'svn patch'; typical slew of various bug fixes 1.8: repository-dictated configuration; tree conflicts improvements; WC-NG-enabled stuff (rename tracking, compressed pristines, shelving/checkpointing, ...) 1.9: Editor v2 (for server->client rename handling improvements) [...] 2.0: FS-NG (ideally a parallelized task), with some (to-be-identified) FS-NG enabled features. That last item is likely to be contentious, so it bears explaining. We believe that our current two filesystem offerings are stifling innovation. On the one hand we have the BDB backend which is a breeze to develop for but is only occasional used; on the other is the far-more-popular FSFS backend whose fundamental principles so constrain the system as to destroy much hope of serious design overhaul; and in the middle lies the feature parity requirement we've been living under thus far which binds Subversion to the union of the two backends' weaknesses. We confidently assert that to break system-wide compatibility with a so-called 2.0 release will be Subversion's great overall detriment, both from the perspective of efficient use of development energy and in user adoption. So we propose that an as-yet-fictional Subversion 2.0 be allowed to break compatibility with the 1.x line only in ways which can be mitigated using the RA layer as a compatibility shim. Additionally, we believe that merely releasing a 2.0 with a new FS backend but without new user-visible features enabled by that overhaul will be ill-received. COMMUNITY We also discussed matters of communication and community. Subversion has historically had a very tight-knit community of developers, and we've provided a resource for communicating with users through the various mailing lists. With the increase in corporate involvement, and the changing roles (and employers!) of committers, the Subversion ecosystem can at times seems a bit fractured. To an enterprise manager responsible for thousands of users, and millions of dollars of investment, and billions of bytes of data, this fracturing appears as a liability when considering an investment in Subversion. Most corporations understand the open source nature of Subversion's development method (indeed, this very thing has driven Subversion adoption), but they still want a unified face when it comes to communication, planning, and project feedback. One proposed solution is a Subversion "planet", to be hosted at a re-purposed subversion.org. The planet would aggregate feeds from individual contributors, as well as the various corporate entities interested in Subversion development. While various commercial interests (CollabNet, WANdisco, elego, etc.) may compete in some areas, they are all committed to improving Subversion as a whole. Enterprise users need to see this unity across Subversion's corporate sponsors, and a communication stream which interleaves these corporate voices works toward that end. Whatever the solution, we need a defined plan which we then communicate to the end users and customers who are deploying Subversion installations. But communication alone isn't enough. We need to find ways to grow our developer community itself. Attendance at the recent Subversion Corporation Annual Members Meeting was low (by design and recommendation, of course), but a startling realization was made there. When the agenda item for voting new full committers into membership was on the table, there were no candidates. Think about that: no new full committers for Subversion in the past year. This is a bad thing. We need to find a way to embrace and empower would-be Subversion contributors. Telling them to troll through the issue tracker looking for bite-sized issues is not the way to do that -- it communicates "we can't be bothered to mentor you". When somebody wants to start making contributions, our community must recognize the value of that contributor and mentor him or her toward full committership, for the good of that contributor and of the community. Further, our public roadmap needs to highlight the items that we really wish we could be working on but lack the manpower to handle. This would provide those looking for ways to accelerate Subversion's roadmap with some cues for doing that in harmony with the Big Picture. SUMMARY I've covered a lot of ground here, but decisions in this community have always been and will be made on this mailing list, and they deserve to be made with as much information as possible. You now know where a small contingent of developers stand on these issues. I'd like to publicize on our project website a *community-endorsed* vision and roadmap ASAP, and then get to the business of moving Subversion forward along those lines. So what say you? -- C-Mike, for the NYC conference attendees -- C. Michael Pilato <cmpilato@collab.net> CollabNet <> www.collab.net <> Distributed Development On Demand