Conclusion

Git has been selected as our new VCS!

http://groups.drupal.org/node/48818?page=2#comment-133893

http://sf2010.drupal.org/conference/sessions/exodus-leading-drupal-out-cvs

http://groups.drupal.org/drupal-org-git-migration-team

Debate archived below:

We actually had quite a productive IRC discussion tonight (no, that is shockingly not an oxymoron!) about the general migration to distributed development, the community fragmentation that it causes, and how the tools on drupal.org might be improved as a way to combat this.

After spirited discussion, we brainstormed this "hit-list" of things that need to happen in order to Drupal.org to ever move to a distributed version control system. These are pretty loose, but there are quite a few actionable tasks that came out of it, which folks interested in scratching this particular itch could start picking off.

If you want to help, please clearly mark one or more of these things with your name (or the name of you and your buddies, make a game of it!), and comment below with progress. Or, help flesh this out a bit more by providing links, things we haven't thought of yet, etc.

Summary of Contenders

We've narrowed the search to Bazaar and Git. For the purposes of our community, the two appear to be basically functionally equivalent (yes?). Here's how they stack up on the other merits:

Bazaar

Provides the benefits of a distributed version control system, while also supporting a traditional centralized workflow, which will make transition easier for people. This advantage should not be under-stated.

Existing drupal.org infrastructure team already has knowledge of bzr, and has agreed internally to move drupal.org to it.

Bazaar is headed up by Canonical, which is likely not going anywhere anytime soon. Good for future-proofing.

Git

Much higher percentage of Drupal community members have familiarity with it, which will help ease transition headaches, which may be considerable since there are not familiar "anchors" with Git as there are with Bzr. This advantage should not be under-stated.

Existing work has already started to integrate Project* modules with Git: http://drupal.org/project/versioncontrol_git

Metrics show a general trend towards Git being the leader in the distributed VCS space with a huge, thriving community. Good for future-proofing.

Bzr vs. Git Smackdown at Drupalcon SF

The best way to figure out which of these two options would be the best fit for our community is to actually see with our own eyes. Drupalcon SF offers a tremendous opportunity for this, with the added advantage of having all the major players in place to lead sessions and make a final decision. Therefore, what we'd like to organize are Git/Bzr "info sessions" where people knowledgeable in the technology sit down with volunteers and observe and take notes, so we can try and get a sense of

So please help by filling out this stuff:

Drupalcon SF attendees who are willing and able to lead these kinds of "info/tutorial" sessions: Bzr: Senpai, matt2000, NAME Git: sdboyer, scor, ceardach, sirkitree, NAME

Drupalcon SF attendees who are willing and able to be Guinea pigs, mainly those with knowledge only of CVS (or CVS and Subversion): <-- yes? Core developers: webchick, beeradb, NAME, NAME Module developers: webchick, NAME, NAME, NAME Theme developers: NAME, NAME, NAME Site builders who manage deployments through version control: webchick, beeradb, NAME, NAME Wtf is a patch? What's version control?: Elijah Lynn, NAME, NAME

Drupalcon SF attendees who are willing and able to act as "clipboard" folks, gathering data during these sessions: webchick, NAME, NAME, NAME



Then we need to brainstorm stuff like:

What criteria are the clipboard folks looking for?

What documentation needs to be prepped ahead of time? "Quickstart" guide for each VCS, akin to http://drupal.org/handbook/cvs/quickstart "Examples" that each "group" will go through.

Who's going to prep it?

What infrastructure needs to be prepared to perform this testing?

Who's going to prep it?

What sort of data are we hoping to gather from this session?

What sort of infrastructure do we need to capture it?

Who's going to prep it?

Other...?

Why are we doing this?

Things about CVS that currently suck:

It's an archaic system that no one in their right mind chooses as a version control system today, and thus it's a big barrier to entry because new contributors must learn it "specially" in order to contribute to drupal.org

Things that should be really simple (submitting changes that add/remove files, renaming files, etc.) are absolutely horrendous in CVS.

It's really difficult to "chase HEAD" since CVS merge tools totally suck, making contributing major changes to Drupal core incredibly painful when there are constant re-rolls due to whitespace changes in other places.

It's difficult for patch authors to have discipline to keep changes to one "context", resulting often-times in "mega-patches" that are impossible to review.

We lose the incremental commit history that happened on said "mega-patches."

It's difficult to share experimental code and encourage others to improve it. <-- this is a d.o issue, not a CVS issue

The credit on commits (as in cvs annotate credit) currently goes to the committers of the code, not the people who actually wrote (and reviewed) them.

It's impossible with CVS to do much of anything while offline, because even a 'diff' operation needs to "phone home" to the parent server. Lots of us are on planes a lot of the time.

In general, the lack of modernization of our contribution tools are causing people to move to other places like Launchpad and GitHub, thus fragmenting both our community, and the strength of Drupal.org as the central hub to find any and all Drupal stuff. <-- both a d.o issue and a CVS issue.

Security in CVS sucks. pserver's encryption is sub-standard, and only a small handful of people (mainly core maintainers and infra team) use public-key encryption.

What don't we want to lose?

Drupal.org being the central collaboration hub for Drupal core and contributed projects: code, reviews, and discussions take place here, where the entire community can participate and learn from them.

The "incremental changes posted to an issue queue where they then get peer review" workflow allows people who are non-developers to participate, and for reviewers/maintainers to see incremental progress instead of the whole thing at once.

Keeping only one canonical "target" for each project in all of Drupal, for the entire community to buzz around. This makes peer reviews much easier, since the way to test changes is always the same, and keeps everyone collaborating on the same stuff, not a fork of a fork of a fork of a....

Things that we've ruled out

Moving to $better_centralised_VCS, such as Subversion. The amount of work required to pull this off is substantial; we need to make sure that we're set for another 10 years, and distributed VCS is clearly the way of the future.

Any non-free (as in beer and freedom) VCS.

Pretty much anything but Git, Bzr, and Hg at this point, unless someone can make an extremely compelling case for something else.

at this point, unless someone can make an extremely compelling case for something else. Mercurial. There just aren't enough existing community members who know it, nor do the folks on the infrastructure team have a background there. Sorry, Hg!

Concrete to-dos for moving Drupal.org to $distributed_VCS

Documentation/Training

HEY! You there! Put your name here. :D



Git: yhager, mike booth, sdboyer, mig5, marvil07, lut4rp, scor, ceardach, reglogge, kyle_mathews, bdragon, asimov, hugowetterberg, tobiassjosten, fago, DamienMcKenna, Frando, Will White, Jeff Miccolis, psynaptic, anarcat (willing to help with CVS -> git migration), slantview, EclipseGc, VoxPelli, gordon, schoobidoo, Pisco, Sean Bannister, mikl, jrglasgow, stepmoz, Jeremy, seutje, corni, ben-agaric, sirkitree, chachasikes NAME

Bzr: David Strauss, Peter Wolanin, Narayan Newton, Josh Koenig, bdragon (again), chx, NeverGone, a_c_m, Nuno Veloso, cha0s, Heine, Garrett Albright, Emma Jane Hogbin, TBarregren, dixon_, frjo, ximo, matt2000 NAME

Hg: Heine, mikey_p, mcrittenden, NAME

Compile a list of our current "use cases" around version control: (DONE?) Collaborative patch authoring : Huge changes like form API, field API, core themes, etc. that are more-or-less equally authored by multiple people. Primary patch authors : There's primarily one person who's "in charge" of a patch, but is taking contributions from other authors. Patch contributors : Contribute more minor things like code-standard compliance, spelling corrections, etc. to a primary path author. Patch reviewers : Eyeball the code, take a change from the issue queue and ensure that it's properly working, post back their results. Contrib module developers : Handle primary development of module, pull in patches from others, maintain branches and releases. Contrib theme developers : Handle primary development of theme, pull in patches from others, maintain branches and releases. Co-maintainers : Help maintain a module/theme, but often checking first with the primary author before changes are made. Users : Run their Drupal site on code directly checked out from d.o repositories, they update to the latest release tag and rebase patches on top of that. ____ (other use cases?)



Comparisons

Compare Git/Bzr/Hg on the following items (see also DVCS-specific pages for Git: http://groups.drupal.org/node/48843, Bazaar: http://groups.drupal.org/node/48848, and Mercurial: http://groups.drupal.org/node/48853 for more detailed details). Comparison charts git-hg-bzr: InfoQ dvcs-guide, May 2008

(Note: Some of these might be "Well all of them do that" and that's cool, then just mention it there; I was trying to pull from further down the wiki.)

Comparison of issue trackers on other Open Source projects

Note: While these statistics are interesting, note that a remotely-hosted service such as GitHub is off the table, because it violates the #1 thing we don't want to lose: keeping drupal.org central collaboration hub for Drupal.

Drupal.org integration

(This part still needs major exploration work done, to determine what exactly we want, and what work is actually involved. It's also the thing that's going to hold everything else up, so the sooner someone jumps on this the better.)

"Phase 1" probably just looks like keeping our existing workflows in place (i.e. sharing changes via patches in the issue queue, hosting only the "canonical" repository for each project), and replacing CVS with $vcs. This requires research into the following:

Evaluate feasibility of moving off of Project* modules in favour of $vcs issue tracker/source code integration tool. Gitorious (Git) Pros: __________ Cons: ____________ Bzr: Launchpad (Bzr) Pros: __________ Cons: ____________ I posted a comparison that wouldn't fit here in a comment below. --David Strauss _________ Other? Neutral? Pros: __________ Cons: ____________ Existing infrastructure that would need to be modified to work with $vcs tool Testing bot _____ (lots and lots of things, I'm sure...) Bad stuff We'd end up forcing our community through two significant learning curves at once (new VCS, new project management tools), which should definitely not be under-stated. In addition to the already significant documentation requirements for simply the VCS move, we'd probably sextuple those having to re-write all of our existing documentation that makes reference to how to use the issue queue, project management tools, etc. (otoh, we could probably externally link to a lot of it) _____ (lots and lots of things, I'm sure...) Good stuff Not having to port project* module every time we want to upgrade durpal.org. :P _____ (lots and lots of things, I'm sure...)

Identify and document places where in our current scripts, etc. there are hard-coded assumptions about CVS. For each item, identify the piece of code currently responsible for performing this job, determine if there are CVS assumptions, and if so create a set of issues for discussing/tracking these, and link them here. Release packaging scripts Pre-commit scripts Testbot Commit log browser - provided by CVS integration module. Access control to repositories - git example Source code browser - provided by ViewVC, which only works on CVS and Subversion. An alternative tool must be found. Git: List on kernel.org wiki, ViewGit, Trac + GitPlugin, Git Browser (module) Bzr: Loggerhead (see an example here), several other options: http://wiki.bazaar.canonical.com/WebInterfaces Neutral: Trac, Redmine, but these are entire project management systems, not just source viewers. Other?

Determine logistics for how $vcs replacing CVS actually looks: What does the new directory structure look like in core/contrib? How do we manage "official" releases (tags) vs. development releases (branches)? How do we facilitate (or don't we, in phase 1) "spooning" of code?



Stuff we need to do /after/ we have chosen a $vcs and have the rest of the stuff above in progress

Documentation/Training

An equivalent of http://drupal.org/handbook/cvs/quickstart for $vcs

FAQs/Troubleshooting/OMG I BROKE IT HELP!!! docs

Screencasts on how to use it.

Scheduling of "info sessions" on IRC (or Skype, or whatever) for $vcs brigade to train existing contributors on the new system.

Future feature requests

When patches are uploaded to d.o, do automatic generation of tarballs with project + patch applied already, to facilitate reviews by non-technical users. (Feature request posted here: http://drupal.org/node/707526)

Crazily pimping out drupal.org to take advantage of more advanced $vcs features, e.g. feed from commit logs on forked branches to the issue queue, create and post patch automatically to issue with a button, etc.

Exploring alternative core contribution workflows, e.g. "authorized" branches for specific features like fieldable user profiles in D8 core, with someone "deputized" to accept changes.

Browser-based editing to files in repo leading to automatic patch creation, for designers + commits via the issue queue/web UI <-- /really/ not sure of this idea.... This is really appealing in my mind, since it blows away the barrier to entry for the vcs, so devs can commit easily. ao5357



(Props to at least Benjamin-Melancon, Lizzard, walkah, yhager, JohnAlbin, dbabbage, catch, jensimmons, hefox, and whoever else helped work on this. :))

Edit: I've removed all +1 comments. Please, no more