Using Mercurial to Work Around Centralized SCM Limitations

I'm frustrated every time I encounter a project I want to/have to work on that uses centralized revision control. I'm not trying to repeat my Centralized Revision Control is Stupid post. Instead, I'm going to describe how mercurial solves these problems for me.

The short answer is mercurial queues. Continue reading to find out how it helps me in environments where I don't choose all of the tools.

Mercurial Queues for Managing External Products

mq is probably most commonly used for maintaining local changes to external projects. This is a really simple use to understand, so I'll cover it first. Note that this usage is also covered by the link above, so I won't go into a lot of detail.

Scenario: I use lighttpd as my web server, but one of the machines on which it runs is a bit older, doesn't use gcc (or any other compiler maintained in the last ten years), and I use some features that aren't distributed in the general source distribution. This leaves me with three categories of customization:

Platform-specific changes that make no sense outside of my environment General changes that were considered valid in gcc, but aren't really valid in compilers of this vintage Third-party features added on separately

I can keep all of my changes categorized, potentially submit parts back, and still track releases. My current tree has one change of type 1, four of type 2, and one of type 3. Whenever I want to upgrade, I pull down the new release, pop all the patches, update the tree, and go to push them all back in. If something goes wrong in one of the patches, it stops and allows me to repair it.

And the best part: My changes are revision controlled. In this usage, mq is what I like to call a meta-revision control system. I'm not checking in my changes to the upstream project, I'm checking in my changes to the changes to the upstream project.

Now, the general work doesn't feel that much different. It's still using whatever development tools are appropriate for the project and having very little interaction with my revision control system.

Mercurial Queues for Corporate Work

My company uses perforce for revision control, thus my group uses perforce for my project. This has several impacts on me that just don't exist when I'm using mercurial:

There's pretty much no offline support.

The speed at which I can work is limited by my access to the perforce server.

My IDE gets confused about what is under perforce support and forgets to open stuff for edit

Knowing what's been added or deleted (or accidentally edited without telling p4) is just painful.

Branching requires me to replicate the project on the server, which all but discourages experimentation.

There's probably more I could say there, but the theme is pretty clear: everything has to go through the centralized server.

I added perforce support to tailor specifically to help ease this pain. Changes are automatically replicated from perforce to mercurial. This effectively looks to me like everyone is using mercurial, so I track this tree.

The really interesting stuff is in my tree management, though. I use mq to do multi-dimensional work. That is, I develop vertical features which I commit horizontally.

For example, last week I was doing some protocol implementation where I need to accept information from a remote device and push changes through our model layer to persist data and fire off notifications. Except, there wasn't model support for what I needed to do...and there wasn't support for the kinds of testing I needed to do. Oh, and whatever else I found when I was working around in there.

I realized the model support was missing after I'd already got started on the upper level stuff. However, I didn't want to mix all of the changes into one giant blob of lots of distinct concepts. So I did the following:

hg qrefresh # Save the current changes hg qpop # move these changes out of the way hg qnew model-fixes # Create some new changes *under* those [do new stuff] hg qrefresh # Save new stuff hg qpush # Bring back work built on this model

At this point, I had two changes I was working on in my series. The lower-level change was model-only, and the upper-level change was using the model layer. At some point along the way, I realized the model was still not yet complete and I needed to do some further edits to it. It was very easy to manage:

hg qrefresh # Save upper-level changes hg qpop # Move upper-level stuff out of the way [fix model stuff] hg qrefresh # Save lower-level stuff hg qpush # Bring back upper-level stuff

Before I was ready to check in any work, I had (I believe) five changesets I was working on concurrently in a single stack. This was the above work, as well as some testing framework changes I needed to support both of the above, and some unrelated changes that came up along the way.

This model is still fresh, but along with a simple script to push my queue patch into perforce, it's working really well so far. I could be deep into a really broad change a large stack of changes and suspend it all for an urgent change without affecting my workflow.

Is This Post Over Yet?

Mercurial along with mercurial queues solves development processes that have been plaguing me for a long time now. Mercurial itself is a very nice revision control system that gives me access to all of the information and tools that I need. I use it by itself for all of my recent projects where I get to choose all the tools.

Mercurial queues builds on a good foundation to create a multi-dimensional meta-revision control system that's really fast and easy to use.

Better tools make it easier to make better software.