I’ve got a lot to say today!

I want to chat about something that I’ve never noticed before, but probably should have. There’s always been a stereotype out there of programmers being nerdy, anti-social people (Q: How do you know when an engineer is outgoing? A: He looks at your shoes!). But my revelation of the week is that most programmers seem to be really insecure about their work. I mean: really, really insecure.

My buddy Fitz and I have long preached about best practices in open source software development — how one should be open and transparent with one’s work, accept code reviews, give constructive criticism, and generally communicate as actively as possible with peers. One of the main community “anti-patterns” we’ve talked about is people writing “code bombs”. That is, what do you do when somebody shows up to an open source project with a gigantic new feature that took months to write? Who has the time to review thousands of lines of code? What if there was a bad design decision made early in the process — does it even make sense to point it out? Dropping code-bombs on communities is rarely good for the project: the team is either forced to reject it outright, or accept it and deal with a giant opaque blob that is hard to understand, change, or maintain. It moves the project decidedly in one direction without much discussion or consensus.

And yet over and over, I’m gathering stories that point to the fact that programmers do not want to write code out in the open. Programmers don’t want their peers to see mistakes or failures. They want to work privately, in a cave, then spring “perfect” code on their community, as if no mistakes had ever been made. I don’t think it’s hubris so much as fear of embarrassment. Rather than think of programming as an inherently social activity, most coders seem to treat it as an arena for personal heroics, and will do anything to protect that myth. They’re fine with sharing code, as long as they present themselves as infallible, it seems. Maybe it’s just human nature.

Check out some of these stories I’ve collected:

Requests at the Google I/O booth : A couple of weeks ago when my team was at the Google I/O conference, we ran a booth demonstrating our Open Source Project Hosting service. Over and over, we kept getting requests like this:

“Can you guys please give subversion on Google Code the ability to hide specific branches?” “Can you guys make it possible to create open source projects that start out hidden to the world, then get ‘revealed’ when they’re ready?” Translation: “I don’t want people to see my work-in-progress until it’s perfect.”

: A couple of weeks ago when my team was at the Google I/O conference, we ran a booth demonstrating our Open Source Project Hosting service. Over and over, we kept getting requests like this: Requests on the Google Code mailing list : Sometimes users need their googlecode.com svn repositories wiped clean. Legitimate reasons include the accidental commit of sensitive data, or the need to load code history in from a different svn repository. But most of the time we get (invalid) requests like this:

“Hi, I want to rewrite all my code from scratch, can you please wipe all the history?” Translation: “I don’t want people to be able to find my old code, it’s too embarrassing.” Call it vanity, call it insecurity… the bottom line is that coders want prior mistakes or failures to be erased from history.

: Sometimes users need their googlecode.com svn repositories wiped clean. Legitimate reasons include the accidental commit of sensitive data, or the need to load code history in from a different svn repository. But most of the time we get (invalid) requests like this: Code-reviews taken as personal attacks . Fitz tells a funny anecdote about a friend of his who went from the open source world to a corporate software job. Vastly paraphrased:

During his first week, he started emailing friendly code reviews to each of his coworkers, receiving strange stares in turn. Eventually his boss called him into his office: “You know, you really need to stop with the negative energy. Your peers say that you’re constantly criticizing everything they do.” Moral: not only is code review not the norm in corporate environments, most programmers are unable to separate their fragile egos from the code they write. Repeat after me: you are not your code!

. Fitz tells a funny anecdote about a friend of his who went from the open source world to a corporate software job. Vastly paraphrased: Distributed version control — in a cave. A friend of mine works on several projects that use git or mercurial. He gave me this story recently. Basically, he was working with two groups on a project. One group published changes frequently…

“…and as a result, I was able to review consistently throughout the semester, offering design tweaks and code reviews regularly. And as a result of that, [their work] is now in the mainline, and mostly functional. The other group […] I haven’t heard a peep out of for 5 months. Despite many emails and IRC conversations inviting them to discuss their design and publish changes regularly, there is not a single line of code anywhere that I can see it. […] Last weekend, one of them walked up to me with a bug […] and I finally got to see the code to help them debug. I failed, because there are about 5000 lines of crappy code, and just reading through a single file I pointed out two or three major design flaws and a dozen wonky implementation issues. I had admonished them many times during these 5 months to publish their changes, so that we (the others) could take a look and offer feedback… but each time met with stony silence. I don’t know if they were afraid to publish it, or just don’t care. But either way, given the code I’ve seen, the net result is 5 wasted months.” Before you scream; yes yes, I know that the potential for cave-hiding and writing code bombs is also possible with a centralized version control system like Subversion. But my friend has an interesting point: “I think this failure is at least partially due to the fact that [DVCS] makes it so damn easy to wall yourself into a cave. Had we been using svn, I think the barrier to caving would have been too high, and I’d have seen the code.” In other words, yes, this was fundamentally a social problem. A team was embarrassed to share code. But because they were using distributed version control, it gave them a sense of false security. “See, we’re committing changes to our repository every day… making progress!” If they had been using Subversion, it’s much less likely they would have sat on a 5000 line patch in their working copy for 5 months; they would have had to share the work much earlier. Moral: even though one shouldn’t depend on technical solutions to social problems, default tool behaviors matter a lot. This was my main theme way back when I wrote about the risks of distributed version control.

OK, so what’s the conclusion here? People are scared of sharing their unfinished work, plain and simple. I know this isn’t headline news to most people, but I really think I’ve been in deep denial about this. I’m so used to throwing my creative output up for constant criticism, that I simply expect everyone else to do it as well. I think of it as the norm, and I can’t comprehend why someone wouldn’t want to do that… and yet clearly, the growing popularity of distributed version control shows just how thrilled people are to hide their work from each other. It’s the classic “testimonial” for systems like git (taken from a blog comment):

“Don’t tell me I should cooperate with other people at the beginning and publish my modification as early as possible. I do cooperate with other people but I do want to do some work alone sometimes.”

Hm, okay. Please just don’t work alone for too long!

Sidetracking a wee bit, I think this is why I prefer Mercurial over Git, having done a bit of research and reading on both systems. Git leans much more heavily towards cave-hiding, and I don’t like that. For example, the ‘git rebase’ command is a way of effectively destroying an entire line of history: very powerful, sure, but it’s also a way of erasing your tracks. Rather than being forced to merge your branch into a parent line, just pretend that your branch was always based on the latest parent line! Another example: when it comes to pushing and pulling changesets, Mercurial’s default behavior is to exchange all history with the remote repository, while git’s default behavior is to only push or pull a single branch — presumably one that the user has deemed fit for sharing with the public. In other words, git defaults to all work being private cave-work, and is happy to destroy history. Mercurial shares everything by default, and cannot erase history.

I know this post has been long, but let me stand on my soapbox for a moment.

Be transparent. Share your work constantly. Solicit feedback. Appreciate critiques. Let other people point out your mistakes. You are not your code. Do not be afraid of day-to-day failures — learn from them. (As they say at Google, “don’t run from failure — fail often, fail quickly, and learn.”) Cherish your history, both the successes and mistakes. All of these behaviors are the way to get better at programming. If you don’t follow them, you’re cheating your own personal development.

Phew, I feel better now.

(This page has been translated into Spanish language by Maria Ramos.)