I’ve been posting much longer articles than I meant to, so today’s will be short. [Later: but, as it turns out, not as short as I intended.]

When I was fourteen, I wrote space-invader games in BASIC on a VIC-20. If you were interested in computers back in 1982, I bet you did the same. When I was 18, I wrote multi-user dungeons in C on serial terminals attached to a Sun 3. When I was 22, I worked deep down in the guts of a text database system — still C, now on a Sun 3/80 of my very own, with one of those HUGE bitmapped screens with a million black-or-white pixels. I was in touch with my friends from university: we were going to write compilers and operating systems and cool stuff like that — and to some degree, we did. We sent each other our in-progress code, complained about each other’s programming-language designs, and laughed at how inefficient each others’ completely unnecessary reimplementations of malloc() were. [I remember a friend’s implementation achieving something like 18% occupancy.]

That was then.

Today, I mostly paste libraries together. So do you, most likely, if you work in software. Doesn’t that seem anticlimactic? We did all those courses on LR grammars and concurrent software and referentially transparent functional languages. We messed about with Prolog, Lisp and APL. We studied invariants and formal preconditions and operating system theory. Now how much of that do we use? A huge part of my job these days seems to be impedence-matching between big opaque chunks of library software that sort of do most of what my program is meant to achieve, but don’t quite work right together so I have to, I don’t know, translate USMARC records into Dublin Core or something. Is that programming? Really? Yes, it takes taste and discernment and experience to do well; but it doesn’t require brilliance and it doesn’t excite. It’s not what we dreamed of as fourteen-year-olds and trained for as eighteen-year-olds. It doesn’t get the juices flowing. It’s not making.

Knuth speaks

If you don’t believe my analysis, will you believe Don Knuth? Here are a couple of extracts from his excellent interview in Peter Siebel’s book Coders at Work (which, yes, I have now finished reading):

“There’s the change that I’m really worried about: that the way a lot of programming goes today isn’t any fun because it’s just plugging in magic incantations — combine somebody else’s software and start it up. It doesn’t have much creativity. I’m worried that it’s becoming too boring because you don’t have a chance to do anything much new. Your kick comes out of seeing fun results coming out of the machine, but not the kind of kick that I always got by creating something new. The kick now is after you’ve done your boring work then all of a sudden you get a great image. But the work didn’t used to be boring.” (page 594)

“The problem is that coding isn’t fun if all you can do is call things out of a library, if you can’t write the library yourself. If the job of coding is just to be finding the right combination of parameters, that does fairly obvious things, then who’d want to go into that as a career?” (page 581)

Are you gonna argue with Knuth? Huh? Are you? Didn’t think so.

We gotta get out of this place

I want to make things, not just glue things together. When people ask me what I like about my job, I always say the same thing: that its the thrill of starting with nothing and making something. That, for me, is the essence of programming, and it hurts that there isn’t as much of it about as there used to be.

We all know that the most enjoyable part of a programming project is early on: when the slate is clean, the emacs buffer is empty, and the world is fresh and alive with possibilities. And then the invigorating rush as the shape of the code starts to emerge, the data structures condense, the algorithms come together; the code becomes runnable, then it does something useful, it passes tests, and then — yes! — it’s not just an idea any more, but an actual program. You’ve completed Phase 1.

And then — anyone who’s ever actually released software will recognise this — then in a sense the actual work begins. For the program to stop being a private project and become a public product, it needs documentation — APIs, command-line manuals, tutorials. It needs unit tests. It needs a home on the web. It needs checking for portability. It needs changelogs and a release history. It needs tweaking, and quite possibly internal reorganisation to make it play nicer with other programs out there. All this is phase 2, and it’s closely related to the issues of programming culture that I talked about last time.

The thing is, no professional programmer begrudges Phase 2. We all recognise the necessity of these things, and we take pride in doing them right. It’s part of professionalism; part of being not just a computer scientist, but also a software engineer. It’s all good.

But Phase 2 is not the core of the work. It’s really all about Phase 1; and even if Phase 2 takes more effort and time, it’s still only the ‘i’s that we dot and the ‘t’s that we cross to give our beautiful code a life outside of our own computers. And the problem with modern software development is that it’s all Phase 2. The ubiquitious availability of nearly-right-but-not-quite libraries and frameworks-that-do-it-all-for-you-except-that-actually-they-don’t wipes out the exhilaration of Phase 1, but leaves us with even more of the drudgery of Phase 2. Instead of designing beautiful data-structures and elegant algorithms, we’re looking up the EnterpriseFactoryBeanMaker class in the 3,456-page Bumper Tome Of Horrible Stupid Classes (Special Grimoire Edition), because we can’t remember which of the arguments to the createEnterpriseBeanBuilderFactory() method tells it to make the public static pure virtual destructor be a volatile final abstract interface factory decorator.

I understand, I think, how we landed up here. I wish I know how we can get out.

Update (9 March 2010)

I should have added this update days ago, but I only just realised that a lot of people are reading this article from links elsewhere and may not have seen what’s on the rest of the site. So:

I wrote a follow-up article in response to the initial batch of thirty or forty comments, which addresses a lot of points that were made there, and much of what has been said subsequently, too. If you liked, or hated, this article, you’ll probably find that one interesting, too.

Update #2 (15 March 2010)

Belatedly, here are some links to other discussions of this article elsewhere on the net:

Update #3 (1 November 2011)

To my great surprise, this article has popped up on Reddit again, twenty months after its first appearance. So if you’re interested in what’s being said now, head over to: