Many, many thanks to all of you who have contributed to the astonishing response to the previous article on the difficulty of getting binary search right. I’ve never seen anything like this: in just over 24 hours since I posted the article, 541 comments have been posted, and they’re still rolling in. (That number is a bit inflated by people re-posting the same code-samples multiple times after seeing WordPress mangle their source, but even taking that into account it still amazes me.)

You guys rule. Seriously.

I have a lot to say in response to it all. Read on …

The lameness of the WordPress commenting system

First up, an apology to all of you who posted code that got mangled. As several commenters pointed out, WordPress comments are not really a great medium for this kind of challenge: their tendency to eat less-than characters, their lack of a Preview facility and the inability to edit a comment once it’s been posted combine to make the post-a-code-fragment process rather more of an adventure than it ought to be. (That said, I remain astonished at how many people used curly brackets around their {source} tags even though my instructions clearly said in bold to use square brackets. Come on, folks — read the article before submitting your code!)

Anyway, the next time I do something like this, I will be sure to include very, very explicit instructions right at the start, and to include links to various code-fragment hosting services such as pastebin.com, pastebay.com and gist.github.com.

In the mean time, see the WordPress Support article on Posting Source Code.

Different kinds of comments

It’s been fascinating to see how different the comments have been in the three places that the binary search article was discussed. Here on The Reinvigorated Programmer, the comment thread was massively dominated by attempts to meet the challenge. The comments on Hacker News were primarily discussions of the problem, of how it applies in real development situations, of what the gotchas are, and of how best to guard against them. The Reddit comments were rather more mystifying — many redditors seemed to feel that the idea of actually coding anything was rather beneath them, and were rather affronted that anyone might suggest such a thing; others found the idea of coding a binary search “elitist”.

Of course there was plenty of variation in all three venues (for example, this guy on Reddit made a serious attempt to solve the problem, then fessed up when he realised his routine was buggy). But that’s the overall impression I got. Sorry, redditors — I still love Reddit, honest! By the way, Reddit had a very ambivalent response to this article, as it often seems to with what I post here: at the time of writing, Reddit scores the binary-search challenge with 44 points, but that is the result of 121 upvotes and 77 downvotes, so only 60% of the votes were positive. At the risk of alienating redditors, I can’t help wondering whether some of the negativity comes from insecurity — programmers who know they can’t write low-level code and prefer not to be reminded of it. I probably shouldn’t have said that. Hmm. Must remember to go back afterwards and delete it.

Two of the more interesting areas of commenting (both of which I disagree with) were that you don’t need to write this routine because it’s already in the libraries; and that coding without testing as you go is nonsensical. I’ll address the first of those in more detail below; and I’ll talk about testing next time, to avoid making this article too long.

Statistics

At this stage I’d hoped to unveil a simple analysis of the form “14 people attempted the challenge, of whom 6 reported bugs in their own code and 8 claimed success, but I found bugs in 5 of those for a total success rate of 3/14 = 21% which means Reinvigorated Programmer readers are 2.1 times as clever as IBM and Bell Labs professionals”.

That neat idea went out the window for three reasons:

The good: there were 557 submissions instead of 14, in maybe fifteen different languages, and there was no way I could test them all.

The bad: because I produced the Jon Bentley quote from Programming Pearls before stating the rules of the challenge, a lot of people eagerly ploughed straight in, and so inadvertently broke the rules. My bad.

The ugly: WordPress mangled the source code of many of the submissions.

So I am not even going to attempt to come up with numbers based on the actual data.

That said, I have read all 562 comments, and gathered at least a general sense of the success rate, and I’d say that based on people’s self-reporting, about half of the binary searches were correct. (Kudos, by the way, to all those of you who were honest enough to report failure; and, for that matter, to those of you who achieved success.)

If that 50% figure is anywhere near accurate, then this group of blog-readers are well ahead of the averages. But bear in mind that probably some people failed but didn’t report that fact; and for sure plenty of people who reported success had simply not found bugs in their programs. For example, Eric Burnett claimed in a comment that he’d found bugs in nine solutions reported as correct. I’m sure that plenty more “correct” solutions still harbour bugs, and I encourage you all to go and hunt down each others’ mistakes! In particular, finsprings, Aaron, Mike J, Langtree, Luke, Marcus, cycojesus, Erik Swanson and donaq (the nine people whose code Eric challenged) might want to see if they can find a mistake in Eric’s own solution!

Those of you who want to find bugs in other people’s solutions — or indeed assure yourselves that your own solutions are as good as you think they are — will find it useful to take advantage of Steve Witham’s set of 4096 test cases. Many thanks to Steve to providing these.

Anyway, even allowing the various biasing factors, it seems possible — maybe even likely — that our hit rate is better than 10%.

Common bugs

It’s been interesting to see that many of the programs that had bugs were going wrong in the same ways. Common bugs included:

Not dealing correctly with a zero-element array

Not dealing correctly with a one-element array

Not finding a value in the last element of the array

Not dealing with failed searches (the sought element is not there)

Not coping with repeated elements in the array

Not omitting the midpoint from the subranges when narrowing the range after a probe. (This results in an infinite loop for some inputs.)

Pointing to the first and last members of the range but coding as though the last-pointer variable pointed to the element after the range (or vice versa).

(And, yes, not dealing with a zero-element array is a bug, for those commenters who asked. Zero is a perfectly good number of elements for an array to have, and an array search that fails for empty arrays is no more acceptable than an addition operator that fails if you add zero. A few commenters also asked whether it’s OK to ignore the case when a null Array reference is passed in: yes, of course — that would violate the precondition that the search is in a pre-sorted array, so all bets are off. No array at all is completely different from an empty array.)

The interesting thing about these common failure modes is that they are all so predictable. If, before starting to code, we’d each taken two minutes to write down the ways in which we could imagine our programs failing, I bet we’d all have listed most if not all of these. Yet even knowing that, we find them hard to avoid.

In light of the list of common bugs, we can easily see what kinds of test cases we need to include in the suite: empty arrays, single-element arrays, arrays where the sought element is absent, or appears only at the end, or only at the start, or multiple times; and of course a healthy does of randomly generated sorted arrays.

Looking at the list of bugs, I find myself thinking of the rules in Kernighan and Pike’s Elements of Programming Style. Back when I reviewed that book, and listed the rules that are gathered at the end of the article, some commenters felt that they were so obvious, so trivial, as to be worthless or even insulting. And yet most of the mistakes listed above are covered in K&P’s list. A few of the relevant rules are:

“Write clearly — don’t be too clever.”

“Make sure your code “does nothing” gracefully.”

“Watch out for off-by-one errors.”

“Take care to branch the right way on equality.”

“Make sure special cases are truly special.”

“Choose a data representation that makes your program simple.”

Then there is the rule that the I didn’t let you follow literally, but the spirit of which should pervade our design process: “Test programs at their boundary values”. And the very general but very relevant rule “Don’t stop with your first draft.” And of course I must mention my favourite:

“Say what you mean, simply and directly.”

Why does this exercise matter?

A lot of the comments on Reddit (and some here) were along the lines that it’s dumb to write a binary search routine, and that we should just use one from a library. For example, Michael Eriksson wrote that “Trying to write an own sorting/searching/whatnot algorithm is a beginner’s error (in and by it self). There are well-tested and highly performant libraries for such basic functions available in any established high-level language”.

That seems wrongheaded to me. A boxer never has to skip rope in the ring, but it’s an important part of his training regime. A concert pianist never has to play D-flat major, four octaves, hands together, ascending and descending, as part of a performance; but he doesn’t for that reason neglect the exercise. An olympic sprinter never lifts weights during the 100 meters final, but he certainly does as part of his preparation. Why would we think that in programming we don’t need to do exercises that are similarly related to our day-to-day work?

I have a hypothesis about that, but it’s not one that’s going to be popular. The boxer, the concert pianist and the sprinter need to be at the absolute top of their game in order to succeed. If the boxer’s not light on his feet, he’ll get beaten up; if the pianist lacks dexterity, he simply won’t get booked, in such a competitive career; the sprinter deals in margins of hundredths of a second. They practice, exercise, do training drills because they must: if they fall to, say, 97% of their best performance, they lose. Could it be that programming is a little too comfortable? Do employers expect too little? Are we content just to stay some way ahead of the pack rather than striving to excel? That’ll work if you’re happy to write Enterprise Beans For The Enterprise for the rest of your career. Not so much if you’re hoping to go and work for Google.

So the primary reason for writing a binary search routine is not because we need to write binary search routines. It’s because we need the mental exercise, the discipline, the pleasingly firm, cool feel of solid code beneath our fingers. The ability to do a binary search right first time probably correlates pretty well with the ability to write more complex code correctly. It builds up the right brain-muscles.

That’s the primary reason. The secondary reason is because actually sometimes you do need to write a binary search, and the library routines won’t get the job done. Or if they will, they’re grotesquely inefficient. For example, suppose you have a 64-bit integer, and you need to find out whether it’s among the nine billion 64-bit integers that are stored in ascending order in a 72 Gb file. The naive solution is to read the file into memory, making an array (or, heaven help us, an Array) of nine billion elements, then invoke the library search function. And of course that just plain won’t work — the array won’t fit in memory.

Or consider a less extreme, and therefore more subtle, case: the file to be searched has only half a billion numbers in it (so it’s a 4 Gb file). That will fit in memory in some modern computers, depending on how space-inefficient the in-memory representation is. But the result of using the naive approach is that you read 4 Gb from disk every time you need to do the search. Whereas if you could implement binary search directly against the disk file you’d only need to do log(2) (half a billion) = 29 probes, of which the last 10 will all be in the same 8 Kb disk block. So you’ll only do 20 seek-and-read pairs instead of a seek and half a million reads.

No, this stuff doesn’t happen every day. But situations do crop up when someone needs to know some actual, you know, computer science. And when that happens, don’t we want to be the ones who can come swooping in and save the day?

Next time, I’ll be addressing all the howls of outrage about Rule 6 (“NO TESTING until after you’ve decided your program is correct”), and trying to escape the pestilence of “test-driven development”. Stay tuned!

Update (the next day)

If you usually just read the articles and skip the comments (and who could blame you in the case of the original binary-search challenge post!), then I urge you at least to read this comment by Antti — it’s solid gold (even if it does rather trespass on the stuff I was saving up for my own next post, bah!)

Update 2: links to this whole series