The Shoddy Statistics of Super Freakonomics

Super Freakonomics is getting a lot of flak for its flip contrarianism on climate change, most of which seems based on incorrectly believing solar panels are black (they're blue, and this has surprisingly large energy implications) and misquoting important climate scientists.

But before people begin believing that the problem with Super Freakonomics is that it annoys environmentalists, let's be clear: The problem with Super Freakonomics is it prefers an interesting story to an accurate one. This is evident from the very first story on the very first page of the book.

Under the heading "putting the freak in economics," the book lays out its premise: Decisions that appear easy are actually hard. Take, for example, a night of drinking at a friend's house. At the end of the night, you decide against driving home. This decision, the book says, seems "really, really easy." As you might have guessed, we're about to learn that it's not so easy. At least if you mangle your statistics.

The next few pages purport to prove that drunk walking is eight times more dangerous than drunk driving. Here's how they do it: Surveys show that one out of every 140 miles driven is driven drunk. "There are some 237 million Americans sixteen and older; all told, that's 43 billion miles walked each year by people of driving age. If we assume that 1 out of every 140 of those miles are walked drunk -- the same proportion of miles that are driven drunk -- then 307 million miles are walked drunk each year."

"If we assume."

But why should we assume that? As the initial example demonstrates, a lot of people walk drunk when they would otherwise drive drunk. That substitution alone suggests that a higher proportion of walking miles are drunk miles. Other people walk, or take transit, when they know they'll be drinking later. That's why they're walking and not driving. That skews the numbers and makes it impossible to simply "assume" parity.

Then there's the implicit assumption that the two types of drunk miles are the same. But there are a number of reasons to question this presumption. For one, the miles walked drunk are probably disproportionately urban, while the miles driven drunk are probably disproportionately rural and suburban. But driving an urban mile drunk is probably a lot more dangerous than driving a rural mile drunk, just as walking an urban mile drunk is probably much more dangerous than walking a rural mile drunk.

For another, the levels of drunkenness probably differ. A lot of those miles driven drunk are probably miles driven by someone who's had a couple of glasses of wine and still feels competent to drive, even though their blood alcohol level is a shade above 0.8. That is to say, a lot of those miles are driven by someone who's not very drunk. A lot of those miles walked drunk are probably walked by someone who is so wrecked that his friends wouldn't let him drive, or he never would imagine trying. It is, of course, safer to be less drunk than more drunk when you're around vehicles, whether you're walking near them or driving by them.

You can go on and on in this vein. It's terrifically shoddy statistical work. You'd get dinged for this in a college class. But it's in a book written by a celebrated economist and a leading journalist. Moreover, the topic isn't whether people prefer chocolate or vanilla, but whether people should drive drunk. It is shoddy statistical work, in other words, that allows people to conclude that respected authorities believe it is safer for them to drive home drunk than walk home drunk. It's shoddy statistical work that could literally kill somebody. That makes it more than bad statistics. It makes it irresponsible.

But hey, it makes for a fun and unexpected opener.

