Another election day in the US is rapidly approaching (Tuesday, Nov. 8—mark your calendars!). So for no reason in particular, we're resurfacing our close examination of the Dunning-Kruger effect from May 25, 2012.

In 1999 a pair of researchers published a paper called "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments (PDF)." David Dunning and Justin Kruger (both at Cornell University's Department of Psychology at the time) conducted a series of four studies showing that, in certain cases, people who are very bad at something think they are actually pretty good. They showed that to assess your own expertise at something, you need to have a certain amount of expertise already.

Remember the 2008 election campaign? The financial markets were going crazy, and banks that were "too big to fail" were bailed out by the government. Smug EU officials proclaimed that all was well within the EU—even while they were bailing out a number of financial institutions. Fast forward to 2012, and the EU is looking at hard times. Greece can't pay its debt. Italy can, but the markets don't trust it to be able to. Spain and Portugal are teetering around like toddlers just waiting for the market to give them one good push. Members of the public are behaving like teenagers, screaming "F**k you," while flipping the bird. The markets are reacting like drunk parents, and the resulting bruises are going to take a while to heal.

In all of this, uninformed idiots blame the Greeks for being lazy, the Germans for being too strict, and everyone but themselves. Newspapers, blogs, and television are filled with wise commentary hailing the return of the gold standard, the breakup of the Euro, or any number of sensible and not-so-sensible ideas. How are we to parse all this information? Do any of these people know what they are talking about? And if anyone does, how can we know which ones to listen to? The research of Dunning and Kruger may well tell us there is no way to figure out the answers to any of these questions. That is kind of scary.

It has been more than 10 years since Dunning and Kruger published their work. I suspect it has become required reading in psychology courses. It's also a paper that has important implications for learning and communication, so what has happened since? Have the results held up? Are they universal? And what can we do to avoid falling victim to our own inabilities?

"The paper gave voice to an observation that people make about their peers, but that they don’t know how to express," Dunning said.

This paper has become a cult classic. It is well-written—humor interspersed with robust data, and conclusions that are discussed in a thorough and accessible way. I wondered if Dunning knew that this paper would become such a classic, and when I spoke with him he responded quite to the contrary. "I frankly thought the paper would never be published," Dunning said. "It really doesn’t fit the usual structure of a modern-day research psychology finding. A wise editor who got it and good reviewers showed me wrong there. I am struck just with how long and how much this idea has gone viral in so many areas."

Clearly, the paper struck a chord with many people outside of the field of psychology. "I presume the paper gave voice to an observation that people make about their peers but that they don’t know how to express," Dunning responded. If you have not read the paper already, I recommend doing so.

Unfortunately, in those places ruled by the smug and complacent, a classic paper has become a weapon. The findings of Dunning and Kruger are being reduced to "Stupid people are so stupid that they don't know they are stupid." Rather bluntly, Dunning himself said, "The presence of the Dunning-Kruger effect, as it’s been come to be called, is that one should pause to worry about one’s own certainty, not the certainty of others." And that humorously suggests the Dunning-Kruger effect is now a candidate to become a second Godwin's law.

Like Dunning, I do not take such a dim view of humanity. In fact, Dunning-Kruger and follow-up papers give us cause for hope. They show that people are not usually irredeemably stupid. You can teach people to accurately self-evaluate—though, in their specific examples, this also involved teaching them the very skill they were trying to evaluate.

Context is everything

It is important to realize that the Dunning-Kruger paper was not such a shocking finding. It was, for instance, already known that seemingly everyone evaluates themselves as above average in everything. Are you a better driver than average? Certainly am. How do you rate your ability at math? Oh, a little better than average. How about mountain climbing? Well, I've climbed the local hill a couple of times. I bet Kilimanjaro can't be much more difficult.

A large pile of research on various groups of people, covering various skill sets, indicates that in the face of all evidence, humans are irredeemably optimistic about their own abilities. That is, by itself, not such a bad thing. The ugly side shows up when we also realize that the norm must be maintained. Studies show that we do this by considering that everyone else is much worse. Being clueless about your own abilities is one thing. Misjudging other's abilities is relatively more serious.