The finding: The best leaders tend to be outsiders who don’t have a great deal of experience.

The research: Gautam Mukunda studied political, business, and military leaders, categorizing them into two groups: “filtered leaders,” insiders whose careers followed a normal progression; and “unfiltered leaders,” who either were outsiders with little experience or got their jobs through fluke circumstances. He then compared the groups’ effectiveness; for instance, with U.S. presidents, he looked at historians’ rankings from the past 60 years. He discovered that the unfiltered leaders were the most effective—and also the least effective—while highly filtered leaders landed in the middle of the pack.

The challenge: Is searching for a leader with a long, impressive résumé a waste of time? Is experience a predictor of mediocre performance? Professor Mukunda, defend your research.

Mukunda: I was surprised by how unambiguous the data were, but they confirmed what I suspected: If you choose an insider who you know can do the job well, most of the time that person won’t perform any differently from any other top candidate with lots of experience. Such insiders—I call them “filtered leaders”—might be good, but they probably won’t be brilliant. It’s the unfiltered leaders, the outsiders without lots of experience, who perform the very best.

HBR: So should firms always hire outsiders without experience?

No, because those people are also more likely to crash and burn. Though the best leaders—Steve Jobs, Abraham Lincoln—were unfiltered, the things that made them so effective, such as their ability to think differently and not feel beholden to a certain way of doing things, often lead to terrible results. Unfiltered leaders are high risk, high reward. Filtered leaders—like Tim Cook and Neville Chamberlain—have deep knowledge and can be very effective in a stable situation. But they often can’t adapt to extreme, sudden change or are unable to disrupt the status quo, which an outsider feels freer to do.

How can experience and knowledge be a drawback for a leader facing change?

Because they’re precisely what prevent you from approaching situations any differently than other experienced people would. Filtered leaders will usually make basically the same decisions. Even if they’re good decisions, their leadership doesn’t have impact. Think of Thomas Jefferson. According to my theory, he’s definitely filtered, so he should be in the middle of the pack in terms of his impact as a president. And he is.

But he’s consistently ranked as one of the top presidents.

This is why I used him as a counterfactual test of my theory. Why is he ranked so high? He completed the Louisiana Purchase. He doubled the size of the country peacefully. But the other filtered leaders who could have been president at that moment, Madison and Adams, would have done the same thing. In fact, Madison wouldn’t have tried to get a constitutional amendment giving the federal government the explicit power to add territory. Jefferson did, and it delayed the purchase so much it might have fallen apart, but Madison (among others) convinced him to forget about it and let the purchase move forward. Jefferson wasn’t bad, but he was not impactful. Not special. You can be a great manager, but you won’t have impact if there are 100 other great managers who would do the same thing you would.

Did you really just dis Thomas Jefferson?

He did a great job as president; he just didn’t matter that much. Madison or Adams would have done a good job, too. The very best leader is one who makes decisions no one else could, and those decisions work out.

Like who?

Lincoln is the ultimate example of the unfiltered leader. Two-time loser in Senate races, and so outside the system that he wasn’t even listed in the top 10 Republican presidential candidates in some newspapers in 1860. Most other Republican leaders thought the South was bluffing about secession and would have let them go peacefully, expecting them to return soon enough. Only Lincoln had the capacity to say “We won’t give up Fort Sumter without a fight,” to come up with a strategy that forced the South to fire the first shot, and to unite the North behind him. I think if anyone other than Lincoln had been president, the North would have lost the war—if there even was a war.

But that was an accident of history. People didn’t consciously elect an outsider in anticipation of civil war.

True. And in Britain in May 1940, when the German offensive had begun and the Chamberlain government fell, nearly everyone wanted Lord Halifax—the quintessential insider—to be prime minister. But Halifax turned the job down, and Churchill was the only other candidate. Then Churchill chose to keep fighting, which Halifax wouldn’t have done. That’s the wrinkle: Leadership doesn’t matter much, until the rare moment comes when it’s the most important thing. So you want filtered leadership, right until you need unfiltered leadership.

If leaders don’t matter much, why do companies pay them exorbitantly?

If academics undervalue leadership, the private sector overvalues it. The right unfiltered leader is definitely worth a lot, but it’s hard to know who that is, since an unfiltered leader is as likely to flame out as to succeed. Also, the market is flawed. HBS professor Rakesh Khurana explains this well. Limiting the pool of candidates based on vague notions of what’s required of a leader, like charisma, makes you believe that the number of people who could do the job is smaller than it really is. And though the market pays leaders for outcomes, you’ll often arrive at the same outcome regardless of the leader you pick.

This all sounds like rhetorical gymnastics.

It’s not, because we can actually say, based on my research, what are the specific conditions when leaders will matter. You need to understand your appetite for risk and choose the right type of leader. Often, the specific person won’t matter.

What are those circumstances?

Dominance and survival. If you want to grow to dominance, you need an unfiltered leader, someone who will think differently and take risks. The problem is, the risk taking might not pay off. But you can’t change that. Start-ups, where the most likely outcome is bankruptcy, and companies on the precipice are good situations for unfiltered leaders. But if you want to be in business in 50 years, pick a filtered leader, and remember, if you have five top candidates, it probably doesn’t matter which one you pick.

So in business, we can rig the process?

To an extent. You can structure an organization to have both kinds of leaders available and use them as appropriate. Being ready to bring in unfiltered types when your May 1940 does arrive could mean the difference between success and failure. But it starts with reframing your thinking. Choosing the “best” leader is the wrong approach to the problem. You can’t do that. You can choose leaders who are likely to lead you to big wins or big losses, or you can choose leaders who will definitely be good at their job but almost certainly won’t be great.