What Matters More in Decisions: Analysis or Process?

We all make decisions. Some of them are large and many of them are small. Few of us understand that the process we use to make those decisions is more important than the analysis we put into the decision.

***

Think of the last major decision you made.

Maybe it was an acquisition, a large purchase, or perhaps it was whether to launch a new product.

Odds are three things went into that decision: (1) It probably relied on the insights of a few key executives; (2) it involved some sort of fact gathering and analysis; and (3) it was likely enveloped in some sort of decision process—whether formal or informal—that translated the analysis into a decision.

Now how would you rate the quality of your organization’s strategic decisions?

If you’re like most executives, the answer wouldn’t be positive:

In a recent McKinsey Quarterly survey of 2,207 executives, only 28 percent said that the quality of strategic decisions in their companies was generally good, 60 percent thought that bad decisions were about as frequent as good ones, and the remaining 12 percent thought good decisions were altogether infrequent.

How could it be otherwise? Product launches are frequently behind schedule and over budget. Strategic plans often ignore even the anticipated response of competitors. Mergers routinely fail to live up to the promises made in press releases.

The persistence of problems across time and organizations, both large and small, indicates that we can make better decisions.

“I have no use whatsoever for projections or forecasts. They create an illusion of apparent precision. The more meticulous they are, the more concerned you should be. We never look at projections.” — Warren Buffett

The best place to start if we’re trying to improve the quality of our decisions is to look at how organizations make decisions. One interesting thing about bureaucracies is that they develop processes to limit the damage the worst people can do at every level. That is they come up with mechanisms to reduce the impact the worst people can have. Yes, this also limits the positive impact that people can have as well. When it comes to decisions, organizations default to gathering data and analyzing decisions.

The widespread belief is that analysis reduces biases. But does it?

Is putting your faith in analysis any better than using your gut? What does the evidence say? Is there a better way?

Dan Lovallo and Olivier Sibony set to find out.

Lovallo is a professor at the University of Sydney and Olivier is a director at McKinsey & Company. Together they studied 1,048 “major” business decisions over five years. The results are surprising.

Most business decisions were not made on “gut calls” but rather rigorous analysis. And yet they were poor decisions. In short, most people did the all the legwork we think we’re supposed to do: they delivered large quantities of detailed analysis.

Yet this wasn’t enough. “Our research indicates that, contrary to what one might assume, good analysis in the hands of managers who have good judgment won’t naturally yield good decisions.”

[Projections] are put together by people who have an interest in a particular outcome, have a subconscious bias, and its apparent precision makes it fallacious. They remind me of Mark Twain’s saying, ‘A mine is a hole in the ground owned by a liar.’ Projections in America are often a lie, although not an intentional one, but the worst kind because the forecaster often believes them himself.” — Charlie Munger

***

Lovallo and Sibony didn’t only look at the analysis, they also asked executives about the process used to make decisions.

Did they, for example, “explicitly explore and discuss major uncertainties or discuss viewpoints that contradicted the senior leader’s?”

So what matters more, process or analysis? After comparing the results they determined that “process mattered more than analysis—by a factor of six.”

This finding does not mean that analysis is unimportant, as a closer look at the data reveals: almost no decisions in our sample made through a very strong process were backed by very poor analysis. Why? Because one of the things an unbiased decision-making process will do is ferret out poor analysis. The reverse is not true; superb analysis is useless unless the decision process gives it a fair hearing.

To illustrate the weakness of how most organizations make decisions, Sibony used an interesting analogy: the legal system.

Imagine walking into a courtroom where the trial consists of a prosecutor presenting PowerPoint slides. In 20 pretty compelling charts, he demonstrates why the defendant is guilty. The judge then challenges some of the facts of the presentation, but the prosecutor has a good answer to every objection. So the judge decides, and the accused man is sentenced. That wouldn’t be due process, right? So if you would find this process shocking in a courtroom, why is it acceptable when you make an investment decision? Now of course, this is an oversimplification, but this process is essentially the one most companies follow to make a decision. They have a team arguing only one side of the case. The team has a choice of what points it wants to make and what way it wants to make them. And it falls to the final decision maker to be both the challenger and the ultimate judge. Building a good decision-making process is largely ensuring that these flaws don’t happen.

Simply understanding our cognitive biases doesn’t make you immune to them. It’s not enough. A disciplined decision process is the best place to improve the quality of decisions and guard against common decision-making biases.

Still curious? Read the ultimate guide to making smart decisions.