You’re really good at what you do and you’ve been doing it a long time. You’ve got a hunch about this big problem you’re facing…

Should you trust it? All those smarty-pants books (and smarty-pants blogs, for that matter) are telling you to be rational. Use this or that fancy logical system. Or a framework developed by the really smart professor at the prestigious University of Wherever.

But your Spidey-Sense is tingling. There’s a disturbance in The Force. This hunch just feels right… Should you trust your intuition? Or when should you trust your intuition? (And, um, what is intuition anyway?)

Gary Klein decided to study the subject. And not just in some sterile lab with a bunch of 19 year-olds for subjects. For 30 years he’s looked at intuition as it was used in real-life difficult situations by everyone from firefighters to chess masters.

He’s the author of a number of books on the subject, including The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work and Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making.

The US Marines, Army and Navy have all hired him to develop training programs for decision-making. In fact, he was one of the leaders of the team that redesigned the White House Situation Room.

We’re going to learn a lot of surprising things about intuition and how to make good decisions including: when logic is a bad idea, why jumping to conclusions can be a very smart choice, and why you might want to make more mistakes. Let’s get to it…

What The Heck Is Intuition Anyway?

No, it’s not ESP and it’s not magic. And you shouldn’t rely on it in areas where you don’t have a solid base of experience.

Intuition is the patterns you unconsciously notice in your field of expertise. When you see a particular set of cues, you recognize that this is an X, not a Y. And knowing it’s an X let’s you make a judgment and respond with the right set of actions.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

A “pattern” is a set of cues that usually chunk together so that if you see a few of the cues you can expect to find the others. When you notice a pattern you may have a sense of familiarity— yes, I’ve seen that before! As we work in any area, we accumulate experiences and build up a reservoir of recognized patterns. The more patterns we learn, the easier it is to match a new situation to one of the patterns in our reservoir. When a new situation occurs, we recognize the situation as familiar by matching it to a pattern we have encountered in the past.

And if you’re good at what you do, you should be relying on intuition more, not less. Leveraging intuition isn’t lazy — it’s the hallmark of experts.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

We discovered that the more experience people have in any particular field, the more they rely on intuition, and ultimately we learned that intuition is a natural and direct outgrowth of experience. I define intuition as the way we translate our experience into action. Our experience lets us recognize what is going on (making judgments) and how to react (making decisions). Because our experience enables us to recognize what to do, we can therefore make decisions rapidly and without conscious awareness or effort.

Now rationally and deliberately processing all the potential possibilities is powerful too, and that’s why when you’re making a left turn in your car — a potentially lethal endeavor — you always take the time to consider every single…

No, you don’t. You just use your intuition.

(To learn the 7-step morning ritual that will make you happy all day, click here.)

But hold on now. Maybe you just read an article about that fancy logical method for improving decision-making. I mean, they even teach it at top business schools! That’s gotta be better than going with your gut…right?

Your Rational System Shall Not Defeat My Intuitive Kung Fu

So you use the cool system-of-the-month for making decisions. Maybe you list out all the relevant criteria in an Excel spreadsheet. You score all the factors from 1 to 10 so they’re properly weighted. It’s logical, rational and systematic. There’s just one little problem…

Research shows those systems don’t work in the real world.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

The only problem is that the whole thing is a myth. The reality is that the classical model of decision making doesn’t work very well in practice. It works tolerably well in the research labs which use undergraduate test subjects making trivial decisions, but it doesn’t do so well in the real world, where decisions are more challenging, situations are more confusing and complex, information is scarce or inconclusive, time is short, and stakes are high. And in that environment, the classical, analytical model of decision making falls flat… Considering all these drawbacks, it’s not surprising that decision researchers haven’t been able to demonstrate that analytical methods actually help people make better decisions.

In fact, Klein cites studies showing these systems can actually produce decisions that are worse than you would have come up with otherwise. Why?

Because they interfere with your intuition.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

There are even several studies that show that the use of analytical methods results in worse decisions. The reason is that these methods seem to interfere with intuition.

Think for a second: how did you go about defining the problem to use in that system? How did you come up with those numerical values to weight each factor?

Yeah. You used your intuition. No doubt, you need both rational decision making and intuition in your mental toolbox. But studies show when you ignore your intuition, decision quality drops.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

There are data showing that when people ignore intuition, decision quality goes down. It also goes down when people are instructed to use decision analysis. Decisions are made subconsciously before people even start to perform the analyses, and the very act of articulating the factors can make decisions less reliable. The evidence is growing that those who do not or cannot trust their intuitions are less effective decision makers, and that as long as they reject their intuitions, they are destined to remain so.

But why? Because neuroscience. Your brain can’t make good decisions without emotions.

Antonio Damasio is a neuroscientist at USC. He did an experiment using decks of cards, first with students and then again with people who had suffered damage to the ventromedial sector of their prefrontal cortex.

Students played a game where they drew cards from 4 decks and received monetary rewards or penalties depending on the cards selected. What they didn’t know was the decks were totally rigged. Some decks gave bigger rewards, others big penalties.

After playing for a while — but before they realized the trickery — the students started showing anxiety responses before drawing from the penalizing decks. Their intuition had noticed the pattern before their conscious mind was aware of it. Emotions can help you make better decisions, even when you don’t consciously know what’s going on.

And what about the group that suffered from brain damage? They never showed anxiety. Ignoring your intuition is like being brain damaged. Literally.

(To learn the 4 rituals neuroscience says will make you happy, click here.)

Okay, but certainly there are times when logic and math triumph. What about Moneyball? Scouts used to go by intuition alone and they were terrible at picking baseball players compared to algorithms. No doubt.

So when is the best time to use intuition and when should we be all Spock-like and rational?

Puzzles Vs. Mysteries

Baseball is a game of stats. It’s a clear system with clear answers. When answers are that straightforward, lead with logic.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

The claim that successful decision makers rely on logic and statistics instead of intuition matters because systematic analysis may work for well-ordered tasks. But it runs into difficulty in complex settings, and it leads to overthinking.

A good distinction you should use here is between “puzzles” and “mysteries.”

“Where is Eric right now?” is a puzzle. It has a clear, definitive answer. If you hacked the GPS on my phone, boom, you’d know.

“Where will Eric be Thursday at 2PM?” Now that’s a mystery. Even I don’t know the answer to that one. Nobody does.

If you’re trying to solve a puzzle, one piece of data can resolve the issue. If you’re dealing with a mystery, no amount of information will give you a definite answer. But in the modern world, we’ve become data junkies.

We think more information can solve all our problems. And that’s not the case. In fact, the more information you have the more confident you get, but once information exceeds a certain threshold the more your accuracy drops.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

Look at Mauboussin’s (2007) study of horse-racing handicappers. The more information the handicappers got, the more confident they were, just as Oskamp found with clinicians. Further, the handicappers’ predictions were less accurate when they got 40 pieces of information than when they got five pieces.

Lots of information is hard to sort through, organize and evaluate. And when you’re dealing with a mystery, more info is never going to guarantee the right answer. So those rational systems break down. Guess what can fill the gap?

When we have lots of factors to consider, we do better when we’re actually prevented from thinking consciously. With complex decisions, intuition rules.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

…Dijksterhuis’s study of shoppers…which showed that when subjects had to consider twelve variables instead of four they made better decisions when they were prevented from deliberating consciously… Nygren and White (2002) reported that college subjects who relied on an analytical style of decision making performed worse on a complex decision-making task in a flight simulator.

So start with intuition. You can always use logical analysis later. But when we try to be rational first, it mucks with our intuition.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

If you begin by analyzing a decision, you are inevitably going to suppress your intuition. You’re best off starting by getting a sense of your intuitive preference— a gut check of your immediate preference, identifying your intuition before it gets clouded.

And when you ignore you intuition, bad things can happen. On September 26, 1983 — the height of the Cold War — the Soviet early warning system went off. It said 5 US nuclear missiles were headed for Russia. Lieutenant Colonel Stanislav Petrov was supposed to launch a counterstrike as soon as that alarm went off. His nation was under attack…

But his intuition asked him a question: Why would the US start World War 3 by only launching five missiles? There was no way that would cripple the USSR’s ability to retaliate.

So he didn’t launch the counter-strike. And it turned out to be a false alarm. Sunlight reflecting off of clouds had been misinterpreted by the satellites as an attack.

Listening to his intuition prevented World War 3.

(To learn how to have more grit and resilience — from a Navy SEAL — click here.)

Okay, so you’re trying your intuition first. Good. What’s the best way to do that? Well, it’s going to sound like heresy…

Jump To Conclusions

I know, you’re “supposed to” generate lots of options and compare them. But if you’re good at what you do, that’s a bad idea. When Klein studied firefighters — who make life or death decisions — he guessed they would think of two options and compare them. He was wrong…

They didn’t compare any. What did his research team hear over and over again from different firefighters? “We don’t make decisions.” They went with their first thought. And it was usually right.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

We interviewed 26 highly experienced firefighters— commanders who averaged 23 years of experience. That’s more than 500 years of cumulative wisdom. To our surprise, the commanders insisted that they didn’t compare any options. They looked at a situation and just knew what to do. That created two mysteries: How could they be so confident in their initial impulses? How could they evaluate an option except by comparing it to others? The answer to the first mystery is that with 23 years of experience you recognize situations as familiar, or typical. The patterns you recognize include typical ways to react. Therefore, the first option the commanders thought of was usually going to get the job done. They were satisficing (looking for a good workable option, not necessarily the best), and their first impulses were very effective.

Now novices should compare options — they don’t have experience to rely on. But experts can actually do themselves a disservice by overthinking. When radiologists look at x-rays for much more than 30 seconds, their decision-making gets worse.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

Radiologists worry about overthinking. If they look at a film too long, they start seeing things that aren’t there. After about 38 seconds, they begin to overreact to slight irregularities in normal structures and identify non-existent malformations.

(To learn the 6 rituals ancient wisdom says will make you happy, click here.)

So just always do the first thing that pops into your head? Sound extreme? Sound lazy? Hold on. That’s just part one…

Tell Yourself A Story

Yes, experts usually select one option and it’s usually right. But before they execute it, they run a “mental simulation.” They do a walk through in their head of what will happen if they act on the idea.

If it seems like it will work, jives with their experience, and they don’t see any problems, it’s go go go.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

When we looked at their decision making more closely we discovered that they were evaluating a course of action by consciously imagining what would happen when they carried it out. We call this process “mental simulation” because decision makers are simulating and envisioning a scenario— playing out in their heads what they expect would happen if they implemented the decision in a particular case. They build a picture of what they expect, and they watch this picture once, sometimes several times. If they like what they see, they are ready to respond. If they spot a problem, usually they can alter the action script. If they can’t find a way around the problem, they jettison the option and look at the next option in the set without comparing it to any other options.

(To learn the number one decision-making tip of samurai, astronauts and Navy SEALs, click here.)

So all of this intuition research hinges on you being very good at what you do. But what if you’re not? What if you’re new to a field? How do you build expertise and intuition?

Improve By Focusing On Understanding (And Screwing Up)

Intuition gets built by experience. So get exposed to as many decisions in your domain as possible.

From The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:

Intuitive decision making improves as we acquire more patterns and larger repertoires of strategies…The more patterns and action scripts we have available, the more expertise we have, and the easier it is to make decisions. The patterns tell us what to do and the action scripts tell us how. Without a repertoire of patterns and action scripts, we would have to painstakingly think out every situation from scratch.

And that can be accelerated the way any other skill is — by practice. Look at the most common decisions that get made. Review prior decisions and how they worked out. Talk to people who know more than you do.

All of that is pretty obvious. So what’s the secret that separates the people who develop expert intuition from someone who has merely memorized standard procedure?

Focus on understanding the underlying causes and effects. Look at the relationships between how things work. Dig for insights, don’t just collect data and follow the rules.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

…mediocre (weather) forecasters relied on procedural guides when collecting data, and also when turning the data into forecasts…In contrast, the highly skilled forecasters tried to understand what was going on. They foraged for data that helped them build a better understanding, and used their understanding to make predictions.

Don’t worry about screwing up here and there. In fact, make sure to screw up. When you focus on reducing mistakes you limit your ability to build expertise. You need to try things and fail to really learn the patterns that produce intuition.

From Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making:

In many cases, our aversion to mistakes may be counter-productive. We must make mistakes in order to learn. Deakin and Cobley (2003) found that the most elite figure skaters fall more often than others during practice sessions because they spent more time attempting jumps they hadn’t mastered. If we discouraged the skaters from falling, they wouldn’t learn as quickly.

(To learn what Harvard research says will make you happier and more successful, click here.)

Do you have a gut instinct that we’re almost done here? Your intuition is right. Let’s round it all up…

Sum Up

Here’s how to master intuition:

Intuition is unconsciously noticing patterns that aid decision-making : Just because you can’t explain them doesn’t mean you should dismiss them. But only trust them if you’re experienced in the domain.

: Just because you can’t explain them doesn’t mean you should dismiss them. But only trust them if you’re experienced in the domain. Formal rational systems often fail in the real world : Your brain can’t make decisions without emotions.

: Your brain can’t make decisions without emotions. Puzzles vs mysteries : Puzzles have clear answers, so use rational processes. Mysteries don’t, so leverage intuition.

: Puzzles have clear answers, so use rational processes. Mysteries don’t, so leverage intuition. Jump to conclusions : Experts go with their first idea. And they’re usually right.

: Experts go with their first idea. And they’re usually right. Tell yourself a story : Walk through what will likely happen before following through on your plan.

: Walk through what will likely happen before following through on your plan. Improve by focusing on understanding (and screwing up): People with solid intuition don’t just look at data, they understand how all the moving parts relate. And if you’re not screwing up, you’re not learning.

Albert Einstein once said:

The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift.

Did this not answer every single question you have? Did I not spell everything out as clearly as possible? Are you not sure exactly what step to take next? That’s okay.

You’ll just have to use your intuition.

Join over 280,000 readers. Get a free weekly update via email here.

Related posts:

New Neuroscience Reveals 4 Rituals That Will Make You Happy

New Harvard Research Reveals A Fun Way To Be More Successful

How To Get People To Like You: 7 Ways From An FBI Behavior Expert

Eric Barker is a writer who has been featured in the The New York Times, The Wall Street Journal, Wired and TIME. He also runs the Barking Up the Wrong Tree blog. Join his 205,000-plus subscribers and get free weekly updates here. This piece originally appeared on Barking Up the Wrong Tree.