Daniel Kahneman talks extensively about how we make reasoning errors because we tend to use mental shortcuts. One mental shortcut is ‘substitution‘. Substitution is what we do when we (often unconsciously) answer an easier question than the one being asked. I find that I sometimes do this in my own research. For instance, when I set out to answer the question, “How can X be rational?” I sometimes end up answering easier questions like, “How does X work?”. In an effort to avoid such mistakes, I will (1) explain the question substitution error, (2) give an example of how we can distinguish between questions, (3) give a personal example of the substitution error, and (4) say what we can do about it.

1. Substitution

In case you’re not familiar with Kahnemen’s notion of ‘substitution’, here is some clarification. In short, substitution is this: responding to a difficult question by (often unintentionally) answering a different, easier question. People use this mental shortcut all the time. Here are some everyday instances:

Difficult Question Easier Question How satisfied are you with your life? What is my mood right now? Should I believe what my parents believe? Can I believe what my parents believe? What are the merits/demerits of that woman who is running for president? What do I remember people in my community saying about that woman?

For further discussion of mental shortcuts and substitution, see Part 1 of Kahneman’s Thinking Fast and Slow (2012).

Now, how does this mental shortcut apply to research?

2. Some Research Questions

Regular readers will know that I am interested in — among other things — the differences between reflective and unreflective reasoning (Pennycook et al 2015). Philosophers ask lots of questions that involve reflective reasoning:

Normative Questions

Must I do this? (Korsgaard 1996)

Should I believe this? (Sosa 2009)

Explanatory questions

Why do I believe that? (Kornblith 2012)

Why did I do that? (Doris 2015; Vargas 2005)

Self-knowledge questions

What do we know about our own minds? (Cassam 2015)

How is my relationship to my own mind different than my relationship to other minds? (Moran 2001).

Obviously there are other questions that involve reflective reasoning. These are just a few. And, obviously, there are also hybrid questions that involve reflective reasoning. For instance, there are normative-explanatory questions like “Why is it wrong to X?” And there are normative-explanatory-self-knowledge questions like, “Why do I believe that it is wrong to X?

The point of this section is just to point out that we can — when we are careful — discern clear differences between similar questions.

3. Falling Prey To Substitution

Like any researcher, I don’t want to make an erroneous mental shortcut in my research. Alas, I sometimes find myself doing exactly that. For instance, I will substitute an answer to an easier question for an answer to my more difficult, research question.

Research Question Easier Question How can reflective reasoning be rational? How is reflective reasoning related to our behavior? How does reflective reasoning work? How does reflective reasoning fail?

I sometimes mistake answers to these easier questions for answers to the target question. (Based on my reading of the literature, I do not think that I am the only one who does this. But that’s a much longer conversation.)

Don’t get me wrong. We might have to answer the easier questions as a means to an answer to the target question. But we shouldn’t mistake the means for the end. That is, we can’t think that the answers to the easier questions just are answers to the target question.

4. General Take-aways

Consuming research. When a researcher tells us that they plan to answer a certain question, we should take note of that question. And then we should pay attention to see whether their research, argument, etc. can and/or actually does answer their target question. If the research can’t or doesn’t answer the original question, then we should take note of it. Heck, we might want to do more — e.g., tell the researcher.