Individual differences in the mere willingness to think analytically has been shown to predict religious disbelief. Recently, however, it has been argued that analytic thinkers are not actually less religious; rather, the putative association may be a result of religiosity typically being measured after analytic thinking (an order effect). In light of this possibility, we report four studies in which a negative correlation between religious belief and performance on analytic thinking measures is found when religious belief is measured in a separate session. We also performed a meta-analysis on all previously published studies on the topic along with our four new studies (N = 15,078, k = 31), focusing specifically on the association between performance on the Cognitive Reflection Test (the most widely used individual difference measure of analytic thinking) and religious belief. This meta-analysis revealed an overall negative correlation (r) of -.18, 95% CI [-.21, -.16]. Although this correlation is modest, self-identified atheists (N = 133) scored 18.7% higher than religiously affiliated individuals (N = 597) on a composite measure of analytic thinking administered across our four new studies (d = .72). Our results indicate that the association between analytic thinking and religious disbelief is not caused by a simple order effect. There is good evidence that atheists and agnostics are more reflective than religious believers.

Introduction

Dual-process theories distinguish between two fundamentally different types of cognitive processes [1]: Type 1 processes that are intuitive and autonomously cued and Type 2 processes that are reflective and require working memory. One of the most important insights that has emerged from the dual-process literature is that the distinction between intuition and reflection is of consequence to more than just researchers interested in thinking and reasoning [2]. For example, the propensity to engage analytic reasoning (as distinct, conceptually, from cognitive ability) predicts paranormal disbelief [3, 4], acceptance of science [5, 6], less traditional moral values and judgments [7, 8], less reliance on Smartphones as an external source of information [9], and a lowered receptivity to bullshit [10]. These results indicate that the interplay between intuitive and analytic processes is an important component of human cognition. The degree to which analytic (Type 2) processes influence reasoning and decision making in lab studies is at least somewhat predictive of specific theoretically grounded real world outcomes [2].

Most importantly for the present investigation, there is now a good deal of evidence that analytic thinking disposition (“analytic cognitive style”) is negatively associated with religious belief [11]. For example, religious believers perform worse than non-believers on cognitive tests that cue incorrect intuitive responses [4, 12, 13]. Consider, for instance, the bat-and-ball problem from the Cognitive Reflection Test (CRT; [14]): “A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?” The majority of participants respond “10 cents” to this question (e.g., 64.9% in [15]). This is the response that typically comes to mind upon an initial read of the problem, but it is incorrect. If the ball is $0.10, then the bat must be $1.10 and in total they would be $1.20. To recognize that the intuitive response is incorrect, the participant must be willing to stop and think analytically about the problem despite having what seems to be a plausible intuitive response. Performance on the CRT is therefore thought to index, at least to some degree, a willingness or propensity to engage Type 2/analytic processing [14, 16, 17, 18]. Naturally, it also requires some degree of cognitive ability (and, in particular, numeracy) to solve the arithmetic involved in each of the problems [19]. As a consequence, a number of studies have also demonstrated that there is a negative association between religious belief and CRT performance (and performance on related tasks) even after controlling for measures of cognitive ability and intelligence ([4, 7, 13, 20], but see [21]). This negative association also remains robust in regression analyses that control for various demographic factors (e.g., age, sex, education; see [11] for a review). Furthermore, religious believers spend less time reasoning when given problems in a lab study [20, 22], as would be expected if they are less willing to engage slow, deliberative reasoning processes. Finally, there is experimental evidence that inducing an analytic or reflective mindset at least temporarily decreases self-reported religious belief [12, 13, 23].

These results were challenged in a recent paper by Finley, Tang, and Schmeichel [24]. Specifically, they hypothesized that the association between analytic thinking and religious belief depends on the order in which analytic thinking and religious belief measures are presented. In a high powered experiment, the authors replicated the negative association between religious belief and CRT performance when the CRT was administered immediately prior to the religious belief measure. However, they did not find an association when the CRT was administered after the measure of religious belief. This is a potentially important finding because many of the previously cited studies measured analytic thinking before religious belief (e.g., [4, 7, 12, 20]).

Importantly, Finley et al. do not question the claim that analytic thinking decreases religious belief, which is supported by experimental evidence [12, 13, 23]. Rather, they argue that “more analytic thinkers are not necessarily less religious” (abstract, p.1). According to Finley et al., the idea that analytic thinking decreases religious belief at the state level does not necessarily contradict the claim that they are not associated at the trait level. Experiments in which analytic thinking decreases religious belief serve only as an existence proof–it is still quite possible that analytic thinking is not a meaningful component of religious cognition in most people’s everyday lives. In other words, the modal analytic thinking disposition may not be sufficiently analytic to be of consequence for religious belief. This line of reasoning seems to contradict the wealth of data that indicates that analytic thinking does have consequences for our everyday lives (as summarized above; see [2] for a review), but religious belief may be an exception.

Finley et al. argue that asking participants to first indicate their religious belief and then answer reasoning problems is the purest way to test whether there is a genuine association between the two variables. In support of this contention, the authors point to previous research wherein moral judgment was only associated with CRT performance if the CRT was given prior to the moral judgment task [25]. It may be that the CRT (and, presumably, related measures) induces an analytic mindset, which then somehow interacts with self-reported religious belief. Finley et al. are, unfortunately, not entirely clear about why they make this prediction. They state that (p. 2): “By having participants complete the CRT prior to reporting religious beliefs, Gervais and Norenzayan likely activated an analytic mindset in participants who answered analytically (i.e., correctly), temporarily suppressing religious beliefs. As a result, the strength of the relationship between individual differences in analytic thinking and religious beliefs may have been inflated by first activating the analytic system, rather than representing a pure (i.e., confound free) relationship between individual differences in analytic thinking and religious beliefs.” Finley et al. qualify their claim by arguing that only those participants who are particularly analytic in the first place would be put in an analytic mindset by the CRT. However, if this is the case, only those participants who have religious beliefs in the first place would have their religious beliefs suppressed. Presumably, then, the differences between religious disbelievers (those scoring near zero on a religious belief measure) and the various levels of religious believers would decrease if the CRT is presented first. Or, perhaps, there is a more complex interaction that requires a necessary level of analytic cognitive style for the CRT to induce an analytic mindset and that only affects some levels of religious belief and not others. These speculations illustrate that the mechanisms proposed by Finley et al. are not at all straightforward. Even if the association between CRT and religious belief is only evident when religious belief is measured first, it is not clear why this would be the case.

Given this uncertainty, the key question is whether one or the other presentation order is more likely to reflect the actual association (or lack thereof) between analytic thinking and religious belief. There is some extant data that indicates that, contrary to Finley et al.’s claims, the apparent lack of correlation when religious belief was measured first in their study is the exception, not the rule. First, six published studies (2 of which were published before Finley et al.) had participants indicate their religious belief prior to solving the CRT and found significant associations [26, 27, 28, 29]. In each case, however, the additional tasks were included in-between the religiosity measure and the CRT. This may explain the discrepancy between these studies and Finley et al. (although, of course, they cannot be accounted for given Finley et al.’s hypothesis about scale administration order).

Second, there is evidence for the association from large surveys with intervening tasks separating analytic thinking and religious belief measures (e.g., [7, 10, 13, 30]). Finley et al. disputed this evidence by suggesting that these surveys may have included other measures that could have induced an analytic mindset. We are not convinced. A number of very different intervening tasks have been used, including, for example, a moral values questionnaire [7], a bullshit receptivity scale and ontological confusions measure [10], and demographic questions [30]. Regardless, it is possible that the CRT induces an analytic mindset that persists during these intervening tasks.

Third, some investigations of analytic thinking and religiosity have included categorical variables that should, in theory, be less susceptible to contextual or experimental modification. For example, Pennycook et al. [4] found an association between CRT performance and the type of God belief (or disbelief). Namely, those who believed in a conventional “personal” God scored the lowest and those who lacked any belief in God (i.e., atheists) scored the highest on the CRT. It seems unlikely that an analytic mindset would be sufficient to make a personal God theist into an atheist in the context of a single study session.

Finally, Pennycook et al. [22] did not measure religious belief in the same study session as analytic thinking and nonetheless found a negative association between the variables. This is inarguably the purest way to investigate a possible association. For this, Pennycook et al.’s participants completed a religious belief scale in a separate “mass testing” survey that opened for participation at the beginning of the semester. The mass testing survey included a number of scales that were submitted by numerous research groups in the same department (e.g., boredom proneness, a Big Five personality scale), but no analytic thinking measures. Importantly, the order of the scales was randomized for each participant, with the exception of a boredom proneness questionnaire that always came first and an aggression questionnaire that always came last. In this case, religious belief was measured prior to analytic thinking and, more importantly, in the context of an entirely different study.

Pennycook et al. [22] also reported an association between response time on a base-rate neglect task and self-reported religious affiliation. This question was completed in a “pre-screen” questionnaire that was separate from both the analytic thinking tasks and the religious belief scale (Note that although the pre-screen questionnaire is typically used for data screening, we did not restrict participation in the discussed study [22] or in any of the new studies reported here). For this, participants simply selected which of a series of religious affiliations or disaffiliations they most strongly identified with (e.g., Christian, Muslim, Agnostic, Atheist, None). The pre-screen questionnaire only included demographic questions of this sort and the religious affiliation question came after questions about ethnic identification. Regardless, as mentioned above, it seems unlikely that an analytic mindset (if it were somehow present) would cause someone who identifies as a Christian to suddenly identify as an atheist (for example).

These results represent a strong challenge to Finley et al.’s claim that there is no genuine association between analytic thinking and religious disbelief. It is unlikely that participants were in a particularly analytic mindset when indicating their level of religiosity in the mass testing and pre-screen surveys, yet the association was present in each of these studies. Nonetheless, it may still be the case that “the link between analytic thought and religious belief is more tenuous than previously reported” (p. 1). As such, we report four additional studies in which analytic thinking and religious belief were measured in separate sessions. In these studies, religious belief was measured in a mass-testing survey administered to university students at the beginning of four different academic semesters. Participants also indicated their religious affiliation (or lack thereof) in a separate pre-screen questionnaire. Participants who completed these questionnaires were then permitted to sign up for an online study with a battery of cognitive tasks. However, when this permission was granted there was no connection made between the online study and the mass testing or pre-screen surveys (i.e., the online study was referred to as a “Thinking Styles and Reasoning” study and neither the mass testing nor pre-screen sessions were mentioned). Finding a negative correlation between performance on analytic thinking and religious belief and affiliation across each of the four studies would constitute strong evidence for a veridical trait-level relation. The scales that were administered directly before the religious belief measures differed across the four studies (see S1 Text for a breakdown) and their order within the mass testing survey was always randomized. As such, the likelihood that any association can be explained by an unexpected confound (which, in our view, is low in the first place) decreases with the increasing number of studies.