By applying the experimental design introduced in Sec. 2 we gathered (what we deem to be) more valid and philosophically relevant data about the folk’s metaethical intuitions than previous research. In what follows we will describe the participants, methods and results of our study.

Participants

We surveyed 172 participants. Of that, 98 participants were from Amazon MTurk (49% male) and were paid $7.25; and 74 participants were from the College of Charleston (26% male) and received two research credits.

Prior to analysis the quality of participants’ responses was assessed according to the following criteria (in the order of their weighting): (1) performance in attention checks (failure in any check automatically eliminated participants from the study), (2) study completion time, both overall and for each individual task, (3) performance in comprehension checks, (4) minimal relevance of verbal explanations. On the basis of these criteria 55 participants (32%) were excluded from analysis. For example, we did not consider the responses of participants who finished the complete survey in less than 20 min (the average time was 45 min); who copied their verbal explanations from Wikipedia and other online sources; and who took the first option in all comprehension checks.

In the end we had 117 participants, 67 from MTurk and 50 from the College of Charleston. Of these participants 63% were female; their age varied between 18 to 64 years (M = 29.6); and they were 86% Caucasian, 4% African American, 4% Asian American, 3% Hispanic, and 3% other.

Methods

The survey was administered online through Qualtrics. Each participant was presented both the abstract measures (theory, metaphor, comparison, disagreement and truth-aptness tasks) and the concrete measures (disagreement and truth-aptness tasks). The order of these measures was counterbalanced, and many of the individual tasks, as well as most of the answer options, were randomly ordered. Responses to the tasks were coded as being indicative of realist or anti-realist intuitions according to the information added in square brackets in Sec. 2.

One potential worry about our methodology is that it still fails to cover many main variants of subjectivist anti-realism (W2) as well as main variants of realism (a worry that has not been addressed in Sec. 1.2 at all). This observation is correct. For example, our answer options neither reflect response-dependence theory (a main variant of subjectivist anti-realism) nor do they disentangle naturalism from non-naturalism (two main variants of secular realism). That said, it would seem methodologically overdemanding to require of any single study to cover all or even only most positions in the realism/anti-realism debate. Our study provides first helpful data because of the particular selection that we made. The positions that were included are either such that we would pre-experimentally expect them to be popular among the folk (other than, e.g., response-dependence theory) or that they necessitate additional answer options or scenarios in our tasks (other than, e.g., naturalism and non-naturalism, which are both covered by our realist answer options).Footnote 16

Critics may also question our classification of divine command theory as a (theistic) variant of realism. According to divine command theory, a thing is morally right, good, virtuous, etc. if and only if it is commanded by God, and morally wrong, bad, vicious, etc. if and only if it is forbidden by God.Footnote 17 This renders moral facts mind-dependent — dependent on the mind of God (e.g., Huemer 2005: 54–55). Yet, it bears noting that God may have several metaethically relevant features that most or all non-divine observers lack, such as that he/she is omni-perfect, that his/her commands never change or that these commands apply to all people at all times and places. The broad majority of metaethicists (e.g., Austin 2006; Evans 2013; Joyce 2007) hence (implicitly) define moral realism as only requiring independence from the mental states of non-divine observers. On this definition — which is assumed in this paper as well — divine command theory does turn out to be a variant of realism.Footnote 18

Another important methodological question is which proportion of a participant’s intuitions about concrete cases must favor realism or anti-realism in order for the participant to count as being drawn towards these views. Elsewhere one of us has argued that from a metaethical perspective there must not be any genuine intrapersonal variation at all (Pölzler 2017, 2018b). That said, when a participant is asked to give many responses, as in our concrete tasks, the likelihood of performance errors increases. We thus stipulated that participants have “consistently” realist or anti-realist intuitions if 100% of their responses to the abstract tasks and at least 90% of their responses to the concrete tasks were in favor of only one of these views. This is the threshold above which the likelihood of realist or anti-realist intuitions having an effect on the participant’s responses is significantly greater than the likelihood of there being no such effect, as calculated by a binomial test, based on the number of separate tasks involved in each set of measures.

Finally, most of our tasks involve four anti-realist but only two realist answer options. It may be objected that this unequal distribution biases our results in favor of anti-realism. We agree that this feature is potentially problematic, especially to the extent that participants answer without sufficient effort or comprehension. But it is also hard to avoid. Our study’s most important tasks are the abstract and concrete disagreement tasks. This is because these tasks have been most widely used, have been most thoroughly refined, and are most likely to reveal more implicit intuitions, as they may be central to metaethical arguments (see Sec. 4). Prior research has shown that in order for these disagreement tasks to be minimally valid researchers need to fully tease apart metaethical positions that entail additional answer options (see W1 and W2; see also Pölzler 2018a, 2018b and fn. 17). By doing so, however, one automatically ends up with at most two options that entail realism, and a larger number of options that entail anti-realism.

To alleviate the problem of our unequally distributed answer options, we decreased insufficient effort and insufficient comprehension responding in various ways, including requiring verbal explanations from participants (see Sec. 3.1). This makes it more likely that those who opted for anti-realist options really felt drawn towards these options. Moreover, in an independent study we also confirmed that our disagreement tasks deliver plausible results for non-moral domains (scientific statements were dominantly rated as realist, and statements about social conventions and personal preferences were dominantly rated as anti-realist).Footnote 19

Results

Comprehension Checks

Our metaethics/normative ethics comprehension check involved a theoretical question and a classification exercise (see Sec. 2.1). 114 participants (97%) answered the theoretical question correctly on the first pass, and all three who got it wrong successfully corrected their answer on the second pass. The classification exercise showed a marked difference in performance between the Mturk and student participants. On average, 69% of the Mturkers classified all 14 sentences correctly at first pass, with the correction rate being 79%. In contrast, only 35% of the student participants initially classified all sentences correctly, with the correction rate being 72%.

Participants had to complete several additional comprehension and memory checks as well. 81%, 82%, 94%, and 71% correctly selected what the abstract theory, metaphor, comparison, and disagreement tasks were about; and our question of what had been presented in the concrete disagreement tasks received 83%, 75%, 75%, 72% and 77% correct responses. The MTurk participants again did somewhat better on most of these checks. This also holds for the extensive comprehension checks prior to each participant’s first truth-aptness task. While both the MTurkers and students mostly correctly answered our theoretical question (69% and 70%), the MTurkers did better at our classification exercise (48% versus 20% correct classifications at first attempt) and were able to correct more of their misclassifications (86% versus 69%).

In total, the analyzed participants’ performance in the comprehension checks was good; and where possible, mistakes were mostly corrected (indicating that participants learned from their mistakes).

Realist/Anti-Realist Responses

Collapsing across all of the opportunities participants had to give either a realist or an anti-realist response, and then averaging these together, we found that participants dominantly gave anti-realist responses. In response to the abstract measures the proportion of anti-realist responses was 77%, and in response to the concrete measures 89% (as opposed to 23% and 11% of realist responses) (see Tables 1 and 2). The order of the measures’ presentation did not have any significant effect; that is, participants who received the abstract measures first responded in the same way as participants who received the concrete measures first.

Table 1 Proportion of realist and anti-realist responses in abstract measures Full size table

Table 2 Proportion of realist and anti-realist responses in concrete measures. Upper value = disagreement task, lower value = truth-aptness task Full size table

Most anti-realist responses were anti-realist in a cognitivist sense. On average, 73% of the responses to the abstract and 76% of the responses to the concrete truth-aptness tasks were in favor of moral sentences being truth-apt. Almost all other tasks showed an even higher proportion of cognitivism vis-à-vis non-cognitivism. In particular, participants’ responses dominantly indicated intuitions in favor of cultural relativism (36% in the abstract and 24% in the concrete condition) and individual subjectivism (25% in the abstract and 42% in the concrete conditions). Intuitions in favor of error theory, secular realism and theistic realism were less widespread (see Tables 3 and 4).

Table 3 Breakdown of realist and anti-realist responses in abstract measures (excluding truth-aptness task) Full size table

Table 4 Breakdown of realist and anti-realist responses in concrete measures (excluding truth-aptness tasks) Full size table

Even though subjects dominantly responded as anti-realists across all of our concrete disagreement tasks, there was nonetheless a significant spread between those tasks that received the least amount of anti-realist responses and those tasks that received the most, t(116) = 5.5, p < .001.

Also, similar to much previous research (e.g., Beebe and Sackris 2016; Goodwin and Darley 2008, 2012), participants’ strength of belief (i.e., the strength with which they agreed or disagreed with moral statementss) and their perception of consensus were also significantly higher for those concrete moral issues they had classified as realist than those they had classified as anti-realist, ts(46) = 4.7 and 5.9, ps < .001.

We did not find a significant correlation between secular realism and theistic realism (r = −.067, p = .47). This suggests that these views are not only conceptually distinct but also represent distinct psychological constructs.Footnote 20

Inter- and Intrapersonal Variation

Above we defined a “consistent” participant as one who gives the same kind of responses in 100% of our abstract and at least 90% of our concrete tasks. None of our participants turned out consistently realist and 59 participants (50%) gave consistently anti-realist responses. This percentage is higher than what we would expect if people were answering randomly (binomial test, p = .016). Of the 58 participants who did not give consistent responses across both kinds of measures, 31 gave consistent anti-realist responses to the concrete measures (but not the abstract), 11 gave consistent anti-realist responses to the abstract measures (but not the concrete), and only 7 did not give consistent anti-realist responses across any of the two kinds of measures (Fig. 1).

Fig. 1 Number of participants who gave consistently realist, inconsistent and consistently anti-realist responses to the abstract and concrete measures Full size image

Breaking participants’ responses down further, there were 18 people who gave consistent responses for all four abstract cases (4 for secular realism, 3 for theistic realism, 7 for cultural relativism, and 4 for individual subjectivism). Moreover, there were 14 people who gave consistent responses (at least 90%) for all ten concrete cases (3 for cultural relativism, 6 for individual subjectivism, 1 for error theory, and 4 for non-cognitivism).

When we grouped people’s responses to the concrete cases into individual subjectivism/cultural relativism and error theory/non-cognitivism, much more consistency emerged: 35 people (30%) gave consistent individual subjectivism/cultural relativism responses and 41 people (35%) gave consistent error theory/non-cognitivism responses. That is, many participants seemed to have the intuition that either moral truth is mainly determined by their own or by culturally dominant moral beliefs, or that there is no moral truth at all (see Tables 3 and 4).

There was a strong positive correlation between participants’ abstract/concrete realist responses and their abstract/concrete anti-realist responses, rs(117) = .52, p < .001 as well as a strong negative correlation between their concrete realist/abstract anti-realist and concrete anti-realist/abstract realist responses, rs(117) = −.52, p < .001. This suggests some consistency between abstract and concrete responses. Nonetheless, we did find a significant difference across the abstract and concrete measures. In the abstract tasks the anti-realist response rate was significantly lower than in the concrete ones (77% vs. 89%, t(116) = 4.4, p < .001). However, this abstract/concrete variation may not be genuine. It may actually simply be explained by differing responses to the disagreement tasks on the one hand and the theory, comparison and metaphor tasks on the other hand (see Tables 1 and 2).

Demographics

The strength of participants’ religious beliefs was negatively correlated with their anti-realist responses in both abstract and concrete conditions, rs(117) = −.34 and .30, ps < .001, as well as positively correlated with their realist responses, rs(117) = .34 and .30, ps < .001. As expected, this is driven almost entirely by participants’ theist realist responses. When they are removed, the correlations between realism and strength of religious beliefs drop to non-significance for the abstract cases and much lower significance for the concrete cases (rs(117) = .18 realist and − .18 anti-realist cases, ps = .048 and .050). These correlations suggest that realist/anti-realist intuitions are at least somewhat dependent on religiosity — a finding that is also supported by participants’ verbal explanations (and see also Goodwin and Darley 2008: 1356–1357; Yilmaz and Bahçekapili 2015).

Political orientation was correlated with participants’ abstract, but not concrete, responses — positively with their realist r(117) = .22, p = .017, and negatively with their anti-realist responses, r(117) = −.22, p = .017. This means the more conservative the person was the more realist their abstract intuitions were. Once again, these relationships went away when theist realism was removed, suggesting that the more conservative participants were more inclined towards theist realism than our more liberal participants.