When people in laboratory studies sample products in a sequence, they tend to prefer options presented first and last. To what extent do these primacy and recency effects carry over to real-world settings where numerous sources of information determine preferences? To investigate this question, we coded archival data from 136 actual whisky tastings each featuring seven whiskies. We analyzed people’s ratings of whiskies featured at different serial positions in the tastings. We found a recency effect: people gave their highest rating to whiskies in the last position, and voted the last whisky as their favorite more frequently. This recency effect persisted when we controlled for the counter explanation that whiskies with higher alcohol content tended to occupy later serial positions. The recency effect also persisted when we controlled for the age of the whiskies. Taken together, our findings suggest that the order of presentation matters in real-world settings, closely resembling what happens in laboratory settings with longer sequences of options.

Introduction

Choice is an everyday dilemma. Every day, we evaluate our options and decide what we prefer. But having clear and complete information when making each decision, along with plenty of time to weigh various options, is a luxury. Instead, we often draw on whatever information is immediately available, make a decision in an efficient manner, and move on to the next dilemma. What kinds of information do we draw on? The opinions of those around us, our accessible memories of similar decisions, or how attractive each of the options appears are but a few obvious sources of influential information—but some are more surprising. Take, for instance, the position of an option in a sequence. If the same option was positioned at the start, in the middle, or at the end of a sequence of options, this could change people’s decisions about how much they like that option. That is, where an option appears in the sequence will result in different decisions about its quality, and whether it is preferred over other options. Here, we examine order effects in a real-world whisky tasting. We aim to determine the extent to which the influence of order on preferences persists when there is little control over other aspects of the environment.

Numerous theories in social psychology and behavioral economics discuss decision-making in real-world contexts. Choices are a product of the environment surrounding the decision-maker, where many features of the situation can influence the choices made. Whether people are aware of the context-specific influences on their choices does not matter, because people can be influenced by all types of information regardless of how noticeable it is [1]. For example, as mentioned in the introductory paragraph, the order of the options is one factor that might influence people’s decision without their awareness [2]. Organizations, governments, and sales-persons can capitalize on what is known about choosing, and change people’s decisions if they know how and when aspects of the social environment will affect people [3]. For example, even slight changes in the language used to inquire about a decision can change the types of decisions people make, such as asking if people would demand, take, or ask for an object [4]. Furthermore, in the presence of others, people automatically alter their behavior and decisions to be more cooperative [5] and will try to avoid negative emotions, such as unhappiness resulting from social rejection, by agreeing with the majority [4].

People, groups, and organizations can use knowledge about environmental effects on decisions to create a situation with particular features to influence people’s choices in a particular way (choice architecture; [1]). The goal of a choice architect is to be mindful of the various influences on choice, and to adjust these factors in the real world to produce desirable choices and outcomes. Choice architecture techniques are seen frequently in everyday life [3]. The layout of a grocery store, advertisements for various insurance plans, or, indeed, which product people decide they prefer at a tasting are all situations that have been “engineered” with economic or behavioral outcomes in mind. So, research has demonstrated that people can be “nudged” to prefer certain options by various means in many circumstances, including simplifying difficult choices, requiring active rejection of a default option, or, as is the focus in the current paper, ordering the options in a particular way [1].

How do we know that the order options appear in is important for how those options are judged? In the literature, the preference for the first and last options for sequentially presented items is called a serial position effect. The evaluation of sequences leads to serial position effects for a large range of stimuli, such as when people try to remember words [6], or judge competitors [7,8]. Furthermore, this preference for the first or last options has been demonstrated with sequences of products in controlled, laboratory settings with fast foods, soft drinks, and jellybeans [9,10]. In perhaps the most surprising example, people who sampled five glasses of wine generally preferred the first or the last wine [2]. The twist? All the glasses contained the same wine poured from the same bottle. But, even still, the sequence context encouraged people to see differences between the glasses of wine. These results in particular suggest that something about the order in which options are presented leads to a preference for the first and last products, given that the presence of other information was controlled in this laboratory study. Taken together, the literature suggests that we prefer the first and last options from among a sequence—even if the “options” are exactly the same. Primacy effects (better memory, or preference for the first item in a sequence) and recency effects (more frequent responses, higher recall, or preference for the last item in a sequence) can be found in a large range of contexts [6], and can occur in controlled laboratory studies (such as the 2009 study by Mantonakis and colleagues [2]) and more naturalistic field studies [11,12].

Why might the position of an option in a sequence change decisions about that option? One of the reasons that serial position effects occur is that a sequence thwarts a person’s ability to make absolute judgments, and instead encourages relative judgments [13]. For instance, if asked to decide how much we like a particular song when it is presented to us alone, we will tend to make an absolute judgment, a process that results in highly variable assessments between judges, because we will create our own comparison standard for the song [13]. But when an obvious comparison standard is available, we tend to make relative judgments whenever we can, such as when we are provided with two or more songs. That is, in a pair, people can make an opinion about which one is better than the other. Furthermore, people can easily rank longer sequences of products relative to one another in a similar way, which also reduces the variability in assessments seen for absolute judgments between people. But relative judgments can result in different evaluations of the same item if circumstances change. For example, people evaluate the same song better when it appears on a playlist of bad songs than great songs [13]. Why? Because when an option appears in a sequence rather than alone, people will judge that option by comparing it to the other options. In other words, any decision made about that option would be determined by how it sized up compared to the other options. Thus, relative judgments can lead to biased judgments in the context of a sequence.

There are circumstances that can reduce or eliminate the effect of serial position on judgments. In fact, for short sequences of fewer than four options, primacy effects are more common—people tend to prefer only the first option. Some research suggests that the first option is immediately put in the lead because, by default, it is the best so far as no other “competitor” has been presented [14,15]. Consider a wine tasting comprised of three different Pinot Noirs. As we try the first one, it becomes not only the lead, but it is also now a standard against which to compare subsequent options to. As we work our way through the other two options, we make relative judgments, comparing each one to the first option initially, unless another option beats it out earlier in the sequence [13]. Interestingly, other research suggests that the first option does not need to be “outstanding” as a competitor. Rather, the first option needs to merely meet a minimum threshold, or “satisfice”, in order to take the lead position initially [16]. We also tend to commit to a decision we have already made, which means that subsequent wines need to be noticeably better to compete with and replace the first wine [17]. In terms of probabilities, the current lead has a 70% chance of staying the favorite, but each challenger has only a 30% chance of becoming the new favorite [2]. Unless a subsequent option is exceptional, we are less likely to abandon a satisfactory first option. Taken together, research suggests that when people are faced with just a few options, the first one remains their favorite.

When sequence lengths are longer, the pattern changes, and two patterns of preferences tend to emerge. To see why, let us now consider a longer wine tasting, this one comprised of eight different Pinot Noirs. As with the smaller tasting, a satisfactory first wine is put in the lead and many people will stick with it [2,17]. But with longer sequences come more competitors that could potentially out rank the first wine. Because probabilities add up over the sequence, the more challengers the first wine faces, the more likely a new challenger will come out on top [2]. By the end of the tasting, therefore, people tend to favor the first wine or last wine. Moreover, people favor the other wines according to the position each one appears in the sequence, such that the least favored wine appears in the middle. Put another way, with longer sequences, people’s preferences form a U-shaped function.

Of course, outside of a laboratory, various other factors might change the “first and last” pattern demonstrated in controlled settings. Field studies examining memory for television commercials [18,19] and judgment of American Idol participants [12] are but a few examples that demonstrate the presence of typical serial position effects in more naturalistic settings. Other research suggests that serial position effects are different under some circumstances that would be found in the real world. For example, wine tastings draw people with a wide range of expertise [2]. Expertise about a product alters the approach people take when evaluating it, and altering that approach to evaluation can then change which option in a sequence people prefer [20]. Take, for instance, a very knowledgeable taster. This “expert” will decide on a favorite using the relative judgment process—pairing up each of the “challengers” in the sequence with the current favorite wine for comparison—and ultimately tend to prefer the first or last of the wines [2,13]. The main reason the expert adopts a strategy whereby each new “challenger” is compared to the current favorite is that experts are more excited by the products and motivated to evaluate them, so experts are more engaged and persistent when it comes to finding a competitor wine that is exceptional enough to demote the “champion” wine holding the lead [2].

A less knowledgeable wine taster will take a different approach. Because the novice wine taster is not well versed in the subtle differences among fine wines, he or she spends less time and effort comparing the “champion” wine to the other wines. A novice is incapable of the fine-grained pairwise comparisons that experts undertake, so less knowledgeable tasters assess each option, one at a time, on its individual merits—and isolated process of evaluation, which is more similar to an absolute judgment than a relative judgment [13,20]. So, rather than using the strategy of assessing whether this wine is better than the current favorite, a novice will simply try each one and evaluate it in isolation. Such an approach tends to result in a preference for the first option in the sequence because a wine novice rarely demotes the first wine, as the first one will “satisfice” their criteria [16]. The first option will remain the “champion” for novice tasters because the novices will lack the motivation to rigorously compare the first one to subsequent options in the way that an expert would [2].

Laboratory studies show that expertise is not the only factor that might influence serial position effects in the real world. In the context of a product sampling, such as a wine tasting, the host can also influence preference judgments. In some ways, a tasting host can be described as a “choice architect” who is creating a situation in which people will make decisions [1]. The host will have goals regarding the choices people make—does the host want people prefer a particular option, or are they seeking honest evaluations of the options? That is, any external guidance a person receives might change the effects of serial position and expertise [20]. One study sought to demonstrate why a host might influence the pattern of people’s preferences. In this study, the computer guided people through a tasting of five wines in a way that matched or did not match the way that people typically evaluate wines at their level of expertise (thereby encouraging or discouraging fine-grained, pairwise comparisons [20]). When the computer “host” was guiding the participants in a way that matched their expertise level, the decision process felt familiar, easy, and required little attention and time [21]. Familiar decision-making conditions discourage analytical decisions, and therefore encourage people to rely on heuristics to decide what they prefer. That is, people will rely on the evaluation strategies that they use most often because familiar circumstances result in fewer of the characteristics that tend to suggest the decision requires more careful thought. Therefore, when using a familiar evaluation strategy, wine experts tended to prefer the first and last options, and the wine novices tended to stick with the first options.

In contrast, when people were guided through the tasting in a way that did not match their level of expertise, the decision process felt unfamiliar—difficult to process and requiring more attention [21]. When people notice circumstances that may make decisions difficult, they will put more effort into decisions [21,22]. In the context of Philp and Mantonakis’ experiment, when the wine evaluation strategy that was encouraged by the “host” was unfamiliar, people of all levels of expertise became more analytical. Pushing people to switch to an analytical decision-making style diminished preferences for the first and last options in the wine tasting [20]. Taken together, laboratory research suggests that at a wine tasting, the guidance of a host could either inspire biases to choose the first and last products, or reduce them, depending on the direction the host takes the tasters, and the expertise of the tasters. Whether or not the host (“choice architect”) intends this outcome does not matter—the way they organize the tasting will influence order effects [1]. In addition, the results from the “computer-host” study add weight to the claim that expertise changes your evaluation strategy: Experts found a pairwise, fine-grained comparison strategy familiar, but novices did not [20]. Therefore, novices and experts can both contribute to order effects when sampling serially presented options, but only when both groups are left to use a preferred evaluation style.

Considered as a whole, laboratory research shows that when it comes to preferences, serial position matters for judgments of a variety of situations and stimuli. Furthermore, serial position effects have been demonstrated in naturalistic settings, when there could be a wide range of factors influencing preference for items that are presented in a particular order. In other words, serial position effects can shine through all other potential influences in the field. However, obtaining field data that has been collected and recorded in a largely systematic way in the real world is hard to come across. As discussed earlier, an ordinary product tasting would have social, individual, and commercial influences all at play [1,5], on top of the cognitive biases predicted by psychological research. Would people in an actual tasting of products prefer the first and last options, as laboratory research predicts, and how can we quantify these influences in the field? To address this question, we obtained and analyzed data from 136 whisky tastings, including a total of 952 whiskies, hosted by a liquor shop over the last decade and administered, for the most part, using an identical procedure at each tasting.

As we have already suggested, at a real tasting a multitude of information can change people’s preferences. This information includes, but is not limited to, memory for whiskies or flavors that people have enjoyed or disliked in the past (see the availability heuristic, [22]); social influence from others at the tasting when they voice their opinions [5,23]; expertise of the taster [2], and the influence of the host [1, 20]. Though we cannot control for all of these, as would be the case in a controlled laboratory study, we can control statistically for alcohol content, whisky age, the differences between each tasting and each whisky, and the number of people in attendance. We cannot control for influences that were not measured, of course, but we expect many such influences would not introduce systematic variance but instead random variance. Therefore, we would expect these unknown influences to randomly affect whiskies at various serial positions, thus creating noise in the preference data, or averaging out over the many different whisky tastings administered over the ten-year period.

If serial position matters in a real-world consumer setting, then we would expect to see tasters show preference for the first and last whiskies in the tastings compared to the whiskies presented in the middle. But to the extent that other sources of information weigh into tasters’ evaluations more than serial position, then we would expect no particular preferences based on the serial position of a whisky. These data are most useful in an applied sense—does a salesperson need to be mindful of sampling or presentation order when marketing their product? Can the salesperson alter a product-sampling situation in their favor by simply ordering the options in a particular way? Will this situational change be influential enough to “nudge” people towards a particular choice [1]? The literature suggests that the answer is “yes”, but these field data can tell us how much serial position can influence people’s judgments of products by drawing on information collected over a large period of time, using a fairly systematic procedure. These field data will allow us to infer the influence of presentation order over and above the factors we can account for in the analyses, and will also tell us whether serial position effects shine through despite the multitude of biases and influences that were unmeasured that could be affecting evaluations of each whisky product.