Attention and emotion are fundamental psychological systems. It is well established that emotion intensifies attention. Three experiments reported here ( N = 235) demonstrated the reversed causal direction: Voluntary visual attention intensifies perceived emotion. In Experiment 1, participants repeatedly directed attention toward a target object during sequential search. Participants subsequently perceived their emotional reactions to target objects as more intense than their reactions to control objects. Experiments 2 and 3 used a spatial-cuing procedure to manipulate voluntary visual attention. Spatially cued attention increased perceived emotional intensity. Participants perceived spatially cued objects as more emotionally intense than noncued objects even when participants were asked to mentally rehearse the name of noncued objects. This suggests that the intensifying effect of attention is independent of more extensive mental rehearsal. Across experiments, attended objects were perceived as more visually distinctive, which statistically mediated the effects of attention on emotional intensity.

The relative intensity of two sensations may be changed when one of them is attended to and the other is not. William James (1890/1952 , p. 275)

Attention and emotion are fundamental psychological systems. What people attend to and how they feel about attended objects shapes everyday experience. How do these systems influence each other? It is well established that emotion intensifies attention (Markovic, Anderson, & Todd, 2014; Mather & Sutherland, 2011; Pessoa, Pereira, & Oliveira, 2010). Emotional objects capture, narrow, and hold attention (Öhman, Flykt, & Esteves, 2001; Pourtois, Schettino, & Vuilleumier, 2013). Here, we examined the reverse causal path: whether attention intensifies perceived emotion.

One possibility is that attention reduces perceived emotional intensity, just as repeated exposure causes emotional desensitization (Campbell, O’Brien, Van Boven, Schwarz, & Ubel, 2014). But actively directing attention toward an object is different from the passive exposure that induces desensitization. We hypothesized that attention not only forestalls desensitization but also intensifies perceived emotional reactions to attended objects.

This possibility speaks to a long-standing question in psychological science: Does attention alter subjective experience? Fechner (1860/1966) was skeptical. James (1890/1952) was not. He believed that attention intensifies sensation, as the opening epigraph illustrates. Recent evidence supports James’s view that attention intensifies perceptual experience (Fuller & Carrasco, 2006). Voluntary visual attention increases the distinctiveness of perceptual experience, including objects’ vividness (Fuller & Carrasco, 2006) and their contrast relative to other objects (Liu, Abrams, & Carrasco, 2009). If attention increases perceived distinctiveness, it might also increase perceived emotional intensity because distinctiveness intensifies emotion (Valdez & Mehrabian, 1994).

Attention is the process of selectively processing some objects more than others (Anderson, 2005; Chun, Golomb, & Turk-Browne, 2011). Attention can be oriented to objects in the external environment or to objects in the internal environment, such as thoughts, memories, and sensations (Chun et al., 2011). Traditional models dichotomized voluntary and involuntary attention (Jonides, 1981; Posner, 1980). Voluntary attention is top down and endogenous, such as when people search for a specific target. Involuntary attention is bottom up and exogenous, as when visually salient objects “pop out” to grab attention (Nothdurft, 1993). More recent models have moved beyond this dichotomization, characterizing attention as interactively shaped by the perceivers’ current goals, the perceivers’ attentional history, and the objects’ salience (Awh, Belopolsky, & Theeuwes, 2012). Our investigation focused on voluntary visual attention because it can be precisely manipulated with visual displays (Liu et al., 2009).

Indirect evidence that visual attention increases perceived emotional intensity comes from demonstrations of the inverse: Inhibiting attention reduces perceived emotional intensity. For example, repeatedly attending to neutral faces in the presence of threatening faces reduces self-reported social anxiety (Amir et al., 2009), and distraction during the presence of painful stimuli reduces self-reported pain (Bantick et al., 2002). If inhibiting attention diminishes perceived emotion, increasing attention might increase perceived emotional intensity.

We first examined whether attention increases perceived emotional intensity in the context of sequential visual search (Experiment 1). Searching for targets mimics the everyday experience of looking for objects such as faces in a crowd, items on a shopping list, or threats in an environment. We hypothesized that participants would perceive their emotional reactions to target images as (a) more intense after they had searched for targets than before they searched for them and (b) more intense than their reactions to images that they did not search for.

We also manipulated voluntary visual attention without search and without explicitly identifying an attentional target. Specifically, we examined whether spatially cued attention would intensify perceived emotional reactions to cued images compared with noncued images (Experiment 2). Finally, we isolated visual attention from more extensive mental processing by orthogonally manipulating spatially cued attention and mental rehearsal (Experiment 3). We hypothesized that spatially cued attention would increase perceived emotional intensity independently of mental rehearsal.

Across experiments, we selected 30 images as the maximum that could be feasibly presented to participants within an experimental session. We analyzed data using mixed-effects models with both participants and stimuli as random effects with maximal random-effects structure (Barr, Levy, Scheepers, & Tily, 2013; Judd, Westfall, & Kenny, 2012). We did not conduct formal power analyses prior to collecting data. Such analyses are complex with multiple random effects because power varies with the number of stimuli, the number of participants, and the variance associated with each random effect (Westfall, Kenny, & Judd, 2014). With 30 stimuli, a medium effect size (d) of 0.50, and reasonable assumptions about variance-partitioning components, power of .80 could be achieved with 30 participants (Westfall et al., 2014). For laboratory experiments, we therefore set 30 as our minimum sample size. We scheduled data collection until the end of an academic term, subject to availability of participants, laboratory hours, and research assistants. For online experiments, we recruited larger samples of 100 because we expected smaller effect sizes owing to increased noise and participant heterogeneity. With 30 stimuli and a sample size of 100, our studies would have a power of at least .80 to detect an effect (d) of approximately 0.40. We collected all data in each experiment before conducting any analyses.

Experiment 1 Participants searched 10 times for a target image from a set of 10 sequentially presented images. Participants reported intensity of emotional reactions to images before and after the presentation procedure. We included sets of neutral, positive, and negative images to examine whether the effect on perceived emotional intensity might be moderated by image emotionality and valence. Method Participants We set a target sample size of 100 for this online experiment, as explained above. We requested 105 participants to allow for incomplete or unusable data. Adults in the United States participated via Clickworker in exchange for $2.00 (N = 100 after we excluded 3 participants who did not complete relevant measures and 2 duplicate participants; 60 female; mean age = 35.12 years). Procedure Participants were told that the purpose of the experiment was to select images for a future study. The experiment would consist of three segments, each containing 10 different images. We included three sets of 10 International Affective Picture System (IAPS) images (Lang, Bradley, & Cuthbert, 1999): neutral, positive, and negative. Set presentation was randomly ordered. For each set, the initial target-identification screen displayed the 10 images, with the target highlighted in yellow. During presentation, the 10 images were displayed one at a time. Participants were to press the “J” key each time the target appeared and the “F” key whenever any other image appeared; both actions required similar motor responses. Each image (5 in. × 5 in.) appeared on the computer screen for 1 s and was separated from the following image by a 1-s fixation cross (1 in. × 1 in.). The set of 10 images appeared 10 times each, and images were randomly ordered (see Fig. 1). Download Open in new tab Download in PowerPoint After image presentation, participants provided two ratings (0 = not at all, 9 = the most possible) of each image’s perceived emotional intensity: “How intense was your emotional reaction to each image?” and “How emotionally intense is each image?” We averaged these two items for analysis (r = .72). Participants also rated how much they liked each image (0 = not at all, 9 = the most possible). Self-reported measures of perceived emotional response such as these are widely used and reflect the core affective dimensions of intensity and valence (Russell, 1980). Participants also rated how distinctive each image was (0 = not at all, 9 = extremely). Half of the participants were randomly assigned to provide baseline ratings prior to learning which image was the target (before the study period shown in Fig. 1a). We did not ask all participants to provide baseline ratings because we were concerned that participants might anchor on initial ratings (Greenwald, 1976). After participants rated all images in a set, they repeated the procedure for the remaining sets. At the end of the experiment, participants answered questions to examine experimenter demand and hypothesis awareness. Participants completed the phrase, “I think the experimenter who designed this study wanted me . . .” by choosing from five options including the following end points and midpoint: “to rate the target images as much less emotionally intense than the other images” (−2), “to not let my emotional intensity ratings be influenced by whether an image was a target or not” (0), or “to rate the target images as much more emotionally intense than the other images” (2); M = 0.29, SD = 0.97. To examine hypothesis awareness, we asked participants to select “the purpose of this study” from eight multiple-choice answers including the actual purpose (16% correct). The pattern of significance reported below remained when we excluded participants who correctly guessed the hypothesis or who provided positive ratings on the demand item (31%; see the Supplemental Material available online). Results Data, analysis scripts, and materials for Experiment 1 are available on the Open Science Framework (http://osf.io/r483u). We estimated ratings from the following contrast-coded predictors and their possible interactions (contrast weights in parentheses): target (target image = ½, control image = −½), neutral (neutral image = ⅔, positive image = −⅓, negative image = −⅓), and valence (positive image = ½, negative image = −½, neutral image = 0). Emotional intensity As predicted, target images were rated as more emotionally intense than control images (target: M = 3.58, SD = 2.72; control: M = 2.87, SD = 2.61), t(39.73) = 4.41, b = 0.70, 95% confidence interval (CI) = [0.39, 1.00], p < .001 (see Fig. 2). There was also a Target × Neutral interaction, t(18.15) = 2.59, b = 0.66, 95% CI = [0.16, 1.16], p = .018. The effect was larger for neutral images, t(26.87) = 4.90, b = 1.14, 95% CI = [0.68, 1.59], p < .001, than for positive images, t(92.54) = 2.66, b = 0.60, 95% CI = [0.16, 1.04], p = .009, and for negative images, t(26.82) = 1.58, b = 0.37, 95% CI = [−0.09, 0.82], p = .125. The Target × Valence interaction was not significant, t(18.13) = 0.73, b = 0.21, 95% CI = [−0.36, 0.79], p = .476. Download Open in new tab Download in PowerPoint At baseline, negative and positive images were higher in emotional intensity than were neutral images (positive: M = 3.69, SD = 2.46; negative: M = 3.50, SD = 2.62; neutral: M = 1.55, SD = 1.92). To test whether target images increased in perceived emotional intensity, we estimated ratings only among participants who provided both ratings. Target images were rated as more emotionally intense than they were at baseline (target: M = 3.71, SD = 2.68; baseline: M = 2.96, SD = 2.65), t(52.30) = 3.94, b = 0.79, 95% CI = [0.40, 1.19], p < .001. Ratings of control images were not significantly different from baseline (control: M = 2.85, SD = 2.62), t(57.15) = −0.77, b = −0.07, 95% CI = [−0.24, 0.11], p = .445. Searching for target images thus intensified perceived emotional reactions relative to baseline. Liking Participants did not like target images significantly more than control images. Using the same model as above, we found no significant effects of target (target: M = 3.06, SD = 2.77; control: M = 2.92, SD = 2.78), t(103.15) = 1.26, b = 0.15, 95% CI = [−0.08, 0.39], p = .209; the Target × Neutral interaction, t(273.16) = 0.21, b = 0.05, 95% CI = [−0.40, 0.50], p = .836; or the Target × Valence interaction, t(95.72) = −1.02, b = −0.31, 95% CI = [−0.89, 0.28], p = .310. The manipulation of attention did not significantly influence liking. Distinctiveness Participants perceived target images as more distinctive than control images. Using the same model as above, we found that target images were rated as more distinctive than control images (target: M = 4.34, SD = 2.94; control: M = 3.37, SD = 2.77), t(44.59) = 4.70, b = 0.97, 95% CI = [0.57, 1.38], p < .001. There was also a Target × Neutral interaction, t(17.36) = 3.22, b = 1.06, 95% CI = [0.42, 1.71], p = .005. The difference between target and control images was larger for the neutral images, t(25.88) = 5.48, b = 1.68, 95% CI = [1.08, 2.29], p < .001, than for the positive images, t(26.00) = 1.84, b = 0.56, 95% CI = [−0.04, 1.16], p = .077, and negative images, t(26.71) = 2.35, b = 0.68, 95% CI = [0.11, 1.24], p = .026. The Target × Valence interaction was not significant, t(17.16) = −0.31, b = −0.12, 95% CI = [−0.84, 0.61], p = .760. Finally, the effect of target on perceived distinctiveness statistically mediated the effect of target on perceived emotional intensity, as indicated by the indirect effect (ab), b = 0.54, 95% CI = [0.29, 0.81], in a bootstrap mediation with 5,000 resamples (Preacher & Hayes, 2008). Discussion and follow-up experiments People perceived target images as more emotionally intense than control images and more intense than the same images at baseline. Two experiments reported in the Supplemental Material provided additional evidence that searching for target images increases their distinctiveness and that distinctiveness increases perceived emotional intensity. First, following the same procedure as in Experiment 1, we added an image-matching measure of recalled distinctiveness (see Experiment S2c and Appendix S1 in the Supplemental Material; see also Risen & Critcher, 2011). Participants (N = 81) viewed an array of 19 versions of each image, in which the color saturation and contrast of the original image were altered by 10% increments (analysis codes in parentheses) from 10% (−9) to 100% (0) to 190% (+9). Participants were asked which image “was in the original slideshow.” Beyond replicating the effect of the target manipulation on emotional intensity (see the Supplemental Material), participants matched target images to more distinctive versions than they did for control images (target: M = 0.52, SD = 2.32; control: M = 0.04, SD = 2.41), t(70.48) = 3.04, b = 0.46, 95% CI = [0.16, 0.75], p = .003. In another study, we experimentally manipulated the saturation and color contrast of one image relative to other images (see Experiment S1 in the Supplemental Material). We presented images using the same procedure as in Experiment 1 but without a target. Participants (N = 56) perceived emotional reactions to distinctive images as more intense than control images (distinctive: M = 3.49, SD = 2.87; pallid: M = 3.12, SD = 2.82), t(23.92) = 2.45, b = 0.38, 95% CI = [0.08, 0.68], p = .022. Together, these results indicate that searching for targets increases perceived image distinctiveness and that more distinctive images are perceived as more emotionally intense.

Experiment 3 We next differentiated attention from increased mental processing. We orthogonally manipulated spatial cuing of attention and mental processing. Participants mentally rehearsed either the cued image, the noncued image, or no image. We hypothesized that cued visual attention would intensify emotional reactions even when participants mentally rehearsed the visually noncued image. Method Participants We set a target sample size of 100 for this online experiment, as explained earlier. We requested 120 participants on Amazon Mechanical Turk to allow for incomplete or unusable data. Adults in the United States participated in exchange for $2.50 (N = 105 after we excluded 15 participants who failed to complete dependent measures; 54 female; mean age = 36.28 years). The experiment began with a cover story about task switching and multitasking. The experiment consisted of two practice trials and 15 test trials, with two images of the same valence in each trial. Procedure In each trial, participants were visually cued toward one randomly assigned image in each pair, as in Experiment 2. The cued image appeared just below the RSVP stream, whereas the noncued image appeared on the other side of the screen but still within participants’ visual field. Meanwhile, we manipulated mental rehearsal of each image by asking participants to “say the following word in your mind repeatedly as you view the images on the next screen” (Mrkva & Van Boven, 2017). We provided the name of the image object (IAPS image label) that they were to mentally rehearse (e.g., “baby”). Each participant was randomly assigned to rehearse the cued image, the noncued image, or neither image. Participants made three forced-choice judgments about each image. Half of the participants were assigned to select which image was more emotionally intense, which was more distinctive, and which they focused their eyes on more. The other half selected which image was less emotionally intense, which was less distinctive, and which they focused their eyes on less. Choice direction (more vs. less) was manipulated between subjects and was the same for all trials. After the final trial, participants answered a multiple-choice question with eight response options; this question was designed to assess whether they were aware of the experiment’s purpose. All participants were included in the primary analyses. The pattern of significance remained when we excluded the 4% of participants who correctly guessed the hypothesis. Participants completed the same recognition check as in Experiment 2, which was high and nearly identical for cued and noncued images (cued: M = .92, noncued: M = .91). Participants also completed the same manipulation check as in Experiment 2, which confirmed that participants reported looking at cued images more than noncued images (cued: M = .59, SD = .48; noncued: M = .41, SD = .50), z = 4.84, exp(b) = 2.28, 95% CI = [1.63, 3.18], p < .001. Stimuli We used the same 30 images from Experiment 1 and the same foil images as in Experiment 2. Results Data, analysis scripts, and materials for Experiment 3 are available on the Open Science Framework (http://osf.io/r483u). Emotional intensity We estimated emotional-intensity selection in a binary generalized mixed-effects model with participant and image as random factors. The model included four fixed effects and their possible interactions: visual cue, neutral, and valence as in Experiment 2, and mental rehearsal (contrast codes: rehearsed image = ½, nonrehearsed image = −½). Participants were more likely to select visually cued images as more emotionally intense than noncued images (cued: M = .56, noncued: M = .44), cue: z = 3.65, exp(b) = 1.61, 95% CI = [1.25, 2.09], p < .001 (see Fig. 5). Visual cue did not significantly interact with either the neutral contrast, z = 1.34, exp(b) = 1.27, 95% CI = [0.90, 1.81], p = .179, or the valence contrast, z = 0.74, exp(b) = 1.17, 95% CI = [0.78, 1.74], p = .462. These results replicated those of Experiment 2. Download Open in new tab Download in PowerPoint Importantly, the effect of mental rehearsal was not significant, z = 0.73, exp(b) = 1.07, 95% CI = [0.90, 1.27], p = .464, nor was the Mental Rehearsal × Visual Cue interaction, z = −0.02, exp(b) = 1.00, 95% CI = [0.70, 1.41], p = .984. The Visual Cue × Choice Direction interaction was not significant, z = 0.86, exp(b) = 1.24, 95% CI = [0.76, 2.05], p = .391. There was, however, a Mental Rehearsal × Choice Direction interaction, z = 2.58, exp(b) = 1.59, 95% CI = [1.12, 2.26], p = .010. Mentally rehearsed images were more likely to be selected both when participants were asked to select the less emotionally intense image (rehearsed: M = .52, nonrehearsed: M = .48) and when they were asked to select the more emotionally intense image (rehearsed: M = .53, nonrehearsed: M = .47). Mental rehearsal thus increased response bias, leading participants to select images as both more and less emotionally intense, but did not increase emotional intensity. Distinctiveness The same model as above indicated that cued images were selected as more distinctive than noncued images (cued: M = .57, noncued: M = .43), z = 4.05, exp(b) = 2.00, 95% CI = [1.43, 2.80], p < .001. The effect of visual cue on distinctiveness statistically mediated the effect on emotional intensity, as indicated by the indirect effect (ab), exp(b) = 7.91, 95% CI = [4.58, 14.07], in a bootstrap mediation with 5,000 resamples (Preacher & Hayes, 2008). Discussion These results replicated the finding that voluntary spatial attention intensifies perceived emotional intensity. The effect of cued attention was not moderated by mental rehearsal. This suggests that the intensifying of attention is not fully attributable to additional mental processing.

Internal Meta-Analysis Because we conducted four additional experiments using attention manipulations that are not reported in the main text (see Experiments S1 and S2a–S2d in the Supplemental Material), we conducted an internal meta-analysis (Goh, Hall, & Rosenthal, 2016). Method We included all of our experiments that examined the effects of voluntary attention on emotional intensity using either of the two paradigms, sequential search or spatial cuing. No studies from our lab on this topic were excluded; we did not seek out studies from other labs (Goh et al., 2016). In addition to Experiments 1 to 3, there were four additional experiments; all four used sequential-search procedures, summarized in the Supplemental Material. Their procedures were the same as in Experiment 1, without the baseline ratings and with the following exceptions. Experiment S2a University undergraduate students (N = 56; 34 female) participated in Experiment S2a. During image presentation, participants were to note on a sheet of paper the slide number on which the target appeared. Only one item of emotional intensity was included. There was no measure of distinctiveness. Negative stimuli were the same as in Experiment 2. Experiment S2b In a replication of Experiment S2a, participants returned for a second session 2 days later, during which they reported intensity of emotional reactions a second time (N = 45; for the initial session, N = 24; 16 female). We included only the initial ratings in the meta-analysis. Experiment S2c Participants in Experiment S2c (N = 81; 33 female) were recruited from Amazon Mechanical Turk and paid $2.00. The experiment included an image-matching measure of distinctiveness, as described in the Discussion section of Experiment 1. Experiment S2d In Experiment S2d, university undergraduates (N = 70; 43 female) participated online in exchange for course credit. We interrupted the image-presentation procedure to measure emotional intensity, liking, and distinctiveness of all 10 images up to four times: after initially presenting all 10 images simultaneously with the target image highlighted, after 3 sequential presentations, after 6 sequential presentations, and after 10 sequential presentations. All participants provided ratings after all 10 sequential presentations. We used only neutral images. Results Because Experiments 2 and 3 used binary measures, we transformed the odds ratios from those experiments to standardized mean differences (Borenstein, Hedges, Higgins, & Rothstein, 2011). We then conducted random-effects and fixed-effects meta-analyses. Manipulated voluntary attention increased perceived emotional intensity, d = 0.33, 95% CI = [0.26, 0.41], p < .001. An analogous analysis revealed no significant effect of attention on liking (see Table S2 in the Supplemental Material). Attention increased emotional intensity of neutral, positive, and negative images (see Table 1). There were significantly positive Attention × Neutral Image and Attention × Valence interactions. The effect of attention was larger for neutral images (d = 0.46) than for positive images (d = 0.27) and negative images (d = 0.14). The Attention × Valence interaction was not hypothesized, however, and should be interpreted with caution. Table 1. Aggregate Analysis and Summary of Effects on Perceived Emotional Intensity Across Seven Experiments (N = 469) From the Main Text and Supplemental Material View larger version

General Discussion In the present experiments, voluntary attention—whether while searching for a target or via spatial cuing—increased perceived emotional intensity. Attended images were also perceived as more visually distinctive (Liu et al., 2009), and this increased distinctiveness mediated the increased perception of emotional intensity. That attention increases perceived emotional intensity is distinct from effects of exposure, which was equated across images. Furthermore, voluntary attention significantly increased emotional intensity but did not significantly increase liking, whereas mere exposure increased liking but not emotional intensity (Zajonc, 1968). The effects of our attention manipulations on perceived emotional intensity were larger for neutral images than for positive and negative images. We suspect that the smaller effects among negative and positive images reflect that these images are more inherently distinctive, evocative, and attention grabbing (Öhman et al., 2001). It may be more difficult to manipulate the attention and distinctiveness of images that are already attention grabbing and distinctive. It is also possible that in our experiments with ratings (but not in experiments with forced choice), the attention manipulations had smaller effects on positive and negative images partly because they had less room to move on the rating scale. Further exploration of these possibilities is left for future research. Another important question for future research is whether the intensifying effects of attention are reduced in contexts with highly heterogeneous stimuli. Contrasts between stimuli, such as when negative and neutral images are presented side by side, may override any effects of cued attention. However, we do not have any reason to believe that our results depend on other characteristics of the participants, materials, or context. We also believe that the intensifying effects would emerge for both voluntary and involuntary attention. Both forms of attention increase distinctiveness of perceptual experience (Fuller & Carrasco, 2006; Liu et al., 2009). In demonstrating that voluntary attention increases perceived emotional intensity, these findings bolster critical assumptions underlying various psychological phenomena (Weber & Johnson, 2009). For example, people are thought to overestimate the intensity and duration of future emotional events because they attend to the focal event and not to the myriad other events that will occupy future attention (Schkade & Kahneman, 1998; Wilson, Wheatley, Meyers, Gilbert, & Axsom, 2000). Our findings suggest that simply directing voluntary attention to focal objects makes them seem more emotionally intense. The attention-drift-diffusion model suggests that attention influences choice by increasing the amount of information that people acquire about attended options (Krajbich & Rangel, 2011). Our findings suggest that decision makers may weight attended alternatives more heavily not only because they have more information about them but also because they experience stronger emotional reactions to them. In conclusion, psychological science has established that emotion intensifies attention. The present findings provide evidence for the reverse: Attention can alter the experience of perception and, in so doing, intensify the perception of emotion.

Acknowledgements We thank Joshua Correll, Steffanie Guillermo, Sean Hudson, and Jennifer Cole for helpful comments.

Action Editor

D. Stephen Lindsay served as action editor for this article. Author Contributions

All the authors contributed equally to the study design. K. Mrkva and J. Westfall conducted the experiments and analyzed the data. K. Mrkva and L. Van Boven wrote the first draft. J. Westfall provided comments. All the authors approved the final manuscript for submission. ORCID iD

Leaf Van Boven https://orcid.org/0000-0003-4187-8779 Declaration of Conflicting Interests

The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article. Funding

This research was supported by National Science Foundation Grants 0552120, 1049125, and 1730611. Supplemental Material

Additional supporting information can be found at http://journals.sagepub.com/doi/suppl/10.1177/0956797619844231 Open Practices

All data and materials have been made publicly available via the Open Science Framework and can be accessed at osf.io/r483u. The design and analysis plans were not preregistered. The complete Open Practices Disclosure for this article can be found at http://journals.sagepub.com/doi/suppl/10.1177/0956797619844231. This article has received the badges for Open Data and Open Materials. More information about the Open Practices badges can be found at http://www.psychologicalscience.org/publications/badges.