In a famous paper psychologist John Bargh and collaborators gave students at NYU a test very similar to that described by Malcolm Gladwell in Blink:

In front of you is a sheet of paper with a list of five-word sets. I want you to make a grammatical four-word sentence as quickly as possible out of each set. It’s called a scrambled-sentence test. Ready? him was worried she always are from Florida oranges temperature ball the throw toss silently shoes give replace old the he observes occasionally people watches be will sweat lonely they sky the seamless gray is should not withdraw forgetful we us bingo sing play let sunlight makes temperature wrinkle raisins

The students were then sent to do another test in an office down the hall. Unbeknownst to them, walking the hall was the real experiment. Scattered in the sentences above are words like “worried,” “Florida,” “old,” “lonely,” “gray,” “bingo,” and “wrinkle.” Bargh reported that students who had been primed with these words took significantly longer to walk down the hall than those not primed with the “old” words.

In the original study there were only 60 participants and the subjects were timed with a stopwatch. A new paper doubles the sample size and uses more accurate infrared sensors. You will probably not be surprised to learn that the new paper fails to replicate the priming effect. As we know from Why Most Published Research Findings are False (also here), failure to replicate is common, especially when sample sizes are small. I haven’t yet described the real surprise, however.

The authors of the new paper, Doyen et al., then took the experiment meta; they ran the experiment again but this time they told half the people supposedly “running” the experiment that they expected the participants to walk slower and the other half they told that they expected the participants to walk faster. (A confederate provided evidence for this effect.) In the second experiment they again used the infrared sensors but they also asked the nominal experimenters to use a stopwatch as the sensors were said to be new and sometimes unreliable.

In the second experiment Doyen et al. were able to replicate the Bargh results. Namely, when using the stopwatch, the nominal experimenters reported that the group primed to walk slow did walk slow and they reported that the group primed to walk fast did walk fast. The results, however, were not entirely due to subtle experimenter bias because in the slow prime case the infrared sensors also found that the slow-primed group walked slow. The infrared sensors, however, did not report an increase in speed when the nominal experimenters expected an increase in speed.

Thus, the old-slow priming results appear to be due to a subtle mix of experimenter bias and standard priming which is cued or amplified via experimenter signaling. Given what are still relatively small sample sizes (50-60) the last should also be taken provisionally.