American culture values all things brash and independent, arrogant and insensitive. Chinese culture, however, upholds politeness and interdependence, humbleness and sensitivity. According to a paper published in the most recent issue of the journal Cognition, such cultural stereotypes—true or not—can influence how we understand language.

Hong Kong researchers Kevin Luk, Wen Ziao, and Him Cheung randomly assigned a group of 60 bilingual (Cantonese- and English-speaking) Hong Kong students to one of three groups. At the beginning of each experimental session, participants were shown pictures intended to “prime,” or bring to mind, cultural stereotypes. The first group saw American flags, Superman, and other iconic Americanisms. The second group saw Confucius and Chinese dragons. The third was shown more neutral pictures: landscapes.

Then began a game so odd it could only have been invented by a cognitive psychologist. Two participants sat across from one another, but one was just pretending to be a participant (the “confederate”). Between the two was a three-dimensional grid, with some slots hollow and others not—imagine an apothecary cabinet, but with the back removed, as well as all but a few of the drawers. Within this grid were a number of small objects. The confederate instructed the participant to move a specific object from one slot to another. “Move the biggest ball one slot down,” she might have instructed.

For five of 10 trials, the object that best fit the confederate’s description (e.g., the biggest ball) was in one of the occluded slots, which the participant could see but the confederate could not. Participants who were able to take into account the confederate’s perspective should thus have reached for the next biggest ball when following the confederate’s instructions.

The researchers found that, while 45 percent of participants in the American priming group reached for an object that the confederate couldn’t see, just five percent of participants from the Chinese or neutral priming groups made this error. Striking, no? This led researchers to conclude that an American “active cultural frame” made people less likely to consider the perspectives of others.

Although this study improves upon earlier work (in which one group of participants was bilingual, the other monolingual), I remain skeptical. Luk and colleagues write, “A common observation was that once the participants realized that they had moved the wrong object, they immediately corrected themselves and made no further errors in the remaining trials.” Nine of the 20 participants in the American group made a mistake, yet the group as a whole still only made nine mistakes in 100 trials. Indeed, the entire experiment produced just 11 errors.

This kind of data pattern is often described as “at floor,” or characterized by values so close to zero that it loses some of its interpretability. Interestingly, “at floor” performance more commonly prevents us from seeing an effect that would otherwise be there. If I compared the running abilities of an elite team of athletes and a department of sluggish psycholinguists, for instance, we would certainly expect a difference. But if I made the task so difficult that nobody could achieve it—if I, for instance, counted the number of times participants could run up and down Mt. Everest in a day—I wouldn’t see a significant difference between groups: the Olympic athletes would, unfairly, look just as out of shape as the psycholinguists.

The current study may have the opposite problem. Because the task was so easy, and nobody ever made more than a single mistake, the existing sample simply contained too few data points to convince me. Luk and his colleagues’ results are suggestive. But I’ll need to see this study replicated—on far more than 20 participants per group—before I’ll bet my brash, independent, arrogant, and insensitive American reputation on the result.