Article content continued

Afterwards, they were asked to fill out a survey including the question, “How old do you feel right now: very young, young, neither young nor old, old, or very old.” They were also asked their father’s age, which allowed the researchers to control for variation in baseline age across participants.

Using a common statistical tool known as analysis of covariance, or ANCOVA, which measures one set of numbers against another, the authors were able to show that, on average, listening to the children’s song made people feel older than listening to the control song.

A second experiment aimed to extend these results with a song about getting old, When I’m Sixty-Four, by the Beatles, with Kalimba again as the control song. But this time, instead of being asked how old they felt, they were asked for their actual birthdate, which allowed precise calculation of their age.

An ANCOVA analysis, controlling for their father’s age, showed a statistically significant but logically impossible effect: listening to When I’m Sixty-Four made people 16 months younger than listening to Kalimba.

Listening to a song obviously has no bearing on how old you actually are. This nonsensical result, they argue, was merely an artifact of flawed analysis within a scientific culture that permits all kinds of relevant details to be excluded from the final publication.

Under their proposed guidelines, though not under current accepted scientific practices, the authors would have been required to disclose that they in fact asked participants many other questions, and did not decide in advance when to stop collecting data, which can skew results. They also would have been obliged to disclose that, without controlling for father’s age, there was no significant effect, and the experiment was more or less a bust.

“Our goal as scientists is not to publish as many articles as we can, but to discover and disseminate truth,” they write. “We should embrace these [proposed rules about disclosing research methods] as if the credibility of our profession depended on them. Because it does.”