PHIL McCARTEN/Reuters/Corbis

He is sometimes called the first rock star. He would whip his long hair around as he played, beads of sweat flying into the audience, and women would swoon or throw their clothes on to the stage. This is not Mick Jagger or Jimmy Page, but Franz Liszt, the nineteenth-century Hungarian pianist whose theatrical recitals made the composer Robert Schumann say that “a great deal of poetry would be lost” had Liszt played behind a screen.

But who cares about the histrionics — it’s the music that matters, right? Not according to the latest study, which shows that people’s judgements about the quality of a musical performance are influenced more by what they see than by what they hear.

The findings, reported this week in the Proceedings of the National Academy of Sciences by social psychologist Chia-Jung Tsay of University College London1, may be embarrassing and even shocking to music lovers. The vast majority of participants in Tsay’s experiments — around 83% of both untrained participants and professional musicians — insisted at the outset that sound was their key criterion for assessing video and audio recordings of performances.

Yet it wasn’t. The participants were presented with recordings of the three finalists in each of ten prestigious international competitions, and were asked to guess the winner. With just sound, or sound and video, novices and experts both guessed right at about the same level as chance (33% of the time), or a little less. But with silent video alone, the success rate for both was about 46–53%. The experts did no better than the novices.

Performance art

The results might force some to rethink what musical performance is really about. “As a classical musician, I was initially somewhat disturbed” by the findings, says Tsay, who is herself an acclaimed pianist. “It was surprising to find that there is such a wide gap between what we believe matters in the evaluation of music performance and what is actually being used to judge performances.”

But philosopher of music Vincent Bergeron of the University of Ottawa in Canada isn’t worried. “One could plausibly argue that music made for performance, such as classical music, is a visual as well as a sonic art, and that it should also be evaluated on the basis of how it looks,” he says, adding: “This is a brilliant paper.”

Tsay thinks that, at least for this kind of music, visual cues carry implications about the degree of passion and motivation that the performer displays. These are qualities that many participants in the study cited as important. Musical novices might identify them even more accurately than the ‘experts’ can, Tsay says, because novices are unencumbered by the sound “that professional musicians unintentionally and non-consciously discard”.

Music neuropsychologist Daniel Levitin of McGill University in Montreal, Canada, agrees that Tsay’s results might have been anticipated. “In a sense, the visual channel is more primordial than the auditory,” he says.

However, there are limits to how much what you see can override what you hear. That might happen for competition finals, where all performers are comparably excellent, but previous studies of the role of visual information in musical assessment have shown that trained musicians have no problem distinguishing between good and significantly poorer performances.

Perhaps the key remaining question is whether the visual information is reliable — helping us to pick out the most deserving winner, for instance — or misleading, making us prefer performers who rely on visual flair rather than on musical depth.

But accepting body language as part of a musical performance could also be seen as entirely reasonable, because it adds more to the quality of the experience than just the sound. “Many performers do have the intuition that the role of visual information is an important one”, says Tsay. Mick Jagger knows that well enough, and so did Franz Liszt.