The old adage tells us not to judge a book by its cover, and now it seems as though we shouldn’t judge a violin by its price. Violins crafted in the so-called "golden age" by expert makers Antonio Stradivari and Guiseppe Guarneri "del Gesu" are worth up to several million dollars each, and they have long been considered the best violins in the world. However, nobody has studied whether or not these instruments are actually superior to other violins in their tonal qualities. New research in PNAS shows that these lofty prices might not actually reflect how musicians actually feel about the instruments themselves.

The research took place at the Eighth International Violin Competition of Indianapolis, a prestigious gathering of violinists, violin experts, and violin makers. Twenty-one subjects were included in the experiment, and all were very experienced violinists. The researchers used six violins in their tests; three were new high-quality violins, ranging from just a few days to a few years old, and three were old violins (two Stradivari and a del Gesu) crafted in the 1700’s. The three old violins were worth a combined total of $10 million, which was about one hundred times the combined value of the new ones. The musicians were unaware of the objective of the experiment, as well as the identities of the six violins used.

The sample sizes here are admittedly small, but as the paper notes, “it is difficult to persuade the owners of fragile, enormously valuable old violins to release them for extended periods into the hands of blindfolded strangers.”

While the violinists weren’t actually blindfolded, their sight was indeed compromised. Both the researchers and the violinists had to wear modified welders’ goggles that limited their vision. And for good reason: experienced violinists can recognize a prized Italian violin instantly by its appearance. Just as wine tasters’ preferences are influenced by a wine’s price, it’s likely that violinists’s perceptions would be swayed by the reputation of an old Stradivari or a Del Gesu violin. Therefore, the experiment was carefully designed to be double-blind; neither the musicians nor the experimenters knew which violin was which. The researchers went so far as to place a drop of scent on each violin’s chinrest, in case the instruments could be distinguished by their smell.

The experiment had two phases. In the first, the researchers wanted to determine whether the musicians had an immediate preference for the old violins, as the instruments’ value would predict. They presented the violinists with pairs of violins, one old and one new (which was unbeknownst to the subjects). The violinists were given one minute to play each instrument, then asked which they preferred. Each violinist tried ten pairs of violins, covering each possible old-new pairing with one pair tested a second time to see whether the preferences remained consistent.

As it turned out, the violinists were remarkably inconsistent in their choices in this test. Barely half of the musicians made the same choice twice when presented with a pair of violins a second time. The experimenters suggest that this method may not have allowed the violinists enough time to choose the better instrument. However, one result was clear: the violinists didn't prefer the old violins to the new ones. This trend was driven by a definite dislike for one particular instrument: the oldest Stradivari. This violin was consistently picked as the poorer of the pair, while the other five violins were chosen with about equal frequencies.

In the second part of the experiment, the violinists had more opportunity to evaluate the violins. They had one hour to play all six of the violins in any order, and could switch back and forth between the instruments as they wished. At the end of the hour, they were asked to choose "the instrument they would most like to take home with them," and to pick the best and worst violin in four categories: range of tone colors, projection, playability, and response. Again, the double-blind controls prevented both researchers and musicians from recognizing the instruments.

Again, the old Stradivari was the least popular of the six instruments. It was chosen just once to “take home,” and was rated by six violinists as their least favorite. It also ended up being deemed the worst violin in a category sixteen different times. In contrast, one of the newer violins was the clear winner. Eight musicians wanted to take it home, and nobody called it their least favorite. The violinists rated this one as the best violin in a category 38 times.

Overall, just eight violinists (less than 40 percent) wanted to take one of the three "golden age" violins home. When asked to guess what era their favorite violin was made in, seven guessed wrongly, just three guessed correctly, and the rest either didn’t know or declined to offer a guess.

While these results aren’t a decisive victory for the new models, it definitely counters the wisdom that these old, highly valuable violins are unmatched in quality. In many cases, the old and new instruments are equal in quality - in some, the new models are superior to their “golden age” counterparts.

There is clearly a lot of variation that wasn’t controlled for in this study. The violinists ranged in age and years of experience, and the violins were tested in only one set of acoustic conditions. Additionally, they were rated only by the players themselves, not by listeners situated where an audience would ordinarily be. And, of course, different violinists have different preferences when it comes to their instruments.

Scientists and music aficionados alike have been trying to figure out what aspects of craftsmanship make a violin sound great and play well, but there are no clear answers yet. There are hypotheses that the density of the wood, certain properties of the varnish, and the way the front and back of the instrument are tuned determine a violin’s quality. All we know right now is that being crafted in the golden age by a famous maker may not be enough to make a good violin great.

PNAS, 2012. DOI: 10.1073/pnas.1114999109 (About DOIs).