My name is Robin Goldstein, and I’m the author of a new book called The Wine Trials (book here; website here). Lately, I’ve become curious about how Wine Spectator magazine determines its Awards of Excellence for the world’s best wine restaurants.

As part of the research for an academic paper I’m currently working on about standards for wine awards, I submitted an application for a Wine Spectator Award of Excellence. I named the restaurant “Osteria L’Intrepido” (a play on the name of a restaurant guide series that I founded, Fearless Critic). I submitted the fee ($250), a cover letter, a copy of the restaurant’s menu (a fun amalgamation of somewhat bumbling nouvelle-Italian recipes), and a wine list.

In order to make the application appear genuine, I also obtained a Milan phone/fax number, as required by the application, and established a small online presence. Aside from creating the menu and wine list, all of this took less than three hours.

Osteria L’Intrepido won the Award of Excellence, as published in print in the August 2008 issue of Wine Spectator.

Since then, the Osteria’s listing has, not surprisingly, been removed from Wine Spectator‘s website. After the story broke, one of Wine Spectator‘s main claims (aside from calling me names) was that its staff had “called the restaurant multiple times.” However, the only message that was ever left on the restaurant’s voice mailbox (before this story broke) was on May 22, 2008, after Osteria L’Intrepido had already won the Award of Excellence. The message was from the magazine’s ad sales department, asking me if I’d like to buy an advertisement for Osteria L’Intrepido to appear in the August issue along with my listing. You can listen to the voicemail here: GSM format or MP3 format.

The main wine list that I submitted was a perfectly decent selection from around Italy that met the magazine’s basic criteria (about 250 wines, including whites, reds, and sparkling wines–some of which scored well in WS). However, Osteria L’Intrepido’s high-priced “reserve wine list” was largely chosen from among some of the lowest-scoring Italian wines in Wine Spectator over the past few decades.

While it’s interesting that the reserve list would receive such seemingly little scrutiny, the central point is that the wine cellar doesn’t actually exist. And while Osteria L’Intrepido may be the first to win an Award of Excellence for an imaginary restaurant, it’s unlikely that it was the first submission that didn’t accurately reflect the contents of a restaurant’s wine cellar.

Restaurants, like all businesses, have strong incentives to embellish their images online. We turn to experts and awards bodies to help navigate the chaotic world of information and misinformation that results. If Google, Chowhound, and a couple of unanswered phone calls suffice to verify not just the existence of a restaurant but also the authenticity of its wine list, then it’s not clear what role the critic is playing.

I presented this result at the meeting of the American Association of Wine Economists in Portland, Oregon, on Friday, August 15.

Osteria L’Intrepido’s so-called “reserve list” appears in its entirety below (with scores and some excerpts from the Wine Spectator reviews of those wines added here):

I rossi italiani “riserva” della nostra cantina

AMARONE CLASSICO 1998 (Veneto) Tedeschi 80,00 €

Wine Spectator rating: 65 points. “…Not clean. Stale black licorice…”

AMARONE CLASSICO “LA FABRISERIA” 1998 (Veneto) Tedeschi 185,00 €

Wine Spectator rating: 60 points. “…Unacceptable. Sweet and cloying. Smells like bug spray…”

AMARONE CLASSICO “GIOÉ” 1993 S. Sofia 110,00 €

Wine Spectator rating: 69 points. “…Just too much paint thinner and nail varnish character…”

BARBARESCO ASIJ 1985 (Piemonte) Ceretto 135,00 €

Wine Spectator rating: 64 points. “…Earthy, swampy, gamy, harsh and tannic…”

BAROLO 1990 (Piemonte) Az. Agr. GD Vajra 140,00 €

Wine Spectator rating: 64 points. “…Earthy, musty, lacking in charm…”

BAROLO RISERVA 1982 (Piemonte) Bruno Giacosa 250,00 €

Wine Spectator rating: 72 points. “…Agressive [sic] tannins that are sharp and harsh…”

BAROLO “ZONCHERA” 1994 (Piemonte) Ceretto 120,00 €

Wine Spectator rating: 74 points. “Quite disjointed…a coarse, chewy texture and an astringent finish. Hard to tell if it will ever come around…”

BRUNELLO DI MONTALCINO RISERVA 1996 (Toscana) Gianfranco Soldera 235,00 €

Wine Spectator rating: 74 points. “…Turpentine. Medium-bodied, with hard, acidic character. Disappointing…”

BRUNELLO DI MONTALCINO “LA CASA” 1982 (Toscana) Tenuta Caparzo 200,00 €

Wine Spectator rating: 67 points. “…Smells barnyardy and tastes decayed. Not what you’d hope for…”

BRUNELLO DI MONTALCINO 1993 (Toscana) Tenuta Caparzo 180,00 €

Wine Spectator rating: 80 points. “…A bit lacking in concentration, but with pretty, round tannins and a soft finish…”

BRUNELLO DI MONTALCINO RISERVA 1995 (Toscana) Tenuta Caparzo 135,00 €

Wine Spectator rating: 81 points. “…The palate is light-bodied with a slightly diluted finish. Light for the vintage. Rather disappointing for this producer…”

CABERNET SAUVIGNON “I FOSSARETTI” 1995 (Piemonte) Poderi Bertelli 120,00 €

Wine Spectator rating: 58 points. “Something wrong here. Of four samples provided, two were dark in color, but tasted metallic and odd…”

SASSICAIA 1976 (Toscana) Tenuta San Guido 250,00 €

Wine Spectator rating: 65 points. “…Even Sassicaia could not apparently escape the wet weather of this memorably bad vintage in Tuscany. It lacks harmony, having oxidized…”

SASSICAIA 1980 (Toscana) Tenuta San Guido 280,00 €

Wine Spectator rating: 77 points. “…Light, watery and diluted vanilla and milk chocolate character…”

SASSICAIA 1995 (Toscana) Tenuta San Guido 300,00 €

Wine Spectator rating: 90 points. “…Rich in currant, blackberry, dried herbs and tanned leather…”