This past weekend my friends and I conducted a blind taste test with 10 different pumpkin beers.

Below are the judges:

a. 30 yr old guy (loves craft beer, strong and darker beer)

b. another 30 yr old guy (loves beer in general, wider range of preferences)

c. 30 yr old woman (all she wanted for her first anniversary was a burger and a beer)

d. 28 yr old woman (lived in Belgium for a long time, really enjoys a good Belgium beer)

e. 25 yr old woman (enjoys a wide range of drinks, slowly maturing out of the college-drinking scene)

f. the father of my friends (great guy, very Irish, Vietnam vet, prefers a nice kettle one on the rocks)

g. the mother of my friends (wonderful woman, school nurse, very Irish, her version of a drink is a white wine spritzer, light on the wine)

So you can see we had some diverse ages and drink preferences among the group of 7 taste testers. For some of them, this was their first taste test with me (and this blog), and they took it way more seriously than I thought they would(it didn’t help that they were all Giants fans and watched their team begin the season 0-3).

We purchased 10 different pumpkin beers! We bought them in New York state, so our options may not include many mid-west or west coast beers.

The beers we tested were:

Sam Adams Fat Jack (Boston)

Harpoon Imperial Pumpkin (Boston)

Elysian Dark O’ the Moon Pumpkin Stout (Seattle)

Elysian The Great Pumpkin (Seattle)

Southern Tier Pumking (New York)

Hoppin’ Frog Frog’s Hollow Double Pumpkin Ale (Ohio)

Dogfish Head Pumpkin Ale (Delaware)

Post Road Pumpkin Ale (Brooklyn)

Timmerman’s Pumpkin Lambicus (Belgium)

Shipyard Pumpkin Head (Maine)

Here are the categories we used to evaluate each beer, using a scale from 1 (dislike, not a preference) to 5 (like, preferred).

1. Pumpkin Aroma

2. Spice Aroma

3. Pumpkin Taste

4. Spice Taste

5. After Taste

6. General Taste (this category tried to generalize the tester’s overall experience with the beer)

7. Beer/Alcohol Taste (this category was meant to get the taster’s reaction to how weak/strong the alcohol came out in the drink. The normal 1-5 scale worked differently here, where 1 meant weak/no alcohol taste and 5 was strong/lots of alcohol taste. For each tester, a high score does not indicate preference, since someone may prefer to have a weaker beer)

8. Rank (at the end, the tester had to rank all 10 beers. We used this category to come up with a final winner).

Taste Test:

One of us needed to know which beers were which when assigning them a number, so the rest of the testers would not know which beer they were drinking. Let’s just say the one who was only having a little beer so she could fit in a bridesmaid dress the next weekend quickly volunteered. So unlike other taste tests I organized, I was going into this one blind, which was fun.

The pourer would fill each of our plastic cups with a beer, and we would all take it and rate it. When we were finished, we washed out the cups with water and handed it back to the pourer. When we left the room, the pourer took the next beer and poured that into our clean/dry cups. When she called us back into the kitchen, we took our cups and rated that beer. We repeated this process until each of us have tried all 10 beers.

Most of us decided to try certain beers once again. We had cups left over, so I asked to put some beers up against one another. For example, if I was between beers #1, #5 and #6, I asked her to pour me those three beers (when I was out of the room) and then taste them side by side…by side.

Here is an example of my score sheet:

Beer # #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Pumpkin Smell 4 3 3.5 4 4.5 2 3 3.5 2 4 Spice Smell 3 4 4.5 5 3 3.5 3 4 4.5 3 Pumpkin Taste 5 4 2 4.5 3 2 2.5 2 2 2.5 Spice Taste 3 3.5 3.5 4 3.5 4.5 2.5 2.5 4 4.5 After-taste 3 3 2 4 2.5 3.5 2 2.5 3.5 2 General Taste 4 3 2.5 4 3 4.5 2.5 3 3.5 2.5 Beer/Alcohol Taste 2 3.5 1 3 1.5 4 3 2 1.5 4 Ranking (1 to 10 – favorite to least) 3 1 5 4 2 6 10 7 8 9

When we were finished, I inputted each of our scores in a spreadsheet and figured out the average for each (including the final rankings).

The Results:

First, the top results from each category.

Most/Best Pumpkin Smell:

1. Southern Tier Pumking

2. Shipyard Pumpkin Head

3. Elysian The Great Pumpkin

Most/Best Spice Smell:

1. Elysian Dark O’ the Moon

2. Elysian The Great Pumpkin

3. Hoppin’ Frog Frog’s Hollow Double Pumpkin

Most/Best Pumpkin Taste:

1. Southern Tier Pumking

2. Elysian The Great Pumpkin

3. Shipyard Pumpkin Head

Most/Best Spice Taste

1. Elysian Dark O’ the Moon

2. Southern Tier Pumking

3. Elysian The Great Pumpkin

Best After Taste:

1. Southern Tier Pumking

2. Elysian The Great Pumpkin

3. Timmerman’s Pumpkin Lambicus

Best General Taste:

1. Elysian The Great Pumpkin

2. Southern Tier Pumking

3. Elysian Dark O’ The Moon

You can see a pattern so far.

Most Alcohol Taste/Strongest:

1. Harpoon Imperial Pumpkin

2. Elysian The Great Pumpkin

3. (tie) Dogfish Head Pumpkin Ale and Shipyard Pumpkin Head

Then we took everyone’s rankings and averaged them together. Here is the final list, in order (with the number of times one of the testers ranked that beer their number 1 beer).

1. Southern Tier Pumking (3 times)

2. Elysian The Great Pumpkin (1 time)

3. Elysian Dark O the Moon Pumpkin

4. Hoppin Frog Frog’s Hollow Double Pumpkin (1 time)

5. Harpoon Imperial Pumpkin (1 time)

6. Sam Adams Fat Jack

7. Shipyard Pumpkin Head (1 time)

8. Timmerman’s Pumpkin Lambicus

9. Post Road Pumpkin Ale

10. Dogfish Head Pumpkin Ale

Conclusion:

It’s very difficult to say whether or not this list is a great indicator of “the best” pumpkin beers…at least out of these 10. It all comes down to preference. I prefer a darker and stronger beer, so it makes sense that my number one beer was the Harpoon (which ranked highest with alcohol taste/strength among all of the testers).

And should we really go by the average of everyone’s rankings? What would the final rankings be if we did an average of all the average scores from each category? Then the rankings would look a little bit different from the list above (I showed the difference between the above rankings, + means it did better, – means it fared worse):

1. Southern Tier (same)

2. Elysian Great Pumpkin (same)

3. Shipyard Pumpkin Head (+ 4 spots)

4. Elysian Dark O the Moon (- 1 spot)

5. Harpoon Imperial Pumpkin (same)

6. Hoppin’ Frog (- 2 spots)

7. Timmerman’s Pumpkin Lambicus (+ 1 spot)

8. Sam Adams Fat Jack (- 2 spots)

9. Post Road Pumpkin Ale (same)

10. Dogfish Head Pumpkin Ale (same)

Should we rate all the beers based on one category over another (like General Taste)? When we did our rankings, Hoppin’ Frog came out at number 4 (which is high) yet it did not score in the top three of ANY category. Dark O’ The Moon scored highest in two categories and made it on almost every other category, yet no one rated it as their favorite beer.

Factors to consider: we did this taste test over a few hours, and the beers did get warmer as time went on. That definitely had an effect on people’s second tastings. For example, I ranked Southern Tier’s Pumking very highly on my first tasting, but during the second tasting it had a stronger buttery taste that was not pleasant at all, so that affected my final score of the beer (even though it still won).

A category that is difficult to test is the overall drinking experience. Perhaps the first few sips or initial tastes are fantastic. But that begs the question: would you enjoy that same beer if you had to drink the whole mug out at a bar? Would you want to drink two of them?? It reminds me of Malcom Gladwell’s criticism of the Pepsi Challenge – in a blind taste test, testers chose Pepsi over Coke…however:

“In his book, Blink, author Malcolm Gladwell presents evidence that suggests Pepsi’s success over Coca-Cola in the “Pepsi Challenge” is a result of the flawed nature of the “sip test” method. His research shows that tasters will generally prefer the sweeter of two beverages based on a single sip, even if they prefer a less sweet beverage over the course of an entire can.” (citation)

Of course, there are many other pumpkin beers out there. But if you live in or near the Northeast and you are looking for a good pumpkin beer, consider this list.

And, as usual, the biggest take away is this: partaking in taste tests are a lot of fun, especially when it comes to foods or drinks we really enjoy.

As the father in this taste test told one of his daughters a few days later, in his typical rhetorical fashion: “Sunday was a really great day, wasn’t it.”