Author: Marshall Schott

Being deeply knowledgeable on one subject narrows one’s focus and increases confidence, but it also blurs dissenting views until they are no longer visible, thereby transforming data collection into bias confirmation and morphing self-deception into self-assurance.

~ Michael Shermer ~

It’s a bit baffling to me when a person proclaims their evaluation of something was completely untarnished by preconceived notions, prior experiences, and expectations of a particular outcome. Indeed, science recognizes the ubiquitous nature of bias and has developed myriad methods to assist in its reduction. When someone claims to be unbiased, I admit to immediately assuming two things– they don’t understand bias very well and they likely fall on the high end of the bias spectrum. I could be wrong.

I’ve always been interested in the stuff that makes us tick, all that goes on under the surface to influence the way we think and behave. Perhaps inevitably, this interest, which motivated my career choice, has intersected with my hobby. Rather than brewing for brewing’s sake, I make beer with the intent to test shit out, to the point I now view my hobby as “homebrew experimentation.” The exBEERiments have been a lot of fun and have opened my eyes to a new way of thinking about brewing, but the results tell us little about the participants… or those reading and interpreting the findings.

Last week, I asked people to complete a brief survey with questions focused primarily on self-perceived ability levels. To be clear, the results of this survey are not super scientifically valid, but rather intended to shine a light on something I find absolutely fascinating about the human condition. I trust some will develop arguments against these findings, while others may use them to confirm their own beliefs, both of which can be explained by the very phenomenon this survey was designed to evaluate– cognitive bias.

| Survey Results & Discussion |

A total of 1,188 people completed the survey, a sample consisting of craft beer drinkers and homebrewers of varying experience levels.

Respondents were asked to rate their beer tasting abilities compared to others. Given the way averages work, it’s expected that somewhere around 68% of people are actually in the average range, with less being somewhat above or below average and even fewer being much better or worse than average. But this didn’t measure actual tasting ability, rather it asked people to rate how good of a beer taster they think they are when compared to others. Did things line up?

Not so much. In fact, a staggeringly disproportionate 56% of respondents believe their tasting ability is better than average, which is a logical and statistical improbability– how can a majority of people be better than average? To add to the absurdity, a mere 6.2% of participants reported their tasting ability to be some degree worse than average. One plausible explanation for these results, and what inspired the question in the first place, is the Dunning-Kruger Effect, a cognitive bias that occurs when “relatively unskilled persons suffer illusory superiority, mistakenly assessing their ability to be much higher than it really is.” It would seem, based on this data, that beer drinkers are just as susceptible to said bias as anyone else, with most thinking they are better beer tasters than others.

I asked stats guru Justin Angevaare from OnBrewing.com to analyze the data to see if there were any relationships between perceived abilities and experience level. When it came to length of time as a craft beer drinker, responses were fairly spread out, though the highest proportion of those rating their ability as being somewhat above average reported being a craft beer drinker for 10+ years; however, when looking at homebrewing experience, those who reported having 1-4 years under their belt seem to think a tad more highly of their abilities than more experienced homebrewers.

Respondents were then asked to rate their ability to detect certain commonly discussed off-flavors in beer. The majority reported their abilities to be in the average range for all off-flavors, with fairly large portions also endorsing not sensitive at all. While it’s possible those who believe they’re very sensitive actually are, I can’t help but wonder how many have ever really tasted the off-flavors they claim to be sensitive to.

The final component of the perceived abilities portion of the survey involved respondents indicating how they think they would have done on previously completed exBEERiments if they’d participated in the triangle tests. The results were rather intriguing.

To put things into perspective, here’s a table of actual results on the same xBmts:

It’s interesting to me that respondents thought the variable I’ve most often heard referred to as “myth,” hot side aeration, as being the most difficult to distinguish, while fairly large numbers thought they would have been able to perform better than our participants on xBmts that failed to achieve significance. To me, this suggests one of two potential things– either people don’t really trust the xBmt findings for one reason or another, which is a-ok, or they think they are more capable tasters than the people we include on our panels. This, to me, is fascinating.

| CONCLUSION |

Us humans certainly are an interesting lot, eh? We’ve evolved the capacity to consider the past, make fairly rational decisions about the present, and predict future outcomes with a decent degree of accuracy. And yet still to be influenced by something deeper, an inexplicable and seemingly omnipresent force. I’m compelled to believe our vulnerability to cognitive biases and logical fallacies serves a self-preserving function, regardless of the context. Superficially, beer is just beer, but for those of us so invested, it’s much more than that. Beer and brewing are meaningful outlets into which many of us plug our ideas of self. Whether overtly acknowledged or not, we view a good palate as a status symbol of sorts, something we ultimately derive meaning from. It makes sense to me that we might hold our own abilities in unrealistically high regard while viewing others as being less able, whether it’s illusory or not, it makes us feel special, like we matter.

I suppose for me, the problem arises when this overly positive self-perspective is used, subconsciously of course, to deride others. In the time I’ve been performing the xBmts, many of which have called into question purportedly necessary methods used to make beer, I’ve come to one conclusion: our abilities are not as good as we think. To take that one step further, what I’m saying is, despite how good of a taster you believe yourself to be, chances are your performance on the xBmts would be no better (or worse) than the participants.

I expect to field some complaints that the survey didn’t specifically ask about perceived abilities relative to a specific population. That was intentional, the idea being to get a glimpse of what people think about themselves in general. Personally, I think the imaginary population to which a person chose to compare their abilities is nearly as interesting as the final results, if only there were a good way to parse out that data. Again, this survey wasn’t designed to be as scientifically valid as a grad school dissertation, but rather provide some potentially interesting insights. I also can’t resist occasionally brülosophizing.

Finally, given the somewhat presumptive tone of this article, I want to make clear that I wasn’t talking about you, I was talking about us. In no way do I think that, just because I study and write about this stuff, I’m somehow less vulnerable to engaging in such self-delusion. I know I’m not and I’ve the evidence to prove it. I still notice insecurities about my brewing, tasting abilities, and even writing skills manifesting in odd ways from time to time, it’s only human.

Thanks to everyone who took the time to complete the survey, I truly appreciate it! If you’re wondering about responses to the last question regarding preferences for different Brülosophy projects, the large majority of respondents view most of what we do very positively, though the xBmts are the most popular.

Please share your thoughts in the comments section below. And for more on the Dunning-Kruger Effect and many other forms of self-delusion, I highly recommend checking out the You Are Not So Smart blog and podcast!

New Brülosophy Merch Available Now!

Follow Brülosophy on:

If you enjoy this stuff and feel compelled to support Brulosophy.com, please check out the Support page for details on how you can very easily do so. Thanks!

Advertisements

Share this: Facebook

Twitter

Pinterest

Tumblr

Email



Like this: Like Loading...