Author: Marshall Schott

We recently published the 150th exBEERiment article, which is crazy to think considering all that’s involved in each one. For some perspective, this roughly amounts to:

150 individual brew days

1,500 gallons of beer

3,000 survey responses

4,000 hours of writing/editing

Looking at it this way, I can’t help but question my own sanity a little bit. When I started this series, unlike many seem to think, my intent was never to disprove anything, but rather explore and learn more about the methods used to make beer. In fact, I distinctly recall thinking my findings might be used to buttress the claims of certainty I’d been regurgitating in online forums and at homebrew club meetings.

That wasn’t really how things panned out. In fact, of the 36 xBmts we performed in the first year after integrating the triangle test, only 8 returned statistically significant results, and strikingly, they didn’t include many of the variables most expected to make a difference. This seemed to create a spectrum onto which people who read the xBmt articles tended to fall, a continuum that remains to this day. On one extreme are those who view the lack of significant findings as a sign that our methodology is in some way, or many ways, flawed. The most ardent in this camp contend our participants’ palates aren’t advanced enough to decipher the differences, the triangle test isn’t sophisticated enough of a measuring tool, or that we’re just not good enough brewers to make the impact of the variable stand out. On the other extreme lie those who seemingly view every result we publish as gospel, proof that the methods of yore are utter hogwash that no longer need to be used.

I could write a dissertation in response to either side, point out how confirmation bias is likely hugely responsible for both sentiments, and proceed to defend our approach. But that’s not why I’m here today. Rather, I’ve noticed a string that ties both sides together, albeit with a slightly different impetus, is this idea that our xBmts are demonstrating that…

Nothing matters!

Some say it because they think it’s funny, others because they’re nihilists, and still others as a way to question the results and/or our methodology. The apparent commonality between them all is the belief our xBmts don’t return “any” significant results, a complete misunderstanding that I hope to address in this overview.

The Big Picture

Of the 150 xBmts we’ve performed, 140 have relied on the triangle test to determine whether participants were capable of distinguishing a beer brewed one way from a similar beer brewed with something different to a reliable degree. The triangle test cannot provide any evidence as to whether a variable had an objectively measurable impact on a beer, but rather can only demonstrate whether a variable made such a difference that tasters were able to tell it apart to a significant degree when relying on a specific threshold, which for us is a p-value of 0.05. To put it simply, the triangle test doesn’t allows us to say with any confidence whether a variable impacts a beer, only whether tasters are capable of tasting that impact.

As humans, we have a tendency to notice and recall those things that are most surprising while sort of ignoring the things we expect to see or that don’t make us go “huh?” Take, for example, the fact people claim “nothing matters” despite 45 of the 140 xBmts (32.1%) utilizing the triangle test have returned significant results. Indeed, many of the variables believed to matter most aren’t among those on this list, which is shocking not just to readers, but to us as well. Unfortunately, this seems to have led to many sweeping fascinating findings under the rug. Let’s have a look, shall we?

What Does and Doesn’t Matter?

The title is tongue-in-cheek, a riff on the theme of this article, I’m not suggesting the variables discussed in this section actually do or don’t matter. It’s all good. In this section, I’ll go over some of the xBmt variables we’ve focused on most that have produced some of the more interesting results.

Yeast Pitch Rate

Since April 2015, we’ve performed 6 xBmts focused on yeast pitch rate, all of which we expected to return significant results, yet only the one comparing ale to lager pitch rates in a Kölsch did. A beer fermented with a single vial wasn’t noticeably different than one fermented with a yeast starter, tasters couldn’t tell apart a beer that was underpitched from one that was overpitched, and pitching a dry yeast starter produced a beer that was generally indistinguishable from one fermented with rehydrated dry yeast. Is this proof that pitch rate doesn’t matter? I don’t think so, which is evident in the fact I still rely on yeast starters when I brew. Do some people worry about pitch rates more than they have to? I guess that’s ultimately up to each individual brewer, but yeah, I think some people maybe take it perhaps a tad more seriously than they need to.

A change I have made is that I rarely rely on pitch rate calculators these days but rather stir up a starter based simply on volume, and it’s been working just fine. Modern yeast cultures are massively more viable than they used to be, which I have to believe is where the focus on pitch rates stemmed from. It’s not that pitch rate doesn’t matter, but rather the yeast we’re using today is so far ahead of where it used to be in terms of overall health. At least that’s how I see it.

One cool thing that came out of the yeast pitch rate xBmts is what we’ve come to refer to as vitality starters, which involves spinning yeast in about 500 mL of wort for 4 hours prior to pitching. Rather than building up the cell count, the purpose of this method is to ensure maximum vitality so the yeast are rearing and ready to go once pitched. With 2 xBmts returning results showing tasters couldn’t reliably distinguish beers fermented with such a starter from beers fermented with standard starters, I’ve become a huge fan of vitality starters and have heard from many people who are successfully using them regularly in their brewing.

Fermentation Temperature

Without a doubt, our fermentation temperature xBmts have led to the most head scratching, confusion, questioning, and whatever else people do when they’re shocked. It’s commonly accepted that yeast of all types produce different compounds when fermented at temperature differences as small as 2°F/1°C, and that at the very least, the temperature during fermentation ought to be controlled to within a specific range.

Crazily, out of over 8 xBmts using myriad yeast strains, only 2 have come back significant, and in both of those the temperatures were pushed to the extreme. Tasters have been otherwise unable to reliably distinguish between beers fermented at drastically different temperatures using WLP029 German Ale/Kölsch, WLP800 Pilsner Lager, Saflager W-34/70, Southern Hills Frankenyeast Blend, and Wyeast 2124 Bohemian Lager yeasts. Moreover, an xBmt focused on temperature stability also returned non-significant results, suggesting variability in temperature during fermentation may not be detrimental.

We’ve primarily focused on lager yeasts when exploring fermentation temperature, partially because we all love lager, but also because I think we expected such strains to easily produce a difference. The fact that hasn’t been the case has certainly caused me to question what I once believed to be true about lager yeasts, to the point I now often ferment personal batches of lager just like I do ale, even bypassing the quick lager method I once swore by.

Speaking of, the quick lager method is a variable I plan to circle back around to sooner than later, as the significant results from the xBmt comparing it to a standard lager fermentation was quite bemusing. Without making excuses, as I’m okay if it does produce a beer with different character than traditional lager fermentation, I was unfamiliar with the yeast used and can’t help but wonder how things will pan out using strains I’ve more experience with.

Water Chemistry

It’s so weird for me to think that just a couple years ago, I claimed in a friendly argument with another brewer that water chemistry was of minor importance, contending that people likely wouldn’t be able to taste a difference between a beer with a defined profile and one where the water was left alone. Could I have been more wrong?

When it comes to water chemistry, the 2 general components are mineral content and pH, both of which we’ve performed multiple xBmts that have yielded fascinating results. Most curious to me has been the evidence suggesting mash pH out of the typically accepted range doesn’t seem to have nearly the impact on efficiency, aroma, or flavor as expected. On the other hand, multiple xBmts have demonstrated how differences in mineral content tend to produce perceptible differences in the finished beer.

Boil Length

In brewing, the boil serves various purposes including the coagulation of proteins (hot break) that settle out into trub, isomerization of alpha acids from hops, and volatilization of DMS/SMM. The standard accepted by most brewers is to boil wort for no less than 60 minutes with many believing it necessary to extend the boil to 90 minutes or longer for wort made with a large portion of Pilsner malt, the failure to do so all but ensuring a beer chock full of creamed corn nastiness. To test this claim out, we performed 2 xBmts, one using standard Pale 2-row malt boiled for 30 or 60 minutes and the other using a Pilsner malt wort boiled for either 30 or 90 minutes. The results from both, to the surprise of many, were non-significant, meaning tasters were unable to reliably distinguish the short boiled beers from the those boiled longer.

Curious whether this was an issue with participants’ bad palates, I sent samples of the finished beers from the Pilsner malt xBmt to a lab for DMS testing and learned the 30 minute boil beer had the same level of DMS as the 90 minute boil beer… which was precisely none.

In a follow-up xBmt, we compared beers produced using either a vigorous or weak boil and, yet again, tasters couldn’t reliably tell them apart. Testing the extreme, we designed an xBmt to test how a batch of Berliner Weisse produced without a boil would compare to a batch produced with a 45 minute boil. Finally, a significant result. In fact, based on the p-value (0.0000000004), this was one of the “most significant” xBmts to date, suggesting boiling does matter, though perhaps due to advancements in malting and a greater understanding of brewing, boil length may not be set in stone.

Gelatin

Filtration in general has traditionally been viewed as a stripper of delicate aroma and flavor in the craft beer and homebrewing worlds, with some proudly proclaiming their skipping of this step as a way to prove how craft-y they are. While mechanical filtration hasn’t really seemed to catch on in homebrewing, fining with chemicals to reduce clarification time is quite popular, particularly super cheap and easy to use gelatin. The hypothesis that fining with such an aid reduces desirable characteristics in beer seems to make sense, as the haze causing particulate being pulled out by the gelatin ostensibly possesses aromatic and flavor compounds. We wondered if the sacrifice was worth the reward of clear beer and put it to the test with startling results. Participants were unable to reliably distinguish a gelatin fined Pale Ale from the same beer that was not fined, even though they looked drastically different.

I’m compelled to believe what makes wrapping one’s mind around these results so difficult is the fact gelatin and other fining agents absolutely do remove stuff from beer, it can very easily be observed, stuff presumed to contribute to the overall quality of the beer, yet people can’t seem to tell a difference.

As a lover of clear beer, I look forward to exploring the impact using an actual filter has by comparing it to unfiltered beer as well as various other fining agents. Until then, I’m sticking with gelatin, which many of us have come to refer to as “powdered time,” as it seems to do in mere hours what many wait weeks for.

Fermentation Vessel

We receive xBmt suggestions all the time with most being added to a list that we pluck from when the time comes. If ever I received a groaner of a suggestion, certain it would return a non-significant result, it would have to be the many asking us to compare different types of fermentation vessels. We spend all this time (and money) adjusting our water profile, making clean wort, boiling it, adding hops at precise moments, quickly chilling it, pitching a bunch of healthy yeast, then fermenting it in a well-controlled environment. Surely, the type of vessel its fermented in wouldn’t matter. Would it?

When the first fermentation vessel xBmt comparing beers fermented in either a plastic bucket or PET carboy came back significant, I was convinced it had to do with the low sample size of only 10 participants. False positives happen, whatever, we’ll readdress the variable later. For the second xBmt comparing a PET carboy to a glass carboy, I had 25 blind participants take the survey and the results were again significant. Assuming the noticeable differences were caused by oxygen, I compared beers fermented in either a stainless Brew Bucket or a glass carboy, and sure enough, people were unable to reliably distinguish them.

I was convinced oxygen was at play and thus confident an xBmt comparing a beer fermented in a glass carboy to one fermented in a corny keg would return non-significant results. But I was wrong, tasters could tell them apart. Could it be an issue of fermentor dimension? We have more fermentation vessel xBmts planned to test it out!

On Participant Preferences

Without fail, and regardless of whether the results are significant or not, there’s one question we receive after publishing every xBmt article:

Which one did people like the most?

I am adamantly convinced preference is wholly subjective, that what someone else perceives as best might be what another person experiences as just okay. Even in competitions and judging where a beer is compared to a standard, personal opinion is going to have an influence, there’s no way it can’t, because we’re not robots. Hence my position on this question– I fucking hate it. I hate the thought of people changing what they do because of the expressed preference of others. It’s possible I’m in the minority on this issue, which is why we continue to include such data in xBmt articles that have reached significance, sharing only the preferences of those participants who were correct on the triangle test. The findings have rarely been conclusive.

Since it’d be laborious to go over the preference results from every significant xBmt, I’ve opted to cover a few from the aforementioned results, which happen to be quite interesting.

Fermentation Temperature

While we didn’t collect preference data in the WLP002 English Ale fermentation temperature xBmt, I thought it was pretty fascinating that of the 12 participants who were correct on the triangle test, 8 believed the beer fermented warm (76°F/24°C) was the one fermented cool (66°F/19°C). I’m not sure what, if anything, can be gleaned from this.

Out of the 12 of 21 blind tasters who were able to distinguish a beer fermented with Saflager W-34/70 at 60˚F/16˚C from the same beer fermented at 82˚F/28˚C, 7 selected the warm ferment beer as their preferred, 2 chose the cool ferment sample, 2 felt there was a difference but had no preference, and 1 thought there was no difference. This doesn’t mean the warm ferment lager was necessarily better, just that of the participants who were correct, a majority liked it more than the cool fermented sample.

Fermentation Vessel

Again, we hadn’t started collecting preference data at the point of the xBmt comparing a plastic bucket to a PET carboy, though we were when 14 out of 25 participants correctly distinguished a beer fermented in a glass carboy from one fermented in a PET carboy. That result was surprising by itself, but the fact 10 of those 14 tasters reported preferring the beer fermented in glass had me floored and really made me consider what it could be that was causing this difference. In a follow-up xBmt comparing a glass carboy to a stainless keg, preference ratings seemed to return to the norm with 6 of the 16 correct tasters preferring the carboy fermented beer, 3 saying they liked the keg fermented beer more, 3 endorsing no preference despite noticing a difference, and 4 reporting they noticed no difference.

Water Chemistry

In an xBmt comparing a beer made with brewing water built to a specific profile to one brewed with filtered tap water, 7 out of 8 correct participants reported a preference for the flavor of the former. Interestingly, preference ratings for Dry Stout made with either “Pale Hoppy” or “Dark Malty” water profiles were split essentially down the middle. When a couple Belgian Pale Ales were adjusted to different sulfate to chloride ratios post-fermentation, half of the tasters who were correct on the triangle test reported preferring the high sulfate beer while only 2 liked the high chloride sample more. This made me wonder the extent to which preference is a function of expectation, for example, since IPA is (or at least used to be) expected to be crisp and dry, people might prefer one made with higher sulfate content. I questioned this hypothesis after participant preferences were equal between a MACC IPA brewed with a 150:50 sulfate to chloride and the same beer brewed with a 50:150 ratio.

The Significance of Non-Significance

Publishing a non-significant xBmt result, particularly when the variable is one presumed to be of massive importance, inevitably leads to a few folks quickly pointing out how the results don’t “prove” anything but are merely inconclusive. Sure, and as we’ve alluded to a gazillion times, one result does not a principle make, whether significant or not.

However, as much I appreciate and mostly agree with this sentiment, I feel like it sort of ignores the concept that an absence of evidence is, or starts to become over time, evidence in itself. If we look at something big like lager fermentation temperature where all but 1 xBmt, which arguably tested an extreme, has returned non-significant results, that seems at least somewhat meaningful. But rather than accepting it for what it is, some people misconstrue this as us claiming fermentation temperature doesn’t matter. That’s not at all true! At best, what we can say is that we haven’t been able to produce a large enough difference such that participants are able to reliably distinguish between lagers fermented cool and warm, or boiled for 30 and 90 minutes, or mashed high and low. We’re certainly not naive enough to believe this “proves” these things don’t matter. Maybe modernization has mitigated the necessity of old techniques, but even so, it’s not our place to tell you how to brew or what to like. Not convinced by an xBmt result? Cool. Want to test it out for yourself? Even cooler!

CONCLUSIONS

Homebrewing is evolving. Like most hobbies, many who get involved end up entrenched, which spurs thinking, experimentation, and ultimately the revelation of new information. For those who have spent years studying classic brewing texts and relying on traditional methods to craft what I trust are quality beers, this can be kind of threatening, a crack in the illusion of certainty held about something one is passionate about. Prior to starting the exBEERiment series, it never dawned on me that brewers might be so married to their approach that findings calling them into question would cause such a stir. The feedback we’ve received over the last 150 xBmts has been a real eye opener, some people really are invested in their way of doing things, and I appreciate that. At the same time, I continue to find very valuable the ongoing search to learn more about brewing beer and look forward to the next 150 xBmts!

Support Brülosophy In Style!

All designs are available in various colors and sizes on Amazon!

Follow Brülosophy on:

If you enjoy this stuff and feel compelled to support Brulosophy.com, please check out the Support Us page for details on how you can very easily do so. Thanks!

Advertisements

Share this: Facebook

Twitter

Pinterest

Tumblr

Email



Like this: Like Loading...