After my last two posts on the price of puerh and age (Parts 1, 2) and then James’ subsequent posts (see 1, 2, and 3), I decided to investigate the concept of “quality” in the puerh market. In our articles, both James and I note that there is no good way for us to control for a tea’s quality in our analysis. Thus, we can’t answer important questions, such as how quality moderates the relationship between age and price. (E.g., Does boutique or factory tea appreciate fastest?) It is unfortunate that we do not measure quality, as it is one of the most common topics discussed in the tea community.



The problem is that quality is a difficult concept to operationalize, and even harder to capture empirically. The term “quality” itself is problematic because it denotes an objective standard that does not exist. At best, we can measure “subjective quality.” Taste is subjective and has a tendency to change over time. A tea that you loved just last year might not fair well if revisited today. Moreover, we also know that our impressions of quality are easily swayed by external factors, such as reputation or price. Thus, despite the fact that we spend a lot of time discussing quality, we don’t actually do a good job measuring it.



I want to address two questions here:



Has the “quality” of puerh changed over the past decade? What is the relationship between price and quality?

To address these questions, I re-purposed my puerh sample with added controls for Steepster rating, a stand-in for “subjective quality.” I included a sample of puerh cakes sold by Yunnan Sourcing, as well as two samples of “private label” cakes produced by Yunnan Sourcing and White2Tea. You can read more about how the initial sample was constructed in this post.

For this article, I was less concerned with having a random sample than I was with having a representative sample. I think I accomplished this, but it is obviously never going to be perfect. By creating a sample of puerh sold by – but not produced by – Yunnan Sourcing, I have a pseudo-random baseline. I also included “private label” puerh from both Yunnan Sourcing and White2Tea, representing two distinct markets. YS is known in the online community for offering a good balance of quality and price. W2T is known for its tightly curated collection of high quality cakes. Both labels are consistently well reviewed. By focusing on these two vendors in particular, this sample offers good representation without being overly complicated.



Let’s look at the data.







Stable Prices

This first thing to note is that over the past decade, prices have remained relatively stable for both Yunnan Sourcing and White2Tea label productions. (Other producers were excluded.) As I noted in the last article, the price of YS label teas have decreased slightly, while W2T label teas have increased slightly. Neither trend, however, is statistically significant (p > 0.05) when controlling for other variables such as style (Sheng = 1) and shape (Cake = 1).



What has changed are the prices of the the highest priced teas offered each year. Among the W2T sample, the highest priced teas produced in 2014 and 2015 are both sold today at around $240 per (200g) cake, while the highest priced teas produced in 2016 and 2017 cost almost 50% more at $360 per cake. (While YS has also produced more expensive teas in recent years, these are not represented in this particular sample.) Thus, while the average price of puerh has not increased over time, there are more higher priced cakes available today than there had been in the past.





Subjective Quality

Using Steepster ratings as a proxy measure, we can also look at changes in “subjective quality” over time. Interestingly, there appears to be no change in the the average Steepster rating of YS and W2T label productions over the past eight years. Looking at the graph below, you can see that the regression lines for both YS and W2T are flat. OLS regression confirms that the relationship between “quality” and production year is not significant (p > 0.05).*

*All regressions included controls for puerh style and shape.



Now, it’s important to note that 43 of the 144 teas in the original sample were dropped from the analysis because they did not have Steepster ratings. Unsurprisingly, among these were some of the highest priced cakes W2T offers, as well as many of the new Spring 2017 productions. For example, neither W2T’s The Treachery of Storytelling series nor Last Thoughts were included in the sample. This has the effect of removing outliers and likely bringing down W2T’s average Steepster rating for 2016 and 2017.



Still, the trend also holds true for the “Other Labels” baseline, which you can see on the graph above. This indicates to me that the null relationship between Steepster rating and production year is a global trend and not specific to recent YS and W2T productions. Puerh quality has remained fairly stable over time, mimicking the trend for price. That said, I want to be careful about what claims can be made with this data. After all, Steepster ratings only offer a snapshot of contemporary preferences, and thus do not reflect how tastes change.







Price by Quality



Finally, let’s look at the relationship between Steepster rating and price for YS and W2T label puerh. Interestingly, while there is no relationship between rating and price (per 100g) for Yunnan Sourcing, there is for White2Tea.



For every additional point in Steepster rating (e.g., 80 to 81), there is an increase of about $0.10 per 200g cake. This relationship is significant (p < 0.01). Granted, this is not a big difference, but it does indicate that there may be a relationship between the quality and price of White2Tea puerh.



I think that there are two interpretations of this finding:

If we take it at face value, then the analysis tells us that White2Tea prices their teas according to their relative quality, while Yunnan Sourcing doesn’t. Higher quality teas just cost more money, cut and dry. I do not think, however, that this is a good interpretation of the data. A better interpretation is that White2Tea’s customers have a slight bias toward higher prices teas, such that they expect that a higher price will indicate a higher quality. We have all heard about wine critics who rate the same bottle differently according to its supposed price. The same is true here, where a higher price tag makes us expect higher quality.



Perhaps that’s not quite right either. The truth is that – as I already noted – subjective quality is a hard concept to measure, and thus its relationship to other variables is hard to pin down. If I were to speculate, I’d say that there are both differences in the biases of YS and W2T costumers and differences in how Scott and Paul price their tea. These two factors both influence the relationship between quality and price, leading to what we see here.

Practically, this analysis highlights that there are many teas out there that are priced low relative to their quality. Deals can be had, if you look for them. According to the analysis, the best deals in the sample include 2015 Colbert Holland 1945 ($79, 88), 2015 2 Late ($55, 87), 2015 Poundcake ($49, 85), 2014 “Ai Lao Mountain” Wild Arbor Raw ($44, 83), 2016 "Big Snow Mountain” Wild Arbor Raw ($34, 82), and 2014 Autumn Jiu Tai Po Village Raw ($44, 81). Each of these cakes are priced slightly lower than average (controlling for vendor and production year), but have higher than average Steepster ratings.

