Is Braden Holtby elite?

It is, believe it or not, hard to tell whether a goaltender in the NHL is actually elite or just having a great but temporary run. The line between above average goaltenders and elite goaltenders is a blurry one.

Elite players are those who, when you look back on their career, were at or near the top every year for many years. They were consistently above their peers. What that looks like is open to interpretation. This is not an easy thing to discern, even with hindsight.

But with young goalies like Holtby the task is even harder. The idea is to look forward not backward, to project into the future whether someone will consistently be better than 90 percent of his peers over the next ten years.

This is what elite means: to be better than just good, better than above average. It means to be among the very best.

There are two separate questions at work in this kind of evaluation. First, has Holtby been consistently above his peers in his career and, if so, by how much? Second, can that performance be projected into the future?

The question of whether a current NHL goaltender is truly performing better than the league is complicated by the fact that over the past two decades, NHL goaltenders as a group have been performing better.

Between 2005 and 2015, the average 5-on-5 save percentage for netminders with at least 30 minutes at even strength has risen from .897 to .917. A .920 save percentage at even-strength just doesn’t mean the same thing that it used to.

Braden Holtby has over his career (2010-2015) an even-strength save percentage of .9298 (via war-on-ice.com). This and his High Danger save percentage (.8567) are generally the basis of most claims for Holtby’s elite status. There is no question that these are very good numbers, but just how good? How far above his peers has Holtby performed?

One way to answer this question is to look at something called a z-score (or a standard score). A z-score is a statistic that gives a standardized distance from average for different samples, so that it is possible to compare numbers across samples that look different.

In other words, by comparing Holtby’s z-scores to the z-scores of other goaltenders in two-year batches, it is possible to account for the changes in group performance over time as well as the fact that save percentages are more tightly clustered now than they were in the past.

Here’s where Holtby ranks in each of three commonly accepted measures of goaltending performance among goaltenders with at least 30 minutes of 5-on-5 ice time. Each two-year batch has a different number of players. With 100 goaltenders, a player needs to be in the top 10 in order to have performed better than 90 percent of his peers.

Seasons Top 10 % 5v5 Sv % Adjusted Sv% High Danger Sv % 2009-2011 (n=98) 10 or above 15th 14th 9th 2011-2013 (n=98) 10 or above 16th 25th 21st 2013-2015 (n=106) 11 or above 21st 28th 22nd

Holtby’s performance has been well above average, but not in the elite range and it has dropped slightly compared to his peers over time.

However, this sample includes players who played very few games. What happens among only players with at least 30 games in each time period?

Seasons Top 10% 5v5 Sv % Adjusted Sv% High Danger Sv % 2011-2013 (n=57) 6 or above 10th 17th 10th 2013-2015 (n=60) 6 or above 9th 12th 9th

Things get a little better for Holtby’s case when compared to goalies from the past ten seasons adjusted for era. Still looking at two-year batches, how far from the average has Holtby’s performance been compared to all goalies since 2005-06?

Again excluding goalies with less than 30 games in the designated time period, Holtby does well against his competition but not as well as his raw numbers suggest.

Top 10% 5v5 Sv% Adjusted Sv% High Danger Sv% N=302 30 or above 24th (2011-13) 91st (2013-15) 29th (2011-13) 79th (2013-15) 97th (2011-13) 41st (2013-15)

So what does this say about Braden Holtby and the question of elite goaltending? Not much, to be honest.

Holtby may not have been in the top 10 percent in most of these statistics, but the top of the save percentage list is not simply a list of elite goaltenders. Some of the names you’d expect are there (Henrik Lundqvist, Tim Thomas, Roberto Luongo), but so are many others (Jake Allen, Martin Jones, Ray Emery).

A recent highly unscientific twitter poll to determine the most frequently named “elite” goaltenders came up with twenty names. Those names account for less than half (43%) of the top 10% in these three measures. The most listed names (Lundqvist, Luongo, Price, Rask, and Rinne) account for less than one-fifth (17%).

The issue is that save percentage is highly volatile as a statistic. It moves around quite a bit for every goaltender and it takes an enormous number of shots to approach stability. While comparing current numbers to past numbers favors current goaltender unduly, using smaller periods of time captures temporary highs from a number of goalies who do not sustain this performance.

Every year there are goaltenders who put up excellent numbers and every year a new player is being “discovered” by the media. But it is not until many years have passed that we actually can tell based on these measures whether a goaltender is truly elite.

That does not mean that Braden Holtby is not an elite talent or that he will not have elite numbers over the next ten years. He is certainly well above average now. He might be elite in the future. It certainly cannot be ruled out.

This is simply a reminder that save percentage as it is currently calculated is a fairly bad way to tell who is and who is not elite. It fools us because slumps and streaks can continue long enough to reinforce our impressions.

Save percentage fools us because what was great a few years ago is now just average. The numbers keep getting better and new goalies keep ending up at the top of lists. Sometimes they stay there. Usually they don’t.

That leaves us with the kind of subjective evaluation that takes a great deal of time, effort, and attention to be good at. The collective wisdom of the goaltending community that comes from years of watching and thinking about the position, years of finding out what works and why, suggest that Braden Holtby is one to keep an eye on.

But it is perhaps too soon to declare him elite just yet.