5 Shares Share Tweet Share Share Email

The position of goaltender in the NHL is by far the most unique position in all of pro sports. The skill set required to keep the puck out of the net is so much different than the skill set required to put the puck into the net, and because of this, goalies have completely different mindsets regarding tactics, style of play, and, of course, analysis. Advanced statistics have long had trouble pinning down the proper way to evaluate goaltenders, and though there have been significant strides made, analysis of goalies is still way behind analysis of players and teams.

With this in mind, Today’s Slapshot reached out to four prominent goaltender analysts, and picked their brains for unique insights into the position, as well as unique insights into how they see the game. Here are the four people we reached out to.

Greg Balloch – In Goal Media

Catherine Silverman – Today’s Slapshot and Leaf’s Nation

Paul Campbell – Today’s Slapshot, In Goal Media

Nick Mercadante – Blueshirt Banter

TSS: Where do today’s advanced statistics for goaltenders capture talent, and where do they miss the mark? Where is there the biggest room for improvement? What major development do you think will come next?

Greg Balloch: I think adjusted save percentage and high-danger save percentage do an adequate job of figuring out which goaltenders make more difficult saves. I say adequate because it at least accounts for shot location, which is a small part of the picture of shot quality, but it’s still a part. They aren’t perfect statistics, but they at least tell us a bit more than regular unadjusted save percentage does.

The biggest room for improvement to me is obviously in adding more information into finding out about shot quality. Changes in release point, deflections, screens, and passes before the shot are all extremely important things that need to be taken into account when figuring out how difficult a shot was to make a save on. They aren’t included in any stat right now because we simply don’t have the data collected to do that. That sort of leads into the answer to the next question: The next big step is collecting that information on every shot, and hopefully computers will be able to do that for us when the NHL begins to track players. For coaches at the minor hockey level, a mix of video analysis and shot-tracking apps like Double Blue will be key moving forward.

Catherine Silverman: Save percentage sucks; it just does, and goals against average is even a fairly flawed stat. I’m starting to like zone save percentages (ie: high danger vs. low danger saves), so I’d say that does a decent job of capturing talent; it’ll be interesting to see how it continues to develop as a metre for assessing goaltending as those collecting stats start to become more consistent in what they consider shots from each zone. Right now, there’s not a ton of consistency in where we consider a high danger shot to come from (or what we consider a high danger shot) versus medium or low danger, and that’s keeping the stat from being as effective as it could be; in time, I think it’ll be more useful.

Paul Campbell: I hesitate to call any current statistic for goaltenders “advanced.” Statistics for teams and skaters have reached a mathematically complex level of refinement that goalie stats only aspire to.

A second initial point is that performance and talent are importantly different. Performance can be evaluated retrospectively, and such evaluations (when accurate) are descriptive of a goalie’s history in a given environment. “Talent” should be a measure of a goaltender’s ability and potential at any given time as it develops through his career. A goalie’s performance is affected by a host of external factors, like the calibre of the team, the quality of the defense, the structure of the team’s defence, the amount of rest between games, and so on. A goalie’s talent is what remains when you eliminate all these factors; it’s his ability to influence his situation, whatever it may be, and it’s far less variable than performance. So, the statistics that eliminate or reduce the effects of external factors as much as possible are the ones most likely to capture talent.

One positive step for goaltender statistics is the increasing use of even-strength save percentage. Even teams of similar calibre often have very different penalty-kill strategies and success rates. Since the goaltender has little or no influence on these factors, eliminating them from consideration creates a more even basis for comparison across the league. This is not to say that a goalie’s penalty-killing success isn’t important: it’s just even more dependent than even strength play on factors the goaltender does not control.

War on Ice’s introduction of danger zones is a solid innovation I’ve used a lot. By breaking down a goaltender’s shots against into low, medium, and high danger zones (based on the likelihood of goals being scored from each area), we begin to see how much harder, or easier, one goaltender’s shots are than another’s. Adjusted save percentage is a single-number measure that takes these zones into account. High-danger save percentage, taken by itself, has been shown to account for most of the year-to-year consistency of raw save percentage, making it the most indicative of a goaltender’s contribution (and indirectly, talent).

Danger zones are just the most easily trackable beginning, however. They miss most of the action taking place before the shot, which matters as much or more than the location from which the shot is taken. The pre-shot situation is a vital aspect of the goalie equation that has not yet been integrated into statistical goalie analysis. Such tracking has been done, but on a limited and proprietary basis. The extension and widespread availability of such data will be the next big thing in goalie analysis. I expected the related project of using tracking data to evaluate goalie-to-defensive system fit will come after.

Nick Mercadante: Ithink it is important to point out that modern stats (I’m not a fan of “advanced,” as most of this stuff isn’t advanced, it is just different and new) are seriously lacking for goaltenders. It is part and parcel to the lack of Real Time Scoring System (RTSS) tracking of what goalies do during a game. Shots. Saves. Goals. Not much more there. But with that being said, we have brought in the element of shot location, and in turn adjusting save percentage, which at least gives us a better sense of overall workload. Couple that with controlling for situation and you at least can account for some of the randomness and environment impact on overall numbers like save percentage.

I think the biggest room for improvement is in adoption of better comparative analysis tools, and the experts trickling such comparative analysis tools down to the non-experts and fans. I want water cooler talk about 5v5 adjusted Goals Saved Above Average per 60, a rate stat that shows how much better/worse a goalie is doing than if a league average goalier were in his shoes, facing the same shot workload. (Added reading on adjSAA/60 can be found here). It is a better way to compare goalies because league average is set to 0. So you know at a glance whether a goalie is better or worse than an average goalie, independent from their environment (or at least as independent as we have at this point). It also is great for comparing goalies across seasons for the same reason. We now know which goalies were better than their peers, how much better, and how that compares to a goalie from another season. It goes beyond save percentage, which doesn’t tell us anything alone without knowing the league average for that year. I’ve utilized adjGSAA/60 and different forecasting systems and gotten intriguing performance prediction results which I want to look into further.

I also think we can improve on our microstat tracking. Of course, this takes man power. Folks like former NHL goaltender Steve Valiquette are doing interesting work showing what types of plays impact goalie performance. Ryan Stimson’s passing project will help reveal additional environmental impacts on overall performance. But we need more folks with the time and interest, to direct that interest towards analyzing things like positional performance, type of save performance, angle line percentage (how often a goalie is square to a shot) and other things that will further unlock the secrets of success at the position.

TSS: Shot quality has proven to be very difficult to quantify. Do you think that shot quality plays a role in today’s NHL? How do you feel your belief on the subject effects how players/goalies should be analyzed?

Balloch: Shot quality is pretty much everything in the NHL. Like I said before, the area that the shot comes from does play a part. Goalies would much rather face Stamkos from the point than the high slot, but that’s a simplistic way of looking at things. Every shot is different, and they should be weighted differently when analyzing if a goaltender should have stopped it.

With the statistics we have right now, goalies absolutely cannot be judged based on their statistics alone. There are so many other factors that go into a goaltender’s success, it’s sometimes painfully obvious why certain goaltenders don’t succeed in certain situations, and others do. I find this especially to be true when judging junior-level goaltenders. If all you do is look at raw save percentage and goals against average (the only stats usually available) when deciding on your thoughts about a goaltender, you’ll miss out on a lot of great talent. To get anywhere with goaltending analysis, you need to see the goalie play, then check the statistics to see if it makes sense.

Silverman: Hey, this kind of goes with how I answered question one!

I think that shot quality is *such* a huge part of how we evaluate today’s NHL talent. High versus low shooting percentages currently don’t factor in the quality of shots taken, and that makes it very hard to accurately look at player AND goalie numbers and truly get a sense for how a certain figure can either regress or be maintained.

That being said, I also think that even shot quality is something that needs more consistency when we look at it. Mike Milbury made a comment to goaltender Brian Boucher during the 2015 Stanley Cup Playoffs that I think really highlights this: he disagreed with Boucher’s observation that New York Rangers netminder Henrik Lundqvist had made a number of key saves in one particular game, instead asserting that Tampa Bay Lightning netminder Ben Bishop was the one who had made all the heroic saves that night. What Milbury doesn’t understand, though, is that Lundqvist was actually facing plenty of high quality shots – he was just tracking the shooters very well, then getting himself into position early on to make what looked like very ‘easy’ saves. In contrast, Bishop was facing shots that he wasn’t reading well, so he was having to dive and contort himself to make the saves; to Milbury, Bishop appeared to be making the fancier (and therefore, in his mind, harder) saves, but that was a fallacy in his understanding of goaltending itself. The way people understand and observe netminders will have to change before we can see consistent tracking of shot quality, because a lack of comprehension on the save end can lead to these false perceptions about the shots being faced themselves.

Just because there’s a low understanding of that problem, though, doesn’t mean that the concept itself is unimportant. If anything, shot quality is probably the biggest thing missing in current analytics.

Campbell: Shot quality has been hard to quantify because no large-scale publicly-available data tracking pre-shot puck movement has been available. Chris Boyle’s Shot Quality Project and Steve Valiquette’s Royal Road work set the stage, but remain limited in scope and mostly hidden from public view. Ryan Stimson’s publicly accessible Passing Project will track (among many other thing) pre-shot movement on a large scale for the upcoming season. This will be significantly more useful than current danger-zone data.Obviously, I think shot quality is key for disentangling goalie performance from talent. A shot resulting from a 2-on-0 break is much more difficult to stop than a stationary wrister from the face-off dot. A thorough, data-driven analysis of shot-quality would be able to give statistical power to expert intuitions, and enable a far more thorough and accurate comparison of goaltenders behind very different teams. Insulated goaltenders behind strong defensive systems cease to look so impressive when you can show, in high detail, how easy their lives are compared to goalies exposed to impossible shots every game.

Mercadante: I do. But we may not have the tools right now to adequately analyze its impact in ways that aren’t already either accounted for in existing numbers, or aren’t swamped by the noise of a million other variables.

On the ice, I know that if I approach a defenseman controlling the gap between me, himself, and the goal, and I don’t add an element of deception to my shot, a good goalie is going to square up on the angle, take away visible net, and probably gobble up my shot. But if I toe-drag across the defenseman’s body, using him as a screen and changing the angle of the shot simultaneously, I’ve likely increased my odds of scoring. Maybe the goalie didn’t see the move, or didn’t adequately adjust to the new angle line. And maybe the action of pulling will give me a quicker wrist-flick on the snap shot which will add velocity, or improve the trajectory of the shot. Those are important quality considerations for the ice.

But in terms of analyzing data, things like large sample shot attempt differential simply evens out the randomness/luck/unpredictibleness of shooting percentage a lot quicker than breaking things into smaller samples of certain shot types. Said a different way, it would take a much longer time to learn everything with any degree of certainty if we only studied a particular type of shot. Or if we traded goal differential for the currently widely practiced shot attempt differential (CF%) analysis. So it can’t be relied upon as much or as quickly in-season, rendering quality less useful for analysis.

I think that gaining access to tracking tools like radio frequency identification (RFID) and camera tracking will certainly yield interesting data. But I reject the notion that it will render the current methods of analysis obsolete, or that everyone will trade in well-founded statistical analysis principles regarding large sample regressions for pure quality analysis. I say that with trepidation, because I am a hockey coach and player first. I know how important quality is to a coach.

Part II will be published Sunday, January 3.