In my last post, I detailed how there is a trend of people placing too much importance on the context behind a player’s statistics, even when such contextual numbers do not at all explain the poor (or great) performance. Part of this is because old research indicated that quality of competition and zone starts have a much greater impact than more recent research has found to be the case.

In particular, the impact of quality of competition is often dramatically overstated. This is a little understandable – after all, it should matter a lot who is on the ice against a player at a given time. And in fact it does:

But here’s the key thing: While it matters if a player is facing Sidney Crosby instead of John Scott at any given moment, the range of competition that a player faces over the course of a season is EXTREMELY SMALL. The gap between the players facing the hardest competition and those facing the weakest competition is the same as facing an average player at most like 4 shot attempts per 60. In other words, the guy with the toughest competition in the league will face an average opponent who is +2 corsi/60, while the guy facing the weakest will face an average opponent who is -2 corsi/60. And nearly all players won’t be in these extremes – most will be within -1 corsi/60 and +1 corsi/60. And as you might expect the gap between opponents who are +1 shot attempts per 60 and those -1 is practically nothing.

Yet you’ll hear people talk about how one player plays “really weak” competition or another player’s bad #s are because he takes “the toughs” – this doesn’t really mean anything.

This can perhaps be illustrated best by looking at Dion Phaneuf and the Maple Leafs’ D Corp:



—————————————————-

Hockey Analysis‘ WOWY pages have a little feature that goes mainly overlooked: it includes at the bottom of every player’s WOWY #s the #s of that player against each forward and then each D man against whom that player has played 5 minutes or more. Without going a step further, these #s are basically useless – knowing how well a player has played in 20 minutes against a single opponent doesn’t tell you anything. However, if you download these numbers to a spreadsheet, you can use them to see how well a player does against different levels of competition. In other words, if you ever wanted to know how well a player does against weak opponents or against elite opponents, aggregating these numbers will give you a decent idea.

There are a few caveats of this type of analysis of course: because it only counts opponents who have played 5 or more minutes against a player, the overall #s will be slightly off (so the sample size a shutdown D-man plays against weaker competition is likely to be understated, and the same for a 3rd pair vs stronger comp). It also isn’t quite counting how tough a particular player’s minutes are at a given time – If a player is facing a 56% Corsi forward on a line with two 40% corsi forwards for some reason, part of that is counted as if the player is facing a 56% corsi opponent, even though the opposition isn’t close to that tough. In addition, we’re not taking into account zone-starts or teammates here, which CAN matter depending upon the team/players you’re looking at.

That said, despite the caveats, this does give us a pretty good approximation of how well a player can do against various types of competition. So I decided to use this method to look at the Leafs’ D Corp (except for Fraser, for whom there simply isn’t enough of a sample to run this analysis) and the results you can see in the below chart:

Essentially I split up the opponents of the Leafs into four groups:

1. Poor Opponents – sub 47% Corsi players who averaged around 45% corsi overall

2. Mediocre Opponents – sub 49.5% corsi players who averaged around 48.5% corsi overall

3. Good Opponents – sub 53% corsi players who averaged around 51.5% corsi overall

4. Elite Opponents – 53%+ corsi players who averaged 55% corsi overall.

In the graph above, the black bar shows how well the average player does against each group of opponents – so of course the average player puts up their best numbers against the poor players, and the worst vs the elite players, and in between for the mediocre and good players.

Take a look at the crimson bar belonging to Dion Phaneuf. Phaneuf supporters often mention how he plays (by some measures) the toughest opposition in the league. And indeed, about 35% of his time on ice is against elite opponents and another 31% is against good opponents – pretty high #s. But as you should have guessed by now, that really doesn’t explain how poor his possession #s are by any stretch. As the chart above shows, Phaneuf turns 55% corsi players into 66% corsi players – these guys basically go from elite to absolutely dominant when facing Dion Phaneuf. And it’s not like Phaneuf does better against weaker players either – against poor opponents, Phaneuf turns 45% corsi players (very bad) into 53.25% corsi players (pretty darn good).

You can’t even blame the forward corp for Dion’s #s – look at how Dion’s #s compare to the gray bar – which is the average Leaf #s – he’s basically worse vs than the Leaf’s average vs every group of opponents (except for mediocre players, against whom there is the least TOI). Nor can you blame zone starts – Leaf zone starts are basically the same across the Leaf D Corp except for Morgan Rielly. And hell, this year he can’t even blame his d-partner – he’s been with Gunnarson practically all year.

In fact using this method, we can also see how well Phaneuf would do if we had him face 25% of his ice time against each group (average competition) – his corsi jumps from 39.67% to 41.6% corsi – from really lousy to fractionally less really lousy.* The effect is basically nothing.

*You’ll note these #s don’t exactly match Phaneuf’s #s at HockeyAnalysis, because of the missing data against players whom Phaneuf hasn’t even faced 5 minutes this year. But it’s close enough to illustrate the point.

In fact, we can see this another way – why don’t we see how the other Leafs’ D #s change if they had to face Phaneuf’s competition instead of their own?

As you can see, basically nothing changes (not surprising for Gunnarson, who mostly plays WITH Phaneuf) for any of the D men despite facing the tougher competition. The only exception is Paul Ranger, which appears to be a sample size quirk that is unlikely to continue (he has very little time vs tough competition in which he has done hilariously poorly which is almost certainly a sample size quirk, but this method acts as if it’s real and uses that # for a large portion of Ranger’s theoretical ice time.) Jake Gardiner goes from the Leafs’ best D Man to….the Leafs’ best D man. It barely makes a dent.

Again, the zone starts for all but Morgan Rielly are basicaly identical (and Rielly’s gap isn’t very large either) and these are with the same teammates. Quite simply put anyone who points out Dion faces top competition as an argument in his favor needs to understand that Phaneuf has been below average against all levels of competition and the top competition he’s faced really doesn’t do much to his numbers.

Conclusion:

Phaneuf’s example is pretty typical actually – he’s not bad because he faces top competition and he’s not likely to be much better against weaker competition either – nor will he be above average against such competition (You can argue he’d be better with a different coach, but that’s another story). This isn’t unique to him – other defensemen who fare poorly against top competition also will show the same situation (I did this same study with Andrew MacDonald of the Isles in a post on Lighthouse Hockey).

Competition on the ice at any given moment can matter a lot. But over the course of the season, it really doesn’t matter that much. Let’s stop pretending that it does.