The difficulty in measuring punter effectiveness

Punter is a difficult position to measure, the low sample of attempts over a season, the lack of a solid metric in which to judge, the NFL no longer publishing hang time statistics. There are a lot of problems with trying to say punter X is better than punter Y. It’s a debate that may not have a right answer and will get me a lot of hate online, assuming I can get you emotionally invested enough in the position. So with that said, let’s do it.

Wins above something

Wins above replacement has been a go to metric for baseball for sometime, though it’s conversion to football has been slow going for a number of reasons, first of all it’s difficult to try and narrowly isolate the impact of the performance of one position on the field from the other 10. This difficulty has made progress slow, though some incredible minds have started to crack the code for positions such as quarterback, running back, and wide receiver because isolating their influence is easier than say a linebacker or center.

For more on NFL WAR, read here.

First, to define wins above replacement we have to set a metric for a win. In the above link, a win is estimated to be approximately 31 points. For the sake of this analysis, we will assume this is true and that any increase in offense or decrease in points allowed on defense of 31 points is worth approximately one win in expectation over the course of the season.

The next problem is that punters are rarely injured and thus replaced mid-season. Thus, defining above “replacement” becomes difficult, instead we will construct wins above average by year. This will still allow us to have insight into the relative performance of a punter without finding the typical replacement punter. The debate over WAR vs WAA has been going on for a long time, and it’s not one I will get into here. The primary goal is to make something we can use.

Punts as expected points prevented on defense

Now for the interesting bit, defining the value of a punt in terms of points. Intuitively, a well positioned punt will allow for less ability to return, and therefore back a team up farther on the field. One would think this would have a measurable effect on the defense’s ability to keep points off the board. If so, we would expect the better a punter positions his punts, the fewer points per game a defense would allow.

For the purposes of this study, punt placement is defined as the point where the ball comes to rest on the field or is fielded by an opposing player. Thus, punt placement does not take into account return, muffs, nor penalties. The point of this is to try and isolate the performance of the punter from the rest of the special teams play. There will be of course still exogenous factors we cannot control, such as the offense not being able to move the ball and thus causing the punter to punt from farther back. While this is a factor, we could control this by omitting punts that occur from inside an team’s own ten yard line. But even for teams with trouble moving the ball this may not capture the effect and in fact may skew our results so I will not filter in this manner at this time.

Why? It may be possible to go through yardage clusters and judge where we expect a punt to land given where it was kicked. But it’s also true that punters tend to kick a lot harder further back and control, assuming they keep it in bounds, is less of a concern. So it’s not just about how far they kicked it, it’s about placement on the field in general. Additionally, controlling for lack of offensive movement of the ball starts to induce sample size issues almost immediately. For example, if I look at punter Y’s punts from on or about his own 45 only I can possibly judge and construct a confidence interval for his punt placements, but what it punter z never punted within 5 yards of that same kick location? Unlikely? Possibly, but we could quickly run into issues where we aren’t able to compare punters from every team. Instead, I propose we use punt placement. Placement is influenced by the offense, a punter may have help from his offense in putting the ball within the opponent’s own 20. But for this first iteration, starting field position is not controlled for—though it is something I will try to explore more it later on.

We can see, measured in this way, the placement of a punt as having a measurable effect on scoring defense. With a p-value below .001, we see that a one yard change in average punt placement correlates to a .4 point fluctuation in ppg allowed on defense. This helps explain roughly 6.8% of the variance of defensive performance. While this isn’t world breaking, the effect is strong and measurable.

Punter wins above average

Assuming the above is true, which warrants more investigation to be sure, we can then measure the number of wins above average a punter provides to his team in terms of points allowed, and therefor expected wins over the course of the season.

Interesting there is a dip at WAA of zero, this could be an artifact of sample size or a flaw in the metric. However, the data does seem to be mostly normal and centered on zero, which is what we would expect.

If our metric has measured this effect correctly and explains the number of wins above average a punter contributes then in 2017 we can see that Jon Ryan was a below average punter last year though not by much. This may be heartbreaking though not unexpected given his performance last year. And for Seahawks fans as his time is likely over given the selection of Michael Dickson in the fifth round and Ryan’s large 2018 cap hit, it’s likely time to say goodbye the the Seattle icon.

The value of good punting

We can see from the above analysis that punting’s quantitative effect may be larger than we often give credit for, though it’s also interesting to note that even the best punting performances over the last five years account for, at most, about one win above average. However, given that punting may correlate to and explain 6.8% of defensive performance, perhaps it’s then less surprising that John Schneider traded up for and picked arguably the best punter in NCAA history Dickson.

Additionally, this insight makes the Oakland Raiders’ decision to cut Marquette King all that much more puzzling. In doing so, they have essentially handed a division rival a punter who performed .7 WAA in 2017 and who is consistently in the top five given this metric. Below, are the top 10 teams by punter WAA:

There are a lot of open questions, is this metric really any good? Can we do a better job by controlling for offense? Does it have predictive value going forward? How stable is it year to year? These are all fair questions and ones that I plan to address over the course of the off season. But judging punters in terms of wins is fairly uncharted waters, the seas may be rough, but maybe we can all learn something together along the way. I hope you’ve enjoyed this little deep dive into the least appreciated position in the NFL, and remember, keep punting.

This article made possible by nflscrapR. Thank you to all involved in its development.