Examining Luck in NFL Turnovers Written by Julian Ryan Written by Julian Ryan

Details Published: 24 October 2014

AFA continues its partnership with the Harvard Sports Analysis Club. The following analysis originally appeared on HarvardSportsAnalysis.org

There are relatively few possessions in an NFL football game, so in short, turnovers matter. While a right back on the soccer pitch can be forgiven for a mislaid pass or a point guard in basketball can get away with the occasional double dribble, if a quarterback throws an interception, it can often change the course of the game. Just ask Tony Romo.

However, aside from Matt Schaub’s natural aptitude for the pick six, how random are turnovers? How much of a team’s turnover differential can be explained by teams protecting the football on offense and causing turnovers on defense, and how much is just plain dumb luck?

First, it is important to stress just how much turnovers matter in deciding a single game of football. Taking data from pro-football-reference from 2002-2013, if a team has even one more turnover than the other team over the course of a game, it will win 69.6% of the time, compare that to home teams winning about 57.2% of the time. Winning percentage only increases as the turnover differential mounts up (ties excluded in table below):

Turnover Differential Wins Losses Win% +1 747 327 69.6% +2 608 117 83.9% +3 342 35 90.7% +4 186 6 96.9% +5 68 3 95.8% +6 18 0 100.0% +7 5 0 100.0%

If we expand to look at turnover differential on a seasonal level, its importance is even clearer. Regressing season wins against seasonal turnover differential indicates that each additional positive turnover is worth about 0.2 wins with an R-squared value of a whopping 0.419. That suggests that the seasonal turnover differential of a team alone – ignoring points scored, points against, strength of schedule and everything else that affects winning percentage – explains 41.9% of variation in regular season win totals from 2002-2013. Again, turnovers matter a lot.

Some of a team’s turnover differential is due to talent —Aaron Rodgers throwing very few interceptions and J.J. Watt strip-sacking anything that moves come to mind as examples—and some is due to luck, because things like this happen. To ascertain how much each of these two factors contribute to turnover differential over the course of a season, we can borrow from the pioneering work of Tom Tango. If we take it as given that luck is independent of talent, then the following formula will hold:

Var (Observed TO Differential) = Var (TO Differential from Luck) + Var (TO Differential from Talent)

Tango then concludes what percentage of wins in baseball is due to skill because we observe the left hand side and can calculate the variance due to luck. This latter amount is what the variance would be if turnovers (or baseball wins for Tango) had nothing to do with skill whatsoever. For baseball wins, this is equivalent to flipping a coin for each game so a season’s win total is a binomial distribution with 162 trials and a success probability of 0.5. Calculating the variance of this ‘luck distribution’ allows us to assign the remaining variance to talent. To repeat this analysis for turnovers, we need to find our own ‘luck distribution’.

The Poisson distribution is used to model rare independent discrete events over a period of time with a constant (low) probability of the event occurring in each period, like earthquakes. It makes sense to model total turnovers per game this way, for if we consider there to be a constant probability of turnover per possession and an average number of possessions per game, then the total per game should follow a Poisson probability density. This isn’t perfect as possessions per game is not constant and turnovers may shorten drives and thus increase possessions per game (and also are not independent) but on the macro level the data almost perfectly matches the Poisson model. The bar graph below shows the observed probability density of total turnovers per game alongside what we would expect if total turnovers followed a Poisson distribution with mean 3.39 (the average in our data set).

Under the assumption that total turnovers per game is now Poisson with a mean of 3.39, we can now calculate our "luck distribution". Under this distribution, neither team is better or worse at producing or creating turnovers, so any change of possession happens entirely by luck. This, together with the fact that the sum of independent Poisson random variables is itself a Poisson random variable (with the mean equaling the sum of the means of the original variables), means we can split up this Poisson into two halves: one for the number of turnovers created on defense, and the other for turnovers conceded on offense. In words, for a single team in a single game, the turnovers it "forces" are a Poisson random variable with mean 1.69 as are the turnovers it "commits".

Each team plays 16 games per season, so the total seasonal turnovers "forced" is the sum of 16 independent Poissons with mean 1.69, which is hence itself a Poisson with mean 27.1. The same process holds for turnovers "committed". Seasonal turnover differential would thus simply be the difference between these two independent random variables.

The difference between two independent Poisson variable is not itself Poisson, but because they are independent (which implies covariance equals zero) it is easy to calculate the variance of seasonal turnover differential under our luck distribution, since an individual Poisson's variance is equivalent to its mean. After calculating the observed variance of seasonal turnover differentials, we are able to see how much of that number is due to plain dumb luck (confidence intervals calculated by considering the error in calculating the mean total turnovers per game in our 3072 game sample):

Variance Value Percentage 95% Confidence Interval Var (Observed Seasonal TO Differential) 99.0 - - Var (Seasonal TO Differential from Luck) 54.2 54.7% (53.7%, 55.8%) Var (Seasonal TO differential from Talent) 44.8 45.3% (46.3%, 44.2%)

So after a few hundred words of statistics, we arrive at a whopping conclusion that just over half of seasonal turnover differential is due to luck. That’s huge, especially when you consider that (from earlier) seasonal turnover differential explains over 40% of seasonal winning percentage.

At first glance, this does seem very high to me but evidence for this magnitude is the extraordinary year-to-year variability in turnover differential, which you would expect if luck was a mega factor as my analysis suggests. While starting quarterbacks absolutely play a role in turnover differential (Tom Brady throws fewer picks than Chad Henne) and tend to be fairly constant from year-to-year, nevertheless the correlation between turnover differential last year and this year is only 0.086 which is not significant at the 5% level.

Ultimately in a 16-game season, there’s just a whole lot of luck involved with winning football games and for all that commentators will talk about defensive schemes forcing turnovers this season, it’s just as important to be lucky as to be good.