American football is a popular sport in the US and around the world. Rabid fans and professional pundits often debate what contributes most to the outcome of games in the National Football League (NFL), and opinions vary widely. In pregame shows it is not uncommon to have one expert speak of the virtues of a strong run defense, while another will marvel why a team with strong passing offense can’t win a game. Evidence of the effect of player actions on game outcomes would obviously be of interest to all concerned with NFL: players, coaches, owners, and spectators. There is limited published research on this topic in any of the football codes. James et al. (2001) developed performance profiles specific to positions in 21 professional rugby union matches. In regards to American football, Stair et al. (2008) analyzed the effects of various factors pertaining to off-field conduct on the performance of NFL teams, with special attention to the number of arrests of team members (which was not statistically significant). Alamar and Weinstein-Gould (2008) investigated the contribution of individual NFL linemen to their teams' passing performance. White and Berry (2002) used logistic regression to rank NFL quarterbacks by finding a quantitative value for various plays, but not game outcomes, that occur in NFL games. However, a comprehensive analysis of the effects of common statistical measures in American football on the likelihood of winning has not been previously published. The aim of this study was therefore to perform such an analysis and to determine how the large number of inter-related statistical measures can be grouped into a smaller set of independent key performance indicators. Methods Over 100 variables were included from a sample of 1,335 NFL games spanning the 2004 through 2008 seasons. The data were collected from the website http://nfldata.com. Variables given in the original data set that do not represent in-game events (playing surface, temperature, weather conditions, predictive point spread, etc.) were removed, leaving 67 team-performance variables (Tables 1 and 2). Each game has data from two teams that mirror each other; therefore, data from only one team (selected at random) were utilized for each game to avoid duplication of data and problems with repeated measurement. For example, an occurrence of offensive rushing for one team is an occurrence of defensive rushing for the other. Terms such as defensive passing yards or defensive first downs refer to yards or first downs attained by the non-selected team when in possession of the ball. See the commentary by George Osorio for clarification of the rules of American football and an explanation of the terminology. The principal-components version of factor analysis was employed to group the standardized game statistics into independent sets. The analysis was realized with Proc Factor in SAS using the defaults for identifying an appropriate number of factors and varimax rotation to combine the statistics into independent factors. The factors were named according to their perceived football-specific characteristics. Univariate logistic regression using Proc Logistic in SAS was used to estimate the individual effect of each factor on the chances of winning. In these analyses, the values of the factor were scores from every game computed from the loadings and the values of the variables in the factor. As described elsewhere (Hopkins, 2010), the magnitude of the effect of the factor in question was estimated first as the ratio of odds of winning for games that differed by two standard deviations; that is, the difference between games with a typically low and high value of the factor. The odds ratio was then converted to a difference in the chances of winning centered on a 50% chance; for example, an odds ratio of 2.0 is equivalent to a difference of 18% in chances of winning (a 59% chance for a game on the high value of the factor vs 41% for a game on the low value). The difference in chances was then interpreted using the following scale: <10%, trivial; 10-30%, small; 30-50%, moderate; 50-70%, large; 70-90%, very large; and >90%, extremely large (Hopkins et al., 2009). The ability of all 14 factors to account for game outcomes was quantified by developing a logistic model with the 14 factors as main-effects predictors. The percent of game outcomes predicted correctly by this model was then calculated. Univariate logistic regression using Proc Logistic in SAS was also used to estimate the individual effect of each game statistic on the chances of winning. For these analyses the one game resulting in a tie was removed from the dataset. A multiple logistic regression was also performed using all variables to ascertain the predictive power of all the original variables. Backward stepwise logistic regression was used to reduce the set of game variables to a more succinct set. In a validation study, the multiple logistic regression analysis was repeated with data from the 2004-2007 seasons, and the resulting logistic model with all 14 factors was used to predict the outcomes of the 2008 games for comparison with the actual outcomes. Because of the large number of effects estimated in this study, uncertainty in all estimates was calculated conservatively as 99% confidence limits. Outcomes were assessed using the paradigm of mechanistic magnitude-based inference. All outcomes were clear, owing to the large sample size, so the magnitudes of the observed effects were interpreted directly as population magnitudes without probabilistic qualifiers. Results The factor analysis yielded 14 factors; the loadings of the variables in the factors are given in Table 1. Fifty-nine of the 67 variables contributed to the factors, and none of the variables contributed to more than one factor. The factors are numbered in descending order based on the amount of variation explained by the given factor. Offensive and defensive factors are those attained while the ball is in possession of the team or the opponent respectively. The choice of 14 factors was based on the total number of eigenvalues >1. Other models with different numbers of factors were analyzed, but the factor loadings made factor interpretation difficult and the analyses are not shown here. Table 1. The game statistics making up the 14 factors and the game statistics that did not contribute to a factor. Numbers in parentheses are the factor loadings. Factor 1 – Defensive Passing & Total Defense Defensive Passing Yards (0.92) Defensive Yards / Play (0.71) Defensive 1st Downs by Passing (0.87) Defensive Passer Rating (0.66) Defensive Yards (0.81) Defensive Passing TDs (0.63) Defensive Passing Completions (0.76) Defensive Passing Completion % (0.63) Defensive 1st Downs (0.75) Defensive Red Zone Attempts (0.51) Factor 2 – Offensive Passing & Total Offense Offensive Passing Yards (0.91) Offensive 1st Downs (0.74) Offensive 1st Downs by Passing (0.88) Offensive Passer Rating (0.70) Offensive Yards (0.83) Offensive Passing TDs (0.69) Offensive Yards / Play (0.76) Offensive Passing Completion % (0.62) Offensive Passing Completions (0.75) Factor 3 – Offensive Rushing Offensive Rushing Yards (0.92) Offensive Rushing Attempts (0.65) Offensive 1st Downs by Rushing (0.84) Offensive Rushing TDs (0.61) Offensive Rushing Yards / Attempt (0.77) Factor 4 – Defensive Rushing Defensive Rushing Yards (0.90) Defensive Rushing TDs (0.65) Defensive 1st Downs by Rushing (0.83) Defensive Rushing Attempts (0.61) Defensive Rushing Yards / Attempt (0.78) Factor 5 – Turnovers Turnover Differential (0.91) Takeaways (0.69) Giveaways (-0.67) Factor 6 – Offensive Ball Control Offensive 3rd Down Attempts (0.80) Offensive 3rd Down Conversions (0.70) Offensive Plays (0.73) Factor 7 – Defensive Ball Control Defensive 3rd Down Attempts (0.83) Defensive 3rd Down Conversions (0.65) Defensive Plays (0.69) Factor 8 – Defensive 4th Down Performance Defensive 4th Down Conversions (0.94) Defensive 4th Down Attempts (0.79) Defensive 4th Down Conversion % (0.84) Factor 9 – Offensive 4th Down Performance Offensive 4th Down Conversions (0.94) Offensive 4th Down Attempts (0.80) Offensive 4th Down Conversion % (0.84) Factor 10 – Good Penalties Good Penalty Yards (0.91) Offensive 1st Downs by Penalty (0.76) Good Penalties (0.90) Factor 11 – Defensive Sack Performance Defensive Sacks (0.87) Defensive Passing Yards / Completion (-0.52) Defensive Sack Yards (0.87) Factor 12 – Bad Penalties Bad Penalty Yards (0.89) Defensive 1st Downs by Penalty (0.76) Bad Penalties (0.88) Factor 13 – Possession Change Defensive Punts (0.62) Defensive 3rd Down Conversion % (-0.54) Offensive Punts (0.62) Factor 14 – Offensive Sack Performance Offensive Sack Yards (0.89) Offensive Passing Yards / Completion (-0.50) Offensive Sacks (0.88) Game Statistics not Loading on a Factor: Offensive and Defensive Time of Possession, Offensive and Defensive Passing Attempts, Offensive 3rd Down Conversion %, Offensive Red Zone Attempts, Offensive and Defensive Red Zone Conversions. Table 2 shows the outcome of the logistic regression with the factors as predictors. Of the 14 factors, 11 have non-trivial effects. Turnovers is the most powerful predictor out of the 14 factors, and the rushing-related factors have slightly larger effects than the passing-related factors. When all 14 factors were included in a multiple logistic regression model, 91% of the 635 won games and 92% of the 699 lost games were predicted correctly. Table 2. Effect of factors derived from game statistics (Table 1) on game outcome. Effects are expressed as odds ratios and the equivalent difference in chances of winning for a team with a high value of the factor in a game vs a team with a low value (a difference between the teams of two standard deviations of the factor). Odds

ratio Difference in

chances (%) Large Effect Factor 5 – Turnovers 19 63 Moderate Effects Factor 4 – Defensive Rushing 0.17 -41 Factor 3 – Offensive Rushing 5.3 40 Factor 1 – Defensive Passing & Total Defense 0.23 -36 Factor 2 – Offensive Passing & Total Offense 4.3 35 Factor 11 – Defensive Sack Performance 4.0 33 Small Effects Factor 14 – Offensive Sack Performance 0.35 -26 Factor 10 – Good Penalties 2.2 20 Factor 9 – Offensive 4th Down Performance 0.51 -17 Factor 12 – Bad Penalties 0.55 -15 Factor 8 – Defensive 4th Down Performance 1.6 12 Trivial Effects Factor 6 – Offensive Ball Control 1.2 5 Factor 13 – Possession Change 1.2 5 Factor 7 – Defensive Ball Control 1.0 0 Magnitudes are based on the following scale for percent differences: <10, trivial; 10-30, small; 30-50, moderate; 50-70, large; 70-90, very large; >90, extremely large (Hopkins et al., 2009). Uncertainties (99% confidence limits) in the odds ratios are ×/÷1.5 (for the largest effect) to ×/÷1.3 (for trivial effects). Uncertainties in the differences in chances are all approximately ±7%. Results of the logistic regressions using each of the original game statistics as a predictor of game outcome are shown in Table 3, with the predictors grouped according to the factors they contributed to. When all the game statistics were included as predictors in a logistic model derived for the 2005-2007 games, the resulting model applied to the 2008 games predicted 93% of the 122 won games and 92% of the 144 lost games correctly. A smaller set of 24 game statistics identified by backwards stepwise selection is shown in Table 4. This set predicted the same proportions of won and lost games as the full set. Table 3. Effect of each game statistic on percent chances of winning derived by univariate logistic regression. As in Table 2, effects are expressed as difference in chances of winning for a team with a high value of the statistic in a game vs a team with a low value (a difference between the teams of two standard deviations of the statistic). Factor 1 – Defensive Passing & Total Defense Defensive Passer Rating (85) Defensive Red Zone Attempts (74) Defensive Yards / Play (64) Defensive Passing Completion % (64) Defensive Passing TDs (54) Defensive Passing Completions (26) Defensive Passing Yards (16) Defensive 1st Downs by Passing (11) Factor 2 – Offensive Passing & Total Offense Offensive Passer Rating (79) Offensive Yards / Play (53) Offensive Passing Completion % (49) Offensive Passing TDs (48) Offensive Passing Completions (27) Offensive Passing Yards (10) Offensive 1st Downs by Passing (7) Factor 3 – Offensive Rushing Offensive Rushing Attempts (91) Offensive Rushing Yards (76) Offensive 1st Downs by Rushing (68) Offensive Rushing TDs (67) Offensive Rushing Yards / Attempt (16) Factor 4 – Defensive Rushing Defensive Rushing Attempts (94) Defensive Rushing Yards (79) Defensive 1st Downs by Rushing (74) Defensive Rushing TDs (71) Defensive Rushing Yards / Attempt (18) Factor 5 – Turnovers Takeaways (80) Giveaways (74) Factor 6 – Offensive Ball Control Offensive 3rd Down Conversions (40) Offensive 3rd Down Attempts (8) Factor 7 – Defensive Ball Control Defensive 3rd Down Conversions (50) Defensive 3rd Down Attempts (1) Factor 8 – Defensive 4th Down Performance Defensive 4th Down Attempts (61) Defensive 4th Down Conversions (21) Defensive 4th Down Conversion % (1) Factor 9 – Offensive 4th Down Performance Offensive 4th Down Attempts (63) Offensive 4th Down Conversions (25) Offensive 4th Down Conversion % (4) Factor 10 – Good Penalties Good Penalties (28) Good Penalty Yards (20) Offensive 1st Downs by Penalty (9) Factor 11 – Defensive Sack Performance Defensive Sack Yards (70) Defensive Sacks (67) Defensive Passing Yards / Completion (59) Factor 12 – Bad Penalties Bad Penalties (20) Bad Penalty Yards (15) Defensive 1st Downs by Penalty (12) Factor 13 – Possession Change Defensive 3rd Down Conversion % (59) Defensive Punts (32) Offensive Punts (23) Factor 14 – Offensive Sack Performance Offensive Sack Yards (59) Offensive Passing Yards / Completion (50) Offensive Sacks (6) Not Loading on a Factor Offensive Time of Possession (75) Defensive Red Zone Conversions (74) Defensive Time of Possession (74) Offensive Red Zone Attempts (73) Defensive Passing Attempts (70) Offensive Red Zone Conversions (68) Offensive Passing Attempts (61) Offensive 3rd Down Conversion % (47) Table 4. A subset of game statistics identified by backwards stepwise selection that correctly predicted the same proportion of won games and lost games as did the model developed with the full set of game statistics. Offensive 1st Downs by Passing Offensive 1st Downs by Penalty Offensive Rushing TDs Offensive Yards/Play Offensive Passing TDs Offensive Passing Yards Offensive 4th Down Conversion % Offensive 4th Down Attempts Offensive Punts Bad Penalty Yards Defensive Rushing TDs Defensive Rushing Attempts Defensive Passing Completions Defensive Passing Yards Defensive Passing TDs Defensive Passer Rating Defensive Time of Possession Defensive Passing Attempts Defensive 4th Down Attempts Defensive 3rd Down Conversion % Defensive 4th Down Conversions Defensive Punts Takeaways Giveaways Discussion The 14 factors have relatively clear and concise interpretations, and they represent characteristics that in our experience are commonly used to describe a team’s game performance. Logistic regression with the 14 factors showed that the factors related to ball control (offensive ball control, possession change, defensive ball control) had little relationship with game outcome (Table 2), but the other 11 factors all made substantial contributions. In particular, turnovers and rushing (offensive and defensive) were especially predictive of game outcomes, in addition to passing (offensive and defensive) and total offense/defense. The casual fan would probably agree that these five factors constitute the pillars of football success for a team. The remaining factors reflect the relatively lesser importance of sacks (defensive and offensive), penalties (good and bad), and 4th down performance (offensive and defensive). When the results of the factor analysis are compared to those of the univariate logistic regressions performed on the original variables, it becomes evident that factor analysis can exclude variables that can be very predictive of the game result. For example, both offensive and defensive time of possession have large differences in chances of winning of approximately 75%. However, neither of these variables loaded significantly on a factor as a result of the factor analysis. This is most likely due to the level of correlation these variables have with other variables that did load on a factor. The predictive power of the factor analysis does exhibit its effectiveness of data reduction. The logistic regression using the factors was successful in predicting the results 91-92% of the time (only slightly less than what was exhibited using the original variables, 92-93%). Owing to substantial correlations among the original variables, interpretation of individual coefficients from any of the original-variable logistic models was not attempted. However, the predictive power of these models is remarkable and could be useful, in the aggregate, in making predictions. Given that prediction was applied to game data mutually exclusive of the data the model was built from, more confidence is given to ability of this group of 60 variables to predict game outcomes. The results presented here suggest the in-game variables from an NFL game are powerful predictors of a game’s outcome. Clearly, the models are not perfect representations of game outcomes due to un-measurable variables and randomness. Future work will focus on building off these results to predict game outcomes before the game. The stepwise logistic regression also yielded interesting results. Note that there were two variables that survived this method that did not load significantly on a factor in the factor analysis (Defensive Passing Attempts and Defensive Time of Possession). Not surprisingly, the predictive power of the stepwise method yielded identical results from the model utilizing all of the original variables. The results of the stepwise method also provided results that were somewhat similar to the factor analysis. All but 3 of the 14 factors were represented with one or more of its represented variables. The factors not represented in the stepwise method were the sack performance factors (both offensive and defensive) and offensive ball control. In conclusion, we acknowledge that many important predictor variables are nearly impossible to collect and analyze. As examples, location (home or away), weather conditions, injuries, and time of year or playoff implications all impact outcomes of NFL games. In the analyses for this article we simply attempted to gain a better understanding of the subset of variables that were collected. It should also be noted that and impending outcome in a game will change players' behavior. For example, it is common for a football team, when leading late in the game, to run more rushing plays, which tend to expend more time and shorten the game. The effects of such changes in behavior need investigation. Reviewer's Commentary References Alamar BC, Weinstein-Gould J (2008). Isolating the effect of individual linemen on the passing game in the National Football League. Journal of Quantitative Analysis in Sports 4, bepress.com/jqas/vol4/iss2/10 Hopkins WG, Marshall SW, Batterham AM, Hanin J (2009). Progressive statistics for studies in sports medicine and exercise science. Medicine and Science in Sports and Exercise 41, 3-12 Hopkins WG (2010). Linear models and effect magnitudes for research, clinical and practical applications. Sportscience 14, 49-57 James N, Mellalieu S, Jones N (2001). The development of position-specific performance indicators in professional rugby union. Journal of Sports Sciences 23, 63-72 Stair A, Neral J, Mizak D, Day A (2008). The factors affecting team performance in the NFL: does off-field conduct matter? Economics Bulletin 26, 1-9 White C, Berry S (2002). Tiered polychotomous regression: ranking NFL quarterbacks. The American Statistician 56, 10-21 Published August 2011 ©2011