This is the 4th in a series of articles I am writing to analyze some common NFL statistics, focusing on how much value they have relative to team wins. I want to acknowledge the work of Brian Burke, Chase Stuart and even our own Matt Grecco, who inspired this analysis and whose methodologies I have leveraged, as well as Pro Football Reference, Armchair Analysis and NFL.com as the sources of my data.

I know what you're thinking: Did he just reference 2 stats in the title?

Dear god, twice as much rambling monotony with double the stab-me-in-the-eye analysis. Fear not, for my promise to you is to deliver only the standard amount of mind-numbing scrutiny.

I say this because like any 2 items on a Taco Bell menu, 3 and outs and 3rd down efficiency are really the same thing . . . except different . . . and opposite . . . and re-scaled. Savvy?

But I’m getting ahead of myself. First, I must confess that I don't really hate 3rd down efficiency as a stat. It is an attempt to measure the ability of a team to move the ball downfield and that's a good thing.

My issue with it lies in its reach. Since it only includes 3rd downs, which is just a fraction of all plays, it is far too limited. It's a lazy stat. It has good intentions but no follow through. It's an employee with one foot out the door phoning in some short-timer half-assery.

Let's define terms. 3rd down efficiency is the percent of 3rd downs that a team is able to convert into a first down. Yeah, duh. OK, but did you know that TDs are scored as a first down and are included as a conversion? Or that first downs converted via penalty, while officially scored as a first down, don't count in efficieny stats? You knew that? Oh.

Three and outs (3&O) are merely a special case of 3rd down efficiency (3DE). Whereas, 3DE is a percentage measure of successful conversions, 3&O's are a simple count (different) of failed conversions (opposite) for the initial set of downs in a drive (re-scaled).

As such, the 3&O measure is already accounted for within 3DE and it would make no sense to gauge it separately unless it contains some special information (it doesn’t) that is otherwise diluted within 3DE (it isn't).

THE RANKINGS

The vertical bars represent the 3DE rates for each team with corresponding values on the left y-axis. The red line is the average number of 3&O's (1) per game for each team with values on the right axis.

In the luck era, the Colts have compiled a 40.9% 3DE rate which is good enough for 9th place.

Notice that as the 3DE decreases the 3&O's generally increase, which makes sense because the fewer 3rd downs you convert the more 3 and outs you will likely have. Before doing any math, it is apparent that there will be a good correlation of these metrics to wins as some of the most prolific offenses in the NFL over the last few years have the highest 3DE numbers and lowest 3&O's (and vice versa for teams that have poor offenses).

Flipping the script, the Colts defense registers a 37.9% 3DE against which is also a 9th place finish. Wait, what? That can't be right.

Hold on a minute . . . sharpen pencil . . . total conversions . . . carry the one . . . gazinta third downs . . . holy crap! T he Colts defense really is in the top ten in preventing teams from converting third downs. That certainly is not what I had expected. I wonder if that is significant in some way? #plotpoint #foreshadowing

THE NUMBERS

As always, I will measure the value of a stat by comparing its correlation to wins (2).

(Darker shades = 16 game correlations, lighter tints = 8 game predictive correlations)

3&O is a bad stat. It has a low explanatory correlation and for predicting wins, -0.23 is about the same level as the ever crappy total passing yards (0.22). This means we can stop talking about 3&O from this point forward, as not only is it inferior to, but also wholly encompassed by, 3DE.

3DE, however, fares well. The explanatory correlation of 0.51 is good and the predictive correlation of 0.31 exceeds the threshold for an acceptable stat. So, not a bad stat . . . still a slacker stat, but not bad.

I do get why the talking heads focus on 3rd down though. It's the down that teams are "supposed" to convert. The league average conversion rate of 39% is much higher than 1st or 2nd downs (19%, 30%). It's also the last chance (usually) for the offense to keep a series alive and the last line of defense for . . . well, the defense.

But it's really a false narrative. People always seem to place undue importance on the last link in a chain of events, as if all glory and blame is to be found there. It's that mystical point where "clutch" diverges from "choke"; barstool wisdom from a pair of wrap-around Oakley's and a backwards baseball cap.

The fact is that of all run and pass plays, 80% are NOT 3rd downs and 3rd down conversions are NOT the most common down where conversions occur.

Both 1st and 2nd down conversions are more prevalent and when combined overshadow third down conversions. 3DE is a stat that ignores 71% of the very thing it is attempting to measure. Indolence, I say.

ALTERNATIVE STATS

It would make far more sense to measure the conversion rate for all downs and I can think of no better game to illustrate why this is so, than the 2009 Miami-Indy game.

That Monday night, the 3rd down conversions were recorded as follows:

Team Down Conv. Downs Conv % Colts 3rd 3 7 43% Dolphins 3rd 15 21 71%

The Colts 43% 3DE is actually above league average, but nowhere near the unbelievable 71% that Miami posted. Looking at that, you would think the Colts had been curb-stomped. But when adding in other downs, a truer story emerges:

Team Down Conv. Downs Conv % Colts 1st 8 18 44% 2nd 3 10 30% 3rd 3 7 43% 4th 0 0 Total 14 35 40% Dolphins 1st 4 33 12% 2nd 7 29 24% 3rd 15 21 71% 4th 1 1 100% Total 27 84 32%

The Colts did so well on 1st and 2nd down, that they actually bested Miami in overall conversion rate 40% to 32%, which was mirrored in the final score: Colts 27 MIA 23.

Expanding this example to all games since 2000, the efficiency for all downs (ADE) is far more correlated to wins than just 3rd downs.

(Darker shades = 16 game correlations, lighter tints = 8 game predictive correlations)

This is a robust result. The predictive correlation of 0.36 is the highest of any of the offensive stats that I have reviewed so far.

This shows that ignoring 1st and 2nd downs often misrepresents the effectiveness of both the offense or the defense **cough**Colts**cough**.

THE UNNECESSARY DETAIL

But, before declaring total victory, if I have learned anything about measuring the effectiveness of stats, it is that the 4th quarter often wreaks havoc with the numbers.

To assess this for ADE, I tracked it separately for the eventual game losers and winners.

For 3 quarters, the wining team has a higher ADE than the losing team which makes complete sense. But in the 4th quarter that is turned on its head.

As games wind down, teams that are ahead stop being aggressive and their first down conversion rates decrease. Conversely, teams that are behind often face defenses playing soft which bolster their conversion rates.

The result is that for 3 quarters the correlation of conversion rate to wins is strong, but then, in the 4th quarter, it plummets.

So there is room for improvement.

In previous analysis, this break in the trend was usually accompanied by a switch from passing to running by the leading team. To see if that is true here, I split ADE into running and passing conversion rates defined as follows:

Passing Down Efficiency (PDE) = Passing First Downs / (Attempts + Sacks)

Rushing Down Efficiency (RDE) = Rushing First Downs / (Carries - Kneeldowns)

I then checked the correlation of those stats to wins by quarter.

You can see that the while the passing correlations remains high and fairly stable, rushing is far less correlated and significantly offset by 4th quarter strategy changes. Therefore, removing the rushing information and just focusing on passing should yield a better overall metric.

(Darker shades = 16 game correlations, lighter tints = 8 game predictive correlations)

Now that’s what I call high quality PDE.

One advantage of this stat is that it can easily be calculated from most box score data, whereas ADE requires knowing the number of first and second downs which isn’t always available without some digging.

Also since this is limited to passing only, it makes for an excellent QB metric; even better than YPA or other commonly touted QB measures (which I will review in future articles).

RE-RANKINGS

Re-ranking offenses from 2012 - 2016 using PDE, the Colts fall 5 spots to 14th. If this ranking seems low to you then here are some points to consider:

This is a team ranking, not a QB ranking. This a stew of Luck, Hasselbeck, a portion of Tolzien and even a dash of Clipboard Jesus. In straight QB rankings, Luck does somewhat better than the team rankings.

This lines up very well with other team offensive rankings like DVOA

This includes 2015, where describing Luck as bad is somewhat generous. When looking at just last year, Luck comes in 5th.

What about the defense?

23rd place. THERE's the Colt's defense we have all come to know and . . . love?

SUMMARY

1) 3 and outs are a stupid metric. Purge it from your lexicon (and add lexicon).

2) Third down efficiency is OK, but it ignores too much of the game to truly be valuable. Not much use for individual games.

3) First down conversion efficiency for all downs or just passing downs are both far superior metrics.

Previous Analysis - Passing Yards

Previous Analysis - TD/INT Ratio

Previous Analysis - Time of Possession

FOOTNOTES:

1) I have defined a 3 and out as any drive that is a single series of downs that gives the ball back without an offensive score. Most of these are standard three downs and a punt, but some are turnovers, safeties, failed 4th down conv etc.