I’ve been doing some research into the importance of depth in the NFL for quite a while now. I thought free agency and the monster contracts being handed out were a good time to share some of what I’ve found.

“Depth” is a hard thing to define. In some cases, like injury events, the important facet of depth is how skilled the reserve players at a given position are. Depth is also a term used to reference the top-heaviness of a team. Recently, the NFL has had a rash of offensively top-heavy teams, where upwards of 70% of the yardage becomes concentrated in two or three skill players.

In my study, I’ve been defining depth a bit differently. Specifically, I’ve been curious as to effects on win totals that can be detected as the distribution of salary allocation changes. In practice, this means looking at how equally distributed player salaries are. To begin, I assembled a database of player salaries from 2005 to 2013. As a basic measure of team salary distribution, I took the standard deviation of player cap numbers, excluding quarterbacks. In order to account for year to year discrepancies, I scaled these standard deviations by expressing them as a percentage of the salary cap for that year.

To illustrate what this looks like, consider two hypothetical teams. Let’s say we have an arbitrary salary cap of 50 dollars, and each team spends the entire cap. One team opts to devote a huge portion of the cap to a single superstar player, while the other team gives a few moderately large contracts out. Call them team superstar and team deep. To simplify, let’s act like each roster is only eight players. Here’s what it looks like:

Despite spending the exact same amount of total salary, Team Superstar’s distribution is essentially twice as disproportionate as Team Deep’s. This is the concept I’m capturing with my spending measure.

To test the effects of depth on wins, I ran a linear regression model with three independent variables: previous year wins, depth, and total salary spent as a share of the cap. If you aren’t familiar with how regression analysis works, it simply means we’re estimating an equation to see if we can predict how many games a team will win in a year just based on three things:

How many games they won the previous year. How evenly they divide their spending among players. How much total money they spend on player salaries.

Obviously, we aren’t expecting a particularly accurate equation. It would be crazy to think we could accurately forecast a team’s season with such simple variables. But we are interested in the relationships between winning games and these data points. To further augment the analysis, I split the regression analysis into two separate data sets, for teams with below and above average distributions.

By splitting the regressions up this way, we can evaluate the effect of depth both as teams add high-level players (moving towards the average level of depth) and as teams progress towards being top heavy (moving upward and away from average).

Here are the results of the regressions:

It’s important not to get caught up in interpreting the coefficients. There’s no real interpretation of a coefficient on the standard deviation of salaries as a share of the cap. It’s not happening. What is important is the sign of the coefficient. For the high depth teams (teams with very evenly distributed salaries) wins are expected to increase as they add more expensive players. But for teams with below average depth, the relationship becomes negative as the team progresses towards being top heavy.

Visually, here’s what the relationship looks like:

There’s obviously a lot of noise, but you can see that the benefits of adding big contracts certainly seem to taper off once the team has reached a saturation point. Also worth noting is that only one Super Bowl winner in the data set had a below average depth metric.

The conclusion here? We shouldn’t overreact to big name additions in free agency. Unlike in a sport like basketball, where two or three superstars can be a 40 win difference, football is a game that relies on the collective contributions of dozens of players. The data strongly suggests that teams suffer a decreasing return on investment as they become top heavy at non quarterback positions.