As we all hunker down with our spreadsheets in preparation for draft season, it’s time for my favorite yearly tradition - evaluating the projection systems! Forgive the somewhat abbreviated post in comparison to past years’ analyses, but trust that all the same meticulous research is here as always. The goals, as always, are to evaluate the best projection systems on a per-category basis for the purposes of fantasy baseball, and to determine the best possible mix of projections (separated into rate stats and playing time) as we look ahead to ‘19.

In this study, I'll focus on the most commonly used projections - the same ones that appear in the Big Board: Steamer, PECOTA, ZiPS, ATC, The Bat, Fangraphs Depth Charts, and Fangraphs Fans. The categories of interest are the typical 5x5 categories, HR/R/RBI/SB/AVG for hitters, and W/SO/SV/ERA/WHIP for pitchers. Since for fantasy purposes we only care about the relative projections made by each system (ie, we only need to know Trout is the best hitter in baseball, not exactly what his AVG will be), I'll primarily use R squared to evaluate how well the projections correlated to actual results, but I'll also be including RMSE to show the absolute error in each projection system. The most common fantasy leagues draft about 300 players, broken out into 180 hitters, 90 SP, and 30 RP, and so I've used the consensus top 300 players as determined by an average of the projection systems, and will only be evaluating the systems based on their projections of those 300 players. One final adjustment - hitters that didn't end up reaching 400 PA and pitchers that didn't reach 35 IP have been thrown out of the sample to reduce the effect of short playing-time outliers (typically from injury).

Hitters

First, a definition - the Big Board mix for hitters this year, which combines the systems to produce the best overall results, is:

Playing Time: 41% Fans, 32% ZiPS, 27% Steamer

Rate Stats: 58% ATC, 42% Steamer

Note that rate stats like AVG were also evaluated as part of the 'total' projections by using a playing-time weighted value indicated by an 'n' (e.g. "nAVG"). The SB (*) projections were evaluated on the basis of two separate populations and averaged: players who stole >5 bases, and those who stole 5 or less. Past analysis has shown that evaluating this as a single population gives undue credit for projecting the low-steal players, and not enough credit for accurately projecting high-steal players.