Most Accurate Fantasy Baseball Projections (2018)

Projecting the performance of baseball players is a very mature science. Projections continue to improve every year, and one factor is the plethora of competition. With some projection systems honing their craft for more than a decade, and new competitors joining every year, the level of competition – and quality of the advice – couldn’t be higher. Our goal here at FantasyPros is to help you win your league, and step 1 is choosing the right projections to inform your decisions.

Methodology

For our accuracy competition, we evaluated 17 different projection systems. There are several ways to evaluate the accuracy of projections. One option is to find a single composite stat like wOBA or WAR that encompasses a player’s entire game and evaluate that. Since we are primarily concerned with fantasy, we’re using a system that isolates the ten standard scoring categories in a 5×5 league (R, HR, RBI, SB and AVG for hitters, W, SV, K, ERA and WHIP for pitchers) as well as playing time (at-bats and innings pitched). For our 5×5 categories, we treated each as a rate stat so as to isolate playing time and talent projections. For example, we evaluate HR/AB rather than just HR. That way a system gets good marks for correctly predicting a player’s talent as a home run hitter, even if their total was off due to a bad playing time projection. Last year, Mike Trout hit 39 HRs on 471 ABs, which is a rate of 8.3%. Suppose a system projected he would hit 48 HRs on 575 ABs. That AB projection is too high, but the projected home run rate is right on target at 8.3%. This system would get a bad accuracy score for at-bats, but would be considered very accurate for home runs because it correctly predicted how effective Trout would be at hitting homers.

To calculate our accuracy metric, we first look at the errors of each projection system for a player, and count the mean and standard deviation. Our 16 systems missed on Trout’s at-bats projection by an average of 60, with a standard deviation of about 18. For each projection system, we divide by the standard deviation and subtract the mean to get a z-score. For example, numberFire projected Trout to have 505 at-bats. Compared to an actual outcome of 471, that’s a miss of just 34, which is 26 better than the average miss of 60. We divide by the standard deviation of 18 to find that numberFire got a z-score of about -1.5 for Trout’s AB projection. A negative number is good here – this means numberFire was about one and a half standard deviations better than the average projection system when it comes to predicting Trout’s at-bats. To get numberFire’s overall score for at-bat projections, we simply average these z-scores across all players.

To get overall accuracy for a projection system, we sum up their average z-scores across each stat category, and then rank the projection systems according to that metric (remember, a lower z-score is better, and a score of 0 is average).

Results

Without further ado, here are the most accurate fantasy baseball projections for 2018:

2018 Fantasy Baseball Projection Accuracy
Projection System Hitters Pitchers Overall
Score Rank Score Rank Score Rank
Zeile Consensus Projections – FantasyPros -0.96 1 -1.027 1 -1.988 1
ATC Projections – FanGraphs -0.926 2 -0.748 5 -1.674 2
Depth Charts Projections – FanGraphs -0.610 5 -0.766 3 -1.376 3
Derek Carty – The BAT -0.702 4 -0.583 6 -1.285 4
Site Projections – Razzball -0.347 9 -0.85 2 -1.197 5
Steamer Projections – Steamer Blog -0.440 7 -0.75 4 -1.190 6
Jason Johnson – Understanding Fantasy Baseball -0.505 6 -0.366 7 -0.871 7
Site Projections – numberFire -0.730 3 0.341 14 -0.389 8
Site Projections – ESPN -0.061 12 -0.026 8 -0.087 9
Clay Davenport -0.023 13 0.039 9 0.016 10
BaseballHQ Projections – Baseball HQ -0.355 8 0.375 15 0.020 11
Site Projections – RotoChamp -0.108 11 0.136 11 0.028 12
ZiPS Projections – Baseball Think Factory -0.130 10 0.33 13 0.200 13
Scutaro Projections – FantasyPros 1.056 15 0.134 10 1.190 14
FANS Projections – FanGraphs 1.175 16 0.306 12 1.481 15
Site Projections – TG Fantasy Baseball 0.505 14 1.11 16 1.615 16
Site Projections – CBS Sports 1.882 17 1.349 17 3.231 17

*Full disclosure – the writer of this article is also the creator of the Scutaro projections.
Here’s the category breakdown for hitters:

Hitter Projection Accuracy
Projection System AB R HR RBI SB AVG Overall
Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Score Rank
Zeile Consensus Projections – FantasyPros -0.280 1 -0.201 2 -0.156 3 -0.199 2 -0.193 1 0.068 13 -0.961 1
ATC Projections – FanGraphs -0.205 4 -0.223 1 -0.146 5 -0.155 3 -0.148 4 -0.048 T-9 -0.926 2
Site Projections – numberFire -0.192 5 -0.185 3 -0.048 10 -0.132 5 0.018 13 -0.192 2 -0.730 3
Derek Carty – The BAT -0.010 10 -0.031 7 -0.045 11 -0.234 1 -0.171 T-2 -0.210 1 -0.702 4
Depth Charts Projections – FanGraphs 0.006 11 -0.155 5 -0.150 4 -0.087 8 -0.062 7 -0.162 3 -0.610 5
Jason Johnson – Understanding Fantasy Baseball 0.031 12 -0.007 10 -0.079 7 -0.144 4 -0.171 T-2 -0.135 4 -0.505 6
Steamer Projections – Steamer Blog -0.213 3 0.001 11 -0.075 8 -0.122 6 0.096 15 -0.127 5 -0.440 7
BaseballHQ Projections – Baseball HQ 0.037 13 -0.117 6 -0.068 9 -0.074 9 -0.078 6 -0.056 8 -0.355 8
Site Projections – Razzball -0.070 6 -0.025 8 -0.129 6 -0.109 7 0.108 16 -0.121 T-6 -0.347 9
ZiPS Projections – Baseball Think Factory 0.371 17 -0.170 4 -0.162 2 0.041 13 -0.089 5 -0.121 T-6 -0.130 10
Site Projections – RotoChamp -0.214 2 0.043 13 0.039 13 0.012 12 -0.037 9 0.050 12 -0.108 11
Site Projections – ESPN -0.054 7 -0.011 9 -0.030 12 -0.034 11 0.078 14 -0.010 11 -0.061 12
Clay Davenport -0.047 8 0.031 12 0.148 16 -0.055 10 -0.052 8 -0.048 T-9 -0.023 13
Site Projections – TG Fantasy Baseball -0.024 9 0.106 14 0.139 15 0.068 14 0.015 12 0.202 15 0.505 14
Scutaro Projections – FantasyPros 0.289 16 0.138 15 0.097 14 0.275 15 0.166 17 0.092 14 1.056 15
FANS Projections – FanGraphs 0.165 14 0.438 17 -0.166 1 0.444 17 -0.013 10 0.308 16 1.175 16
Site Projections – CBS Sports 0.177 15 0.246 16 0.448 17 0.399 16 0.011 11 0.601 17 1.882 17

And for pitchers:

Pitcher Projection Accuracy
Projection System IP W SV K ERA WHIP Overall
Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Z-Score Rank Score Rank
Zeile Consensus Projections – FantasyPros -0.274 1 -0.196 4 -0.091 5 -0.190 1 -0.138 2 -0.137 6 -1.027 1
Site Projections – Razzball -0.130 4 -0.218 3 -0.112 T-1 -0.133 5 -0.102 5 -0.153 3 -0.850 2
Depth Charts Projections – FanGraphs -0.116 6 -0.079 10 -0.088 6 -0.152 2 -0.175 1 -0.155 2 -0.766 3
Steamer Projections – Steamer Blog -0.086 7 -0.223 T-1 -0.057 7 -0.136 4 -0.100 6 -0.147 4 -0.750 4
ATC Projections – FanGraphs -0.138 3 -0.107 8 -0.102 3 -0.138 3 -0.105 4 -0.158 1 -0.748 5
Derek Carty – The BAT -0.120 5 -0.172 6 0.000 T-13 -0.046 8 -0.113 3 -0.131 7 -0.583 6
Jason Johnson – Understanding Fantasy Baseball 0.160 16 -0.156 7 -0.097 4 -0.072 6 -0.096 7 -0.104 8 -0.366 7
Site Projections – ESPN 0.069 11 -0.223 T-1 -0.054 9 0.162 15 0.023 13 -0.002 10 -0.026 8
Clay Davenport -0.028 8 -0.024 11 -0.055 8 -0.025 10 0.079 14 0.092 13 0.039 9
Scutaro Projections – FantasyPros 0.117 15 0.315 16 -0.112 T-1 -0.030 9 -0.012 9 -0.144 5 0.134 10
Site Projections – RotoChamp -0.178 2 -0.181 5 0.471 17 -0.053 7 0.009 11 0.066 12 0.136 11
FANS Projections – FanGraphs 0.023 10 0.011 13 -0.028 10 0.165 16 0.014 12 0.121 T-14 0.306 12
ZiPS Projections – Baseball Think Factory 0.278 17 0.221 14 0.000 T-13 -0.004 11 -0.089 8 -0.077 9 0.330 13
Site Projections – numberFire -0.008 9 -0.095 9 0.046 15 0.394 17 0.005 10 0.000 11 0.341 14
BaseballHQ Projections – Baseball HQ 0.088 12 0.005 12 -0.027 11 0.082 13 0.107 15 0.121 T-14 0.375 15
Site Projections – TG Fantasy Baseball 0.090 13 0.251 15 0.053 16 0.092 14 0.279 17 0.344 16 1.110 16
Site Projections – CBS Sports 0.098 14 0.611 17 -0.002 12 0.038 12 0.244 16 0.359 17 1.349 17

 

Analysis

These results are extremely encouraging if you are a fan of the wisdom of the crowd. The top three projection systems are each based on a consensus, though they represent three very different philosophies on how to go about that.

  • The FantasyPros Zeile Consensus Projections, named for Todd Zeile, are a straight average of as many freely available projection systems as we can find. In 2018 we used 11 different projection sources: site projections from ESPN, CBS, Razzball, numberFire, RotoChamp and TG Fantasy Baseball, as well as my own Scutaro projections, ZiPS, Steamer, and projections from Clay Davenport and Jason Johnson. No weight was given to the past accuracy of these systems – each contributed equally to the final product.
  • The Average Total Cost (ATC) projection system, as described in this article, was created by Ariel Cohen and Reuven Guy and is hosted by FanGraphs. It is a consensus of ZiPS, Steamer, FANS and other projection systems (the full list is not specified). Unlike the Zeile Consensus, ATC doesn’t treat each projection system equally. Rather, each projected stat is a weighted average of projections from each system, with the weight determined by the past accuracy. This means each stat has a different set of weights, based on how accurate each system is for that particular stat. It also includes some manual adjustments by the author for playing time projections.
  • The FanGraphs Depth Charts projections, as described in this article, project playing time manually based on the opinions of Fangraphs’ staff of baseball writers. For performance stats, they are a straight average like Zeile, but unlike Zeile they chose to include just two trusted sources: ZiPS and Steamer.

The power of aggregating projections is clear in the results. Both Zeile and ATC ranked in the top 5 for all but 2 of the 12 stat categories: Zeile ranked 6th in WHIP and 13th in batting average, while ATC was 8th in wins and tied for 9th in batting average. Zeile was the most accurate system for both playing time metrics (at-bats and innings pitched) as well as stolen bases and strikeouts. ATC ranked in the top 5 for both playing time metrics, and was the #1 system for both runs scored and WHIP. The only other system to rank in the top 5 for both playing time categories was the RotoChamp site projections, which ranked 2nd in both metrics, beating every other non-consensus source. While Zeile’s rank as the most accurate system is a win for the gather-as-much-data-as-possible strategy, the race was tight for these top 3. The fact that Depth Charts were able outperform any single projection system by aggregating just two (very respected) sources is a great indicator that aggregating multiple sources is a good approach.

Congratulations are also due to Derek Carty at THE BAT, Razzball’s site projections, Steamer and Jason Johnson’s projections. Those were the top four of the non-consensus projection systems, and the only ones to place in the top-10 overall for both hitting and pitching.

Another thing we can glean from the z-scores is how remarkably tight the race is. In every stat category, the best projection system had a score at or above -.280, meaning the winners weren’t huge outliers. Even the best system can only be expected to beat an average system by less than a third of a standard deviation.

This wraps up our analysis of the accuracy of 2018 fantasy baseball preseason projections. For a similar analysis of expert rankings, check out our Most Accurate Fantasy Baseball Experts for 2018.

Jacob Herlin is a Senior Data Analyst for FantasyPros. You can find him on twitter @jacoblawherlin.