Accuracy of Projections from Yahoo, ESPN, CBS & numberFire

by dave
Nov 27, 2010

Have you ever wondered how good the player projections are at the big league management sites? A few weeks ago I wrote a guest post at the NY Times Fifth Down about the accuracy of these “computer” projections.

Since then, we’ve received many requests to add them into our weekly accuracy rankings, so we went ahead and made the update over Thanksgiving break (it’s ok; we needed a break from the stuffing and the hyper nephews anyway).

We’ve also received a surprising number of requests to track a newcomer to the space called NumberFire.  It’s a slick startup that uses a unique algorithm to produce – according to them – significantly more accurate results than Yahoo and ESPN’s projections. Care to see how accurate they all are? Read on to find out!

Let’s start with NumberFire. Here’s what they say on their homepage:

“You can go by Yahoo! or ESPN’s projections, but they’re wildly inaccurate. There’s no science, no reasoning behind them – just some fat guy and his opinion…Our crew of stat geeks and math nerds have a unique, custom algorithm that matches current situations to historical games, gives them a rating based on the similarities, and makes a smart, hyper-accurate prediction based on the historical results.”

In Week 11, NumberFire was the #39 expert out of the 44 that we scored. In Week 10, they were slightly better at #34. We’re fans of the site, so we were definitely surprised by these results. Given how aggressively they claim to be more accurate than who they refer to as “the twin behemoths of inaccuracy,” it was equally surprising to see them rank lower than Yahoo and ESPN in both weeks. You can view the weekly rankings by going to the Week Tab of our Accuracy page.

We need to take these results with a huge grain of salt though. First, we’ve only been tracking them since Week 10. We’re a firm believer that you need a lot of data over a significant period of time before you can call your accuracy assessments accurate. We definitely don’t think you should draw too many conclusions from two weeks of data! Second, and probably more important, how we assess accuracy is very different from how NumberFire does it (from what we can tell from published information).

They compare their projected fantasy points for each player to those provided by Yahoo and ESPN. The site with projected points that are closer to the player’s actual points is deemed to be more accurate. Here’s NumberFire’s Week 11 Test Case (Note that I’m not sure if their accuracy claims are based on just the handful of players shown here or something more comprehensive). Here’s an example using two players from their spreadsheet:

They projected Chad Ochocinco to have 10.39 points while Yahoo predicted that he’d have 6.43. Chad actually scored 8.8 points, so NumberFire was deemed to be more accurate because their projection was closer to the actual result. The same logic was used for Brandon Lloyd. NumberFire’s projection of 10.07 was closer to Lloyd’s actual score of 13.8 than Yahoo’s projection of 9.69. This is all pretty straightforward and logical.

The problem is, if you have Ochocinco and Lloyd on your team, you probably care more about which guy to start. NumberFire’s projections would suggest that you start Ochocinco (they predicted that he would score more points), while Yahoo’s projection would suggest that you start Lloyd. From this perspective, NumberFire got it wrong and your team would have scored 5 less points for following their advice (Lloyd’s 13.8 points minus Ochocinco’s 8.8 points).

Our accuracy methodology is about predicting “relative” fantasy performance between players. We believe most of us use projections and rankings to make lineup and roster decisions, so it’s the accuracy of these predictions that matter the most. An expert gets credit for correctly advising you on which player to start, and the credit we award is directly related to the difference in fantasy points (since some start/sit decisions are more costly than others).  In the previous example, our system would say Yahoo made the right call, not NumberFire.

Moving on to the Big Three, here’s how each site’s projections rank in overall accuracy through 11 weeks of the season, out of the 41 experts that we have season long data for:

Yahoo! Sports Projections – #16
CBS Sports Projections – #18
ESPN Projections – #37

It’s probably a good idea at this point to clarify what we mean by “site projections.” Using Yahoo as an example, we track six different “experts” that they have. This includes four human experts: Behrens, Evans, Pianowski and Funston. The “Staff Composite” rankings that we track are the average player rankings of these human experts that get published on Yahoo’s rankings page. The “Site Projections” that we’re talking about in this post are the player projections that you see in the Yahoo league management system (for example, on the Matchup page that shows whether you or your opponent is projected to win).

So where do these site projections come from? A company called AccuScore actually powers the projections that you find on several big league management sites. Here’s what they say on their site:

“For Fantasy Players, AccuScore’s Fantasy Scout data is the #1 Service in the industry and we are the only company that is and has ever been simultaneously paid by ESPN.com, Yahoo! and CBS Sports for fantasy data…AccuScore uses past performance statistics to describe how players perform under different environmental, matchup, and game-situation conditions. Using projected starting lineups AccuScore simulates each game of the season one play at a time.”

Yahoo’s projections come directly from AccuScore. On CBS, users have the option to use projections from AccuScore, one of their resident fantasy experts (Jamey Eisenberg and Dave Richard), or an Average of all three projections. Since the Average is the default setting, this is the version that we use for “CBS Projections.” It’s not as clear how ESPN’s projections incorporate AccuScore, if at all. So, even though all three big sites may be using AccuScore in some way, their accuracy rankings are different because each site’s player projections are different. Had we used the “Accuscore” option on CBS, their scores would have been identical to Yahoo’s.

We hope you found this background information helpful. We’ll continue to track these sites as we work to gather a much longer history of the accuracy of their fantasy advice. Also note that these are not the only sites that use some sort of computer-based algorithm to project and rank players. We already track a few others, such as WhatIfSports.

Our goal is to discover and promote the best fantasy experts – human or computer – so contact us if you know of a site that provides great weekly rankings or projections. If there’s a critical mass of folks that want the same info, we’ll gladly pull it together.

Thanks and good luck this weekend!



What's your take? Leave a comment

Fantasy Games
DRAFT photo
Get a FREE $3 Best Ball entry with your 1st deposit
FanDuel photo
Play for your share of $1m+
for FREE with first deposit
CBS Sports photo
Serious Fantasy Football
Get our Mobile App!

Enter your phone number below, and we'll text you a link to download the app.

1Le'Veon Bell (PIT)RB
2Todd Gurley (LAR)RB
3Antonio Brown (PIT)WR
4David Johnson (ARI)RB
5Ezekiel Elliott (DAL)RB
6DeAndre Hopkins (HOU)WR
7Odell Beckham Jr. (NYG)WR
8Leonard Fournette (JAC)RB
9Michael Thomas (NO)WR
10Kareem Hunt (KC)RB
 View All Rankings 
11Alvin Kamara (NO)RB
12Julio Jones (ATL)WR
13Melvin Gordon (LAC)RB
14Mike Evans (TB)WR
15A.J. Green (CIN)WR
16Dalvin Cook (MIN)RB
17LeSean McCoy (BUF)RB
18Devonta Freeman (ATL)RB
19Keenan Allen (LAC)WR
20Davante Adams (GB)WR
21Mark Ingram (NO)RB
22Jordan Howard (CHI)RB
23Brandin Cooks (NE)WR
24Tyreek Hill (KC)WR
25Derrick Henry (TEN)RB
26Rob Gronkowski (NE)TE
27Adam Thielen (MIN)WR
28Doug Baldwin (SEA)WR
29Christian McCaffrey (CAR)RB
30Carlos Hyde (SF)RB
1Mike Trout (LAA)CF
2Jose Altuve (HOU)2B
3Nolan Arenado (COL)3B
4Trea Turner (WSH)SS
5Paul Goldschmidt (ARI)1B
6Charlie Blackmon (COL)CF
7Bryce Harper (WSH)RF
8Mookie Betts (BOS)RF
9Clayton Kershaw (LAD)SP
10Giancarlo Stanton (NYY)RF
 View All Rankings 
11Kris Bryant (CHC)3B,RF
12Max Scherzer (WSH)SP
13Carlos Correa (HOU)SS
14Joey Votto (CIN)1B
15Corey Kluber (CLE)SP
16Chris Sale (BOS)SP
17Manny Machado (BAL)3B
18Freddie Freeman (ATL)1B,3B
19Anthony Rizzo (CHC)1B,2B
20Aaron Judge (NYY)RF
21Francisco Lindor (CLE)SS
22Jose Ramirez (CLE)2B,3B
23George Springer (HOU)CF,RF
24Cody Bellinger (LAD)1B,LF
25Josh Donaldson (TOR)3B
26Gary Sanchez (NYY)C
27Stephen Strasburg (WSH)SP
28J.D. Martinez (BOS)RF
29Jose Abreu (CWS)1B
30Madison Bumgarner (SF)SP
1Kevin Durant (GSW)SF,PF
2Giannis Antetokounmpo (MIL)SF,PF
3James Harden (HOU)PG,SG
4Stephen Curry (GSW)PG,SG
5Russell Westbrook (OKC)PG
6Karl-Anthony Towns (MIN)C
7Anthony Davis (NOR)PF,C
8Kawhi Leonard (SAS)SG,SF
9LeBron James (CLE)SF,PF
10Nikola Jokic (DEN)PF,C
 View All Rankings 
11John Wall (WAS)PG
12Chris Paul (HOU)PG
13Damian Lillard (POR)PG
14Jimmy Butler (MIN)SG,SF
15Rudy Gobert (UTH)C
16Kyrie Irving (BOS)PG,SG
17Hassan Whiteside (MIA)C,PF
18Myles Turner (IND)PF,C
19Paul George (OKC)SG,SF
20Kyle Lowry (TOR)PG
21Draymond Green (GSW)SF,PF
22Kemba Walker (CHA)PG
23CJ McCollum (POR)PG,SG
24Bradley Beal (WAS)SG
25Klay Thompson (GSW)SG,SF
26Marc Gasol (MEM)C
27Kevin Love (CLE)PF,C
28Paul Millsap (DEN)PF,C
29Joel Embiid (PHI)PF,C
30Eric Bledsoe (MIL)PG,SG