UC-Irvine statistician advocates boycotting the BCS

Slate’s Bill James has a great article on the failures of the BCS system. Most of James’s points are based on a 2006 paper (requires login) in the Journal of Quantitative Analysis in Sports by UC-Irvine’s Hal S. Stern.

I couldn’t access Stern’s original article, but James summarizes his main points in the Slate piece:

The problems with the BCS are:

1. That there is a profound lack of conceptual clarity about the goals of the method;
   
2. That there is no genuine interest here in using statistical analysis to figure out how the teams compare with one another. The real purpose is to create some gobbledygook math to endorse the coaches' and sportswriters' vote;
   
3. That the ground rules of the calculations are irrational and prevent the statisticians from making any meaningful contribution; and
   
4. That the existence of this system has the purpose of justifying a few rich conferences in hijacking the search for a national title, avoiding a postseason tournament that would be preferred by the overwhelming majority of fans.

The excellent play of the teams at the top of the polls this year has illustrated the need for a college football playoff. Despite the legitimate cases for USC, Texas, and Utah to the national championship, it will be decided by two other teams (who also have legitimate claims on the title) in a one-off game that will settle nothing.

James does a nice job elaborating on Stern’s objections, as in this comment relating to 1.:

There are several things that a ranking system could do. It could rank teams based on their accomplishments over the course of the season—whom they played and whom they beat—or it could rank them based on the probability that they would win against a given opponent. It could rank teams based on how they have played over the course of the season, including perhaps in some early-season games against teams that were not quite sure who their quarterback was, or it could rank them based on how strong they are at the end of the season. It could rank the teams based on consistency, or it could rank them based on dominance.

Which of these is the goal of the BCS system?

Nobody has any idea. It's never been debated. There is a perception among the people who are in charge of this monkey that if you just turn the rankings over to a computer, the computer will figure those things out. The reality is that it can't. It is very difficult to objectively measure anything if you don't know what it is you are measuring.

Update: it appears that the list of objections to the BCS come from James, not from Stern, as I originally stated. -John