Accuracy of 2007 Preseason Predictions
For the next couple weeks, I'm going to be auditing the various statistics and systems I use to see what works, what doesn't. First up, the win total projections based on plugging in preseason stats into a linear regression system trained on regular season stats and win totals. The offensive stats used were those for projected starters only, and the defensive stats were total defensive stats. Only rushing and passing yards per play stats were used.
Across the board, the preseason predictions were OK as far as preseason predictions go, which is to say not very good. The average absolute error was 2.66 games, the difference between 7-9 and 10-6. It called Miami's 1-15 season but was 7+ games short of New England's perfect regular season. Randy Moss, however, did not play in preseason. Hmm... I wonder what that bodes for next season...
The correlation coefficient comes out a little better at 0.40641. So the preseason win projections matched up somewhat strongly with actual win totals. It predicted Cleveland to have a good season, as well as Tennessee and Washington. It, however, also predicted New Orleans to be the best team in the league this season at 14-2. Again, as far as these things go, it was normal, having a mix of dead-on predictions and mile-wide misses. Looking at it from a divisional, rather than a league-wide perspective, shows pretty much the same thing.
| Team | Expected Wins | Actual Wins | ||
| AFC East | ||||
| BUF | 4.8474 | 7 | ||
| MIA | 0.7279 | 1 | ||
| NE | 8.4262 | 16 | ||
| NYJ | 6.6077 | 4 | ||
| AFC North | ||||
| BAL | 6.0087 | 5 | ||
| CIN | 5.3328 | 7 | ||
| CLE | 8.3653 | 10 | ||
| PIT | 13.8218 | 10 | ||
| AFC South | ||||
| HOU | 7.1054 | 8 | ||
| IND | 9.7991 | 13 | ||
| JAX | 11.5104 | 11 | ||
| TEN | 8.7978 | 10 | ||
| AFC West | ||||
| DEN | 7.1972 | 7 | ||
| KC | 3.3213 | 4 | ||
| OAK | 9.4303 | 4 | ||
| SD | 9.2745 | 11 | ||
| NFC East | ||||
| DAL | 11.5798 | 13 | ||
| NYG | 4.0315 | 10 | ||
| PHI | 12.6319 | 8 | ||
| WAS | 9.7338 | 9 | ||
| NFC North | ||||
| CHI | 9.4675 | 7 | ||
| DET | 10.1605 | 7 | ||
| GB | 5.2574 | 13 | ||
| MIN | 7.8810 | 8 | ||
| NFC South | ||||
| ATL | 7.3336 | 4 | ||
| CAR | 8.8436 | 7 | ||
| NO | 14.4020 | 7 | ||
| TB | 3.5238 | 9 | ||
| NFC West | ||||
| ARI | 7.5798 | 8 | ||
| STL | 4.5614 | 3 | ||
| SF | 8.1366 | 5 | ||
| SEA | 11.1896 | 10 | ||
For 4 out of the 8 divisions, the projected win totals had a very strong correlation with regular season standings. The only complete miss was the NFC North, where the order was reversed. But with the exception of Oakland, the AFC West predictions were also very good. The NFC East and South predictions were essentially a wash with weak correlations. So only 3 of 8 divisions were complete failures. The p-values, however, show that the correlation coefficients have a higher-than-desired probability of being achievable with random data. The small sample size (n=4) for each division plays a large role in this.
| Team | Corr. Coef. | P-value | Mean Abs. Err. |
| AFC East | 0.8070 | 0.1930 | 3.1516 |
| AFC North | 0.7424 | 0.2576 | 2.0331 |
| AFC South | 0.7025 | 0.2975 | 1.4520 |
| AFC West | 0.4754 | 0.5246 | 2.0079 |
| NFC East | -0.0104 | 0.9896 | 3.1886 |
| NFC North | -0.9553 | 0.0447 | 3.3724 |
| NFC South | -0.2303 | 0.7697 | 4.5139 |
| NFC West | 0.8829 | 0.1171 | 1.5770 |
I think the lesson to take away from this, however, is that preseason does have some meaning. Most studies into the meaning of preseason have looked at win-loss records, which any football researcher would know is not the most accurate reflection of team ability, especially with a sample size of four. Skill is skill and should be visible even when the effort isn't 100%. Filtering out the play of benchwarmers from the statistics is important, and with some more development, it looks like reasonably good predictions can be made with preseason stats.
Read More......
