Accuracy of 2007 Preseason Predictions
For the next couple weeks, I'm going to be auditing the various statistics and systems I use to see what works, what doesn't. First up, the win total projections based on plugging in preseason stats into a linear regression system trained on regular season stats and win totals. The offensive stats used were those for projected starters only, and the defensive stats were total defensive stats. Only rushing and passing yards per play stats were used.
Across the board, the preseason predictions were OK as far as preseason predictions go, which is to say not very good. The average absolute error was 2.66 games, the difference between 7-9 and 10-6. It called Miami's 1-15 season but was 7+ games short of New England's perfect regular season. Randy Moss, however, did not play in preseason. Hmm... I wonder what that bodes for next season...
The correlation coefficient comes out a little better at 0.40641. So the preseason win projections matched up somewhat strongly with actual win totals. It predicted Cleveland to have a good season, as well as Tennessee and Washington. It, however, also predicted New Orleans to be the best team in the league this season at 14-2. Again, as far as these things go, it was normal, having a mix of dead-on predictions and mile-wide misses. Looking at it from a divisional, rather than a league-wide perspective, shows pretty much the same thing.Team Expected Wins Actual Wins AFC East BUF 4.8474 7 MIA 0.7279 1 NE 8.4262 16 NYJ 6.6077 4 AFC North BAL 6.0087 5 CIN 5.3328 7 CLE 8.3653 10 PIT 13.8218 10 AFC South HOU 7.1054 8 IND 9.7991 13 JAX 11.5104 11 TEN 8.7978 10 AFC West DEN 7.1972 7 KC 3.3213 4 OAK 9.4303 4 SD 9.2745 11 NFC East DAL 11.5798 13 NYG 4.0315 10 PHI 12.6319 8 WAS 9.7338 9 NFC North CHI 9.4675 7 DET 10.1605 7 GB 5.2574 13 MIN 7.8810 8 NFC South ATL 7.3336 4 CAR 8.8436 7 NO 14.4020 7 TB 3.5238 9 NFC West ARI 7.5798 8 STL 4.5614 3 SF 8.1366 5 SEA 11.1896 10
For 4 out of the 8 divisions, the projected win totals had a very strong correlation with regular season standings. The only complete miss was the NFC North, where the order was reversed. But with the exception of Oakland, the AFC West predictions were also very good. The NFC East and South predictions were essentially a wash with weak correlations. So only 3 of 8 divisions were complete failures. The p-values, however, show that the correlation coefficients have a higher-than-desired probability of being achievable with random data. The small sample size (n=4) for each division plays a large role in this.Team Corr. Coef. P-value Mean Abs. Err. AFC East 0.8070 0.1930 3.1516 AFC North 0.7424 0.2576 2.0331 AFC South 0.7025 0.2975 1.4520 AFC West 0.4754 0.5246 2.0079 NFC East -0.0104 0.9896 3.1886 NFC North -0.9553 0.0447 3.3724 NFC South -0.2303 0.7697 4.5139 NFC West 0.8829 0.1171 1.5770
I think the lesson to take away from this, however, is that preseason does have some meaning. Most studies into the meaning of preseason have looked at win-loss records, which any football researcher would know is not the most accurate reflection of team ability, especially with a sample size of four. Skill is skill and should be visible even when the effort isn't 100%. Filtering out the play of benchwarmers from the statistics is important, and with some more development, it looks like reasonably good predictions can be made with preseason stats.
