How Often Does the Better Team Win?

In the National Football Leagues, no win can be guaranteed. One play can have a large impact on any game. It's part luck, but it's part happenstance too. It takes skill to get an interception, but it's the happenstance of what play is being run, and thus where the players are, that largely dictate if the interception is returned for a touchdown or not. If the interception occurred in the red zone and was returned for a touchdown, that's at least a 10 point swing. So a play with a 2.96% probability of occurring (league average interception rate) has an inordinately large impact on the game. Brian Burke's blog had a very good piece on how much luck is involved with winning and concluded that half of winning games is luck (52.5% to be exact) so the better team is going to win around 74% of the time. I was curious to see how it worked out in reality and further validate my assertion that interconference games have more inherent variance and less predictability than intraconference games. To decide the better team, I simply used total DVOA from Football Outsiders (1996-2006). Please note that the DVOA stats are over the entire regular season and postseason, so they are retrodictive, not predictive. The predictive ability of total DVOA is not as good. If DVOA had a predictive accuracy of 70%, I wouldn't be working as hard on a prediction system.



Average result means how many more points the better team scores on average. Average margin of victory means how many more points the winning team scores on average. The averages are by year, so the total proportion of games won by the better team, etc. will vary slightly from the numbers listed here. Part of the original study on interconference study was to see how much year-to-year variance there was in the outcomes of those games.

	Interconference			Interdivision
	Better Team Win %	Avg. Result	Avg. Margin of Victory	Better Team Win %	Avg. Result	Avg. Margin of Victory
Mean, 1996-2001	0.68056	6.9222	12.228	0.66446	5.694	11.042
Std. Dev, 1996-2001	0.079873	2.5031	0.99776	0.047733	1.579	0.75167
Mean, 2002-6	0.65	6.9313	12.213	0.68958	6.5396	11.169
Std. Dev, 2002-6	0.045015	1.1537	0.89953	0.048524	1.4997	0.92161

	Intradivision			All Games
	Better Team Win %	Avg. Result	Avg. Margin of Victory	Better Team Win %	Avg. Result	Avg. Margin of Victory
Mean, 1996-2001	0.6906	6.4449	11.232	0.68096	6.3569	11.42
Std. Dev, 1996-2001	0.047567	1.1109	0.70574	0.02639	1.0343	0.24626
Mean, 2002-6	0.73333	7.0729	11.356	0.69609	6.8375	11.5
Std. Dev, 2002-6	0.050281	1.089	0.53004	0.027538	0.64852	0.31041

So with the 20/20 hindsight of each entire season, the better team has wins about 69% of games, close to the 74% reported in Brian's blog, which was based on 2002-6. When looking at 2002-6 intradivision games, he was pretty much dead on. 73.333% vs. 74%. Most of the discrepancy can be traced back to interconference games. The divisional realignment in 2002 reduced year-to-year variance in the percentage of games won by the better team, but it's also reduced the average percentage from 68% to 65%. It's interesting that the average margin of victory is larger in interconference games than in the other types, but I'm not sure what that means.

I have two ideas on possible reasons why fewer interconference games are won by the better team. First, maybe coaches have more problems adapting strategy to opponents they don't see as often. An interconference matchup occurs only once every four years now (before, some matchups were much more common than others). Coaches have only a week to prepare for games, so they can only learn so much about a team's strengths and weaknesses. Obviously, the more time they have to study opponents, the more they will learn about them. So every time the interconference matchup comes up, the coach probably has to throw out a good deal of what he learned the last time. With intradivision matchups, you see the opponent twice a year and can re-use knowledge gained from previous matchups. Second, maybe stats should be adjusted for conference quality in addition to specific opponent quality like in baseball. I'm not sure this would work, given that the rules in both conferences are the same, unlike in baseball. Given that 75% of the season is intraconference, though, perhaps it's slightly inaccurate to judge a team based on the whole league, rather than their specific conference, when trying to predict an intraconference game. I've toyed with implementing this idea and might pursue it sometime in the near future.

Read More......

Refining the Model Episode II - Attack of the Current Inputs

To bring things up to speed: the following table lists the current subset of possible inputs that have the best correlation with the final score margin and provide the best predictions. If you are a careful enough reader, you'll notice some new inputs, which I'll explain in a second. All of the inputs are expressed in terms of value over league average, which is a percentage.

Abbreviation key:
H = Home, A = Away, O = Offense, D = Defense, M = Made, A = Allowed, G = Given, T = Taken
R = Rush, P = Pass, SR = Sack Rate, 3C = Third down conversion rate, IR = Interception rate, FR = Fumble rate
v = versus (Home VOLA - Away VOLA)
U = Unadjusted, A = Adjusted for opponent quality (relative to league average)

Unadj/Adj	Input	Correlation with Margin
A	HROvARD	0.12565
A	AROvHRD	-0.070102
A	HPOvAPD	0.20396
A	APOvHPD	-0.17716
U	HSRMvASRA	0.12016
U	ASRMvHSRA	-0.12116
U	H3CMvA3CA	0.17253
U	A3CMvH3CA	-0.12753
U	HIRGvAIRT	0.076802
U	AIRGvHIRT	-0.13572
U	HFRGvAFRT	0.11939
U	AFRGvHFRT	-0.10935

As you can see, the passing stats are now more highly correlated with the margin than the running stats. NFL Stats pointed out that yards per attempt statistics are more significant than yards per game stats. Also, a yards-per-pass attempt stat should include the yards lost in sacks. I'll get to some of the correlations of my inputs with win totals at the end of the post, as they back up those assertions.

Without getting into gross detail, using VOLA instead of raw stats bumped up the correlation coefficients slightly but consistently, and adjusting for opponent quality significantly increased the correlation coefficients for rushing and passing inputs. It's interesting to note that the home team's quality is more important (in terms of the correlation) than that of the away team. In the turnover stats, the home team's ability to pick off passes is more important than their own QB's ability to not throw interceptions. Similarly, their passing game and rushing game is more important than those of the away team. This could be another effect of teams simply performing better at home and another justification for adjusting stats to account for home field advantage. That article will come some time next week.

Since May, I've made the following changes to the data set:

• No more punt return data. Too lowly correlated.
  
• Tried kickoff return data. Same problem.
  
• Penalty first downs given to opponent and penalty yards lost. Per game. Same problem.
  
• Third down conversion data. Being able to sustain drives by converting third downs is, of course, important. And 3rd down attempts are frequent enough to justify adding an input, unlike fourth downs attempts.
  
• Rushing stats are based on yards per carry. Adjust for opponent in similar fashion to sack rates (adjusting rate/average based on league average rather than totals).
  
• Passing stats are now yards per attempt but yards lost on sacks are no longer added back to totals.
  
• Instead of a broad turnover ratio, I'm using interception rates and fumble rates. As it turns out, a stat of the combined VOLAs of the inputs listed in the table above has a .13 correlation coefficient, about as high as the turnover ratio. Like sack rates, it makes more sense to judge a team by how often they throw picks rather than how many they throw.
  

Correlation Coefficient of Year-End Stats with Season Win Totals
Stat	Unadj Raw	Unadj VOLA	Adj Raw	Adj VOLA
RO	0.2052	0.21076	0.20991	0.21529
RD	-0.13296	0.1418	-0.11469	0.12303
PO	0.5875	0.59369	0.61753	0.62381
PD	0.12777	-0.12764	-0.44051	0.44867
SRM	0.29453	0.30683	0.25853	0.27186
SRA	-0.35812	0.36049	-0.33423	0.33633
PR	0.14976	0.15024	0.12454	0.12407
PC	-0.10693	0.10282	-0.10179	0.09751
KR	0.080139	0.082233	0.029962	0.030679
KC	-0.051189	0.050584	-0.052803	0.053027
3CM	0.4948	0.50087	0.47661	0.48294
3CA	-0.33792	0.34373	-0.27612	0.2809
PFD	-0.090883	0.090432	-0.061634	0.060372
PY	-0.11465	0.1219	-0.10817	0.11399
IRG	-0.38033	0.38235	-0.36197	0.36454
IRT	0.34652	0.3513	0.28993	0.29535
FRG	-0.4307	0.43468	-0.37416	0.37877
FRT	0.34938	0.35141	0.32798	0.33012

PR = Punt Return, PC = Punt Coverage (yards per punt return), KR = Kick Return, KC = Kick Coverage (yards per kickoff), PFD = Penalty First Downs, PY = Penalty Yards

Using unadjusted sack, punt, kick, penalty, and turnover data along with adjusted rush, pass, and third down conversion data, the model has an average R² of 0.7995 and average mean absolute error of 1.38 wins. The predicted win totals have a correlation coefficient of 0.83151 with the actual win totals.


Where to Go from Here/A Preview of Upcoming Work

• Better special teams stat. I was thinking of using average starting field position after punts and kickoffs, which I can get from NFL.com's game books.
  
• Try a penalty rate stat. This would have to include all defensive plays as well. How often does a team make a mental mistake and how costly is it overall?
  
• Try to adjust stats to account for home field advantage.
  
• Try to adjust stats for conference.
  
• Retry climate variables. Perhaps a ternary variable for each climate matchup. 0=N/A. 1=Applicable. During weeks 1-8. 2=Applicable. During weeks 9-17.

Numbers corrected on 7/12/2007 after finding errors in some box scores.

Read More......

Refining the Prediction Model Part I - Linear vs. Logistic Regression

Recently, I came across NFL Stats, a blog similarly focused on predicting football games. Somehow, Brian, the creator, was able to achieve 63% accuracy on 2006. So I'm going to be playing catch-up for a while. Let's take a look at what separates are two systems, won't we? The biggest difference between our two systems is that he uses logistic regression for simple win/loss predictions rather than linear regression. Logistic regression tries to fit a probability distribution function between two or more classes, so the output is discrete rather than continuous.

Plugging my current dataset (as will be described in the next article) into a logistic regression model, I achieve 61.789% accuracy on average from 1997-2006, a slight uptick compared to linear regression (61.547%). The original idea behind using linear regression was that the better a team was than their opponent, the higher the margin would be in their favor. In logistic regression, we'd expect the predicted probability of winning to increase proportionally to the quality gap between two teams. So as it turns out, the predicted probabilities of winning are almost as highly correlated with the final score margin (.28357) as the predicted margins of linear regression (.28617). In this case, the predicted P(win) might be used to determine whether or not to bet on a team against the spread. Logistic regression did slightly worse in terms of classifying too many games as home team wins (76.098% to 73.672%).

R² measures the variability in a data set (i.e. the final score margins) accounted for by a model (i.e. the input data). Unfortunately, the R² for the linear regression model is .13813 on average. Less than 14% of the variance is accounted for by the model. NFL Stats uses this measurement in the context of how their model predicts season win totals (i.e. do the expected win totals match up with the actual win totals). From 1997-2006, the R² for my model of VOLA stats after week 17 predicting season win totals was .80287. From 2003-2006, the R² is .82432. For NFL Stats, the R² is 0.85 now, I believe, though it was .75 not too long ago. So I am not completely off track with my stats. In fact, the expected win totals for 2006 were off only by 1.3085 games on average, down from 1.3523 in 2005. (San Diego, Baltimore, Indianapolis, and New England all outperformed their expected win totals in 2006.)

Originally, I said that predicting exact final scores was nearly impossible because so many non-predictable factors went into the final score. Perhaps, if even the spread is off by 10 points on average, then the final score margin is determined by enough non-predictable factors that make the problem overly difficult. From now on, I'll be testing logistic regression in addition to linear regression.

Note: For the article, I used glmval and glmfit in MATLAB to do logistic regression with a binomial model.

Read More......