Monday, August 6, 2007

How Often Does the Better Team Win?

In the National Football Leagues, no win can be guaranteed. One play can have a large impact on any game. It's part luck, but it's part happenstance too. It takes skill to get an interception, but it's the happenstance of what play is being run, and thus where the players are, that largely dictate if the interception is returned for a touchdown or not. If the interception occurred in the red zone and was returned for a touchdown, that's at least a 10 point swing. So a play with a 2.96% probability of occurring (league average interception rate) has an inordinately large impact on the game. Brian Burke's blog had a very good piece on how much luck is involved with winning and concluded that half of winning games is luck (52.5% to be exact) so the better team is going to win around 74% of the time. I was curious to see how it worked out in reality and further validate my assertion that interconference games have more inherent variance and less predictability than intraconference games. To decide the better team, I simply used total DVOA from Football Outsiders (1996-2006). Please note that the DVOA stats are over the entire regular season and postseason, so they are retrodictive, not predictive. The predictive ability of total DVOA is not as good. If DVOA had a predictive accuracy of 70%, I wouldn't be working as hard on a prediction system.

Average result means how many more points the better team scores on average. Average margin of victory means how many more points the winning team scores on average. The averages are by year, so the total proportion of games won by the better team, etc. will vary slightly from the numbers listed here. Part of the original study on interconference study was to see how much year-to-year variance there was in the outcomes of those games.

Better Team Win %Avg. ResultAvg. Margin of VictoryBetter Team Win %Avg. ResultAvg. Margin of Victory
Mean, 1996-20010.680566.922212.2280.664465.69411.042
Std. Dev, 1996-20010.0798732.50310.997760.0477331.5790.75167
Mean, 2002-60.656.931312.2130.689586.539611.169
Std. Dev, 2002-60.0450151.15370.899530.0485241.49970.92161

IntradivisionAll Games
Better Team Win %Avg. ResultAvg. Margin of VictoryBetter Team Win %Avg. ResultAvg. Margin of Victory
Mean, 1996-20010.69066.444911.2320.680966.356911.42
Std. Dev, 1996-20010.0475671.11090.705740.026391.03430.24626
Mean, 2002-60.733337.072911.3560.696096.837511.5
Std. Dev, 2002-60.0502811.0890.530040.0275380.648520.31041

So with the 20/20 hindsight of each entire season, the better team has wins about 69% of games, close to the 74% reported in Brian's blog, which was based on 2002-6. When looking at 2002-6 intradivision games, he was pretty much dead on. 73.333% vs. 74%. Most of the discrepancy can be traced back to interconference games. The divisional realignment in 2002 reduced year-to-year variance in the percentage of games won by the better team, but it's also reduced the average percentage from 68% to 65%. It's interesting that the average margin of victory is larger in interconference games than in the other types, but I'm not sure what that means.

I have two ideas on possible reasons why fewer interconference games are won by the better team. First, maybe coaches have more problems adapting strategy to opponents they don't see as often. An interconference matchup occurs only once every four years now (before, some matchups were much more common than others). Coaches have only a week to prepare for games, so they can only learn so much about a team's strengths and weaknesses. Obviously, the more time they have to study opponents, the more they will learn about them. So every time the interconference matchup comes up, the coach probably has to throw out a good deal of what he learned the last time. With intradivision matchups, you see the opponent twice a year and can re-use knowledge gained from previous matchups. Second, maybe stats should be adjusted for conference quality in addition to specific opponent quality like in baseball. I'm not sure this would work, given that the rules in both conferences are the same, unlike in baseball. Given that 75% of the season is intraconference, though, perhaps it's slightly inaccurate to judge a team based on the whole league, rather than their specific conference, when trying to predict an intraconference game. I've toyed with implementing this idea and might pursue it sometime in the near future.

1 comment:

Brian Burke said...

Here's a theory about interconference games. I don't think the difference is in how often the better team wins. I think it's more about a flaw in how we decide who the "better" team is.

For the past few years, the AFC has been the consensus stronger conference. Before that, the NFC tended to have stronger teams.

Let's assume the current perception that the AFC is stronger is true. A team that comes from the NFC might appear to be better than an AFC opponent, when it really isn't.

VOA is dependent on opponent strength. And DVOA is intended to adjust for opponent strength. But, if a perponderance of a team's opponents come from a weaker conference, DVOA will still be inflated.

A team's DVOA is in large part relative to the rest of its opponents. Imagine the NFL had no inter-conference games and one conference was twice as strong as the other. The aggregate DVOAs of the teams from each conference would be equal.

Now imagine each team played 1 inter-conf game. The conference DVOAs would only be slightly adjusted to reflect the difference in strength. Now consider 2 inter-conf games, 3, and 4. Each extra inter-conf game would help adjust the DVOAs, but not fully.

You would need close to an equal number of AFC and NFC opponents for DVOA to truly balance out the strength of schedule effects.

If true, our prediction models would suffer the same flaw.

My luck study was based on theoretical distributions and not statistical matchups, so I believe it is free of this effect.