Preseason Analysis Part 1b - Final Regular Season Projections Based on Preseason Stats

In this article, I used the rushing and passing averages of projected offensive starters and the total rush/pass defense averages to project regular season win totals. Now that the preseason is over, I'm just redoing the projected win totals. The method is simple: train a linear regression model of regular season win totals using regular season stats (1996-2006), and then plug in 2007 preseason stats as the 2007 regular season stats. For simplicity's sake, I am using only rushing and passing yards per play in the model.

So what's changed with an extra two seasons of preseason? The projected win totals are all within the 0-16 range, which is good. There don't seem to be any projected division standings that feel like the reverse of what they should be, but most of the divisions still seem to place one team too high. One funny thing is that the NFC is projected to be the stronger conference. The AFC has 2 10+-win projections, as opposed to 5 for the NFC alone. Perhaps the balance of power is shifting? Detroit is still projected to win the division with 10 wins. "You know what's really strange? Jon Kitna was right..." "I know, kids. I'm scared, too."

Below are the projected standings for the 2007 season.


AFC East

1. New England Patriots, 8.4262 wins
   
2. New York Jets, 6.6077
   
3. Buffalo Bills, 4.8474
   
4. Miami Dolphins, 0.72791
   

AFC North

1. Pittsburgh Steelers, 13.822 wins
   
2. Cleveland Browns, 8.3653
   
3. Baltimore Ravens, 6.0087
   
4. Cincinnati Bengals, 5.3328
   

AFC South

1. Jacksonville Jaguars, 11.51 wins
   
2. Indianapolis Colts, 9.7991
   
3. Tennessee Titans, 8.7978
   
4. Houston Texans, 7.1054
   

AFC West

1. Oakland Raiders, 9.4303 wins
   
2. San Diego Chargers, 9.2745
   
3. Denver Broncos, 7.1972
   
4. Kansas City Chiefs, 3.3213
   

NFC East

1. Philadelphia Eagles, 12.632 wins
   
2. Dallas Cowboys, 11.58
   
3. Washington Redskins, 9.7338
   
4. New York Giants, 4.0315
   

NFC North

1. Detroit Lions, 10.16 wins
   
2. Chicago Bears, 9.4675
   
3. Minnesota Vikings, 7.881
   
4. Green Bay Packers, 5.2574
   

NFC South

1. New Orleans Saints, 14.402 wins
   
2. Carolina Panthers, 8.8436
   
3. Atlanta Falcons, 7.3336
   
4. Tampa Bay Buccaneers, 3.5238
   

NFC West

1. Seattle Seahawks, 11.19 wins
   
2. San Francisco 49ers, 8.1366
   
3. Arizona Falcons, 7.5798
   
4. St. Louis Rams, 4.5614
   

Read More......

Preseason Analysis Part II - Correlation with Regular Season, More on Expected Wins

Previously on Preseason Analysis...
And now Part II of Preseason Analysis.

Preseason wins and losses are essentially meaningless, but maybe team performance, as measured by things like run efficiency and pass efficiency, still has some information whose accuracy carries over into the regular season. Intuitively, first-string players are still playing first-string players, so some true reflections of skill are bound to show up. So I decided to take preseason box scores from 1997-2006, and see what the stats show about teams' regular season performance. Please note that separating out first-stringer offensive stats only is too time-consuming (and possibly for something not that useful), so I'm just using total team stats for the game. Because of missing stats and the questionable usefulness of those stats, I've dropped punt returns, kick returns, and penalty first downs from my models.

Average preseason and regular season league averages
Stat	Preseason	Regular Season
R	3.8239	4.0708
P	5.5977	5.8866
SR	0.069901	0.068252
3C	0.37317	0.3777
PY	61.886	54.266
IR	0.026319	0.029888
FR	0.04069	0.031635

Correlation of preseason league average with regular season league average
Stat	Corr. coef.	P-value
R	-0.023	0.94972
P	0.53682	0.10961
SR	-0.03985	0.91297
3C	-0.16464	0.64945
PY	0.72912	0.016728
IR	0.052921	0.88456
FR	0.41608	0.23171

R=Rush, P=Pass, SR=Sack Rate, 3C=3rd Down Conv., PY=Penalty Yards, IR=Int. Rate, FR=Fum. Rate

Simply put, the preseason game is appreciably different than the regular season game. The first thing you notice is that the preseason favors defense. Yards per play is lower, sack rates are higher, fewer third downs are converted, and fumble rates are higher. On the other hand, interception rates are lower. More penalties are called. Perhaps coaches are more conservative on offense, saving most of their plays for the regular season. Maybe it's because coaches give almost all of their QBs playing time.

Meanwhile, pass yards per play, penalty yards per game, and fumble rate are the only three stats whose preseason averages correlate significantly with regular season averages, but only penalty yards has a significant p-value. The p-value is essentially the probability that a correlation coefficient that extreme could be achieved with entirely random inputs. Usually, the level for a stat to be considered significant is at 5% or less. Given the small sample size of postseason, we'll excuse the higher p-values of fumble rates and pass efficiency. Preseason average rushing efficiency, meanwhile, has essentially no correlation with regular season average rushing efficiency. If the preseason has little meaning on a league-wide level, then how does it fare on a team scope?

Correlation of preseason stats with regular season stats, Unadj. VOLA
Stat	Corr. coef.	P-value
RO	0.18156	0.0012547
RD	0.15345	0.0065266
PO	0.28506	2.8993e-007
PD	0.25502	4.8902e-006
SRM	0.10118	0.073865
SRA	0.24271	1.4125e-005
3CM	0.20165	0.00033031
3CA	0.17538	0.0018427
PY	0.22266	7.0842e-005
IRG	0.10879	0.054507
IRT	0.118	0.036931
FRG	0.047389	0.40343
FRT	0.11199	0.047738

O=Offense, D=Defense, M=Made, A=Allowed, G=Given, T=Taken

Offensive performance seems to correlate better overall between preseason and regular season than defensive performance, with turnover rates being the only exception. Surprisingly, almost all of the p-values are below 5%, with fumble rate given being the only significant exception. In other words, it's highly unlikely that random inputs could create similar correlation coefficients, so it's safe to assume that overall team preseason performance means something, just not much. If your team does well in the preseason, that's great, but it's hardly a guarantee of success. If your team does poorly, it's really not all that much to sweat about. Of course, this meets our expectations because second-string and third-string players get playing time they won't get in the regular season. If someone wants to take the time to sort through the box scores to figure out the efficiency stats for first stringers, they can be my guest. It's questionable how much the correlation coefficients would actually improve. Similar results can be seen with the correlation coefficients of preseason stats with regular season wins.

Correlation of preseason stats with regular season wins, Unadj. VOLA
Stat	Corr. coef.	P-value
RO	0.03494	0.53798
RD	0.049808	0.37983
PO	0.18824	0.00081724
PD	0.18979	0.00073831
SRM	0.087023	0.12445
SRA	0.14418	0.01065
3CM	0.14428	0.010596
3CA	0.21909	9.2999e-005
PY	0.054488	0.33663
IRG	0.11554	0.04107
IRT	0.029016	0.60908
FRG	-0.023628	0.67711
FRT	0.093994	0.096927

Out of curiosity, I decided to create a linear regression model of regular season win totals using the following preseason stats: pass efficiency, sack rates, and third down conversion rates. With 1997-2006 stats, I tested on each year in 1998-2006, using all previous years as training data. On average, the predicted win totals have a correlation of 0.30593 with the actual win totals, not very high. The yearly average of mean absolute error was 3.1519 games, about twice what it is when using regular season stats. The average R² was 0.67495, which took me by surprise a little. 67.945% of the variance is accounted for by this data, compared to 79% for the regular season stats? I was expecting 40-50% tops.

What's more interesting about the model, however, is its ability to predict which teams will improve/decline the following season. In a manner similar to what I did here, I looked at which teams exceeded or fell short of their predicted win totals by more than the mean absolute error. Teams that outperformed their projected win total based on preseason stats are predicted to decline the next year, and teams that underperformed their projected win total are predicted to improve. Because there is some positive correlation between regular season and preseason stats, I expected some of the success using regular season stats to carry over. What I found, however, was that the model with preseason stats is slightly more accurate than the model with regular season stats. This might be a result of noise created by the extra inputs in the regular season stats model (e.g. kick and punt returns).

1998-2005 (predicting 1999-2006)
Accuracy predicting risers: 74%
Accuracy predicting fallers: 61.818%

2002-2005 (predicting 2003-2006)
Accuracy predicting risers: 67.857%
Accuracy predicting fallers: 67.742%

Preseason seems to have some useful meaning then. On the other hand, we're talking about a 6-game range that of which a team has to fall outside for it to be a faller/riser. If the projection is 8 wins (average), a team could be bad (5 wins) or very good (11 wins) and still be within the average error. The method predicts about 6-7 risers and 6-7 fallers every year, so it's accurately predicting 8-9 teams to improve/decline each year. That's pretty good, I think. Without further ado, here are the projected risers and fallers for 2007:

Risers

• Houston Texans (9.7623 expected wins vs. 6 actual wins)
  
• Jacksonville Jaguars (12.371 vs. 8)
  
• Oakland Raiders (10.005 vs. 2)
  
• Dallas Cowboys (15.354 vs. 9)
  
• New York Giants (11.979 vs. 8)
  
• Tampa Bay Buccaneers (8.6278 vs. 4)
  

Based on the accuracy and what other projection models have shown, I'd pick Jacksonville, Oakland, Dallas, and Tampa Bay as the ones to actually improve.

Fallers

• New York Jets (4.9253 expected wins vs. 10 actual wins)
  
• Baltimore Ravens (8.2891 vs. 13)
  
• Kansas City Chiefs (2.9868 vs. 9)
  
• Chicago Bears (8.6448 vs. 13)
  
• New Orleans Saints (5.9049 vs. 10)
  
• San Francisco 49ers (4.1093 vs. 7)
  
• Seattle Seahawks (2.0842 vs. 9)
  

Of these, I'd pick the Jets, Ravens, Chiefs, and Bears to decline. It really could go either way with the 49ers and Seahawks. That division is chaos.


After jumping through some hoops, I do seem to have found some relevance to the preseason. But it's nothing you couldn't find using regular season performance. As intuition would tell you, preseason performance is only slightly indicative of regular season performance.


2001 BAL@PHI box score was missing.

Read More......

Preseason Analysis Part I - Using Preseason Stats to Predict Regular Season Wins

As a Dolphins fan, the preseason has been less than encouraging despite the 2 wins. The offensive line's run blocking sucks to put it mildly, and the pass protection has been mediocre at best. Trent Green's accuracy is off, and the secondary hasn't been great either. But the sample sizes have been extremely small, which is the most important reason why I think preseason performance has little meaning. Many of the first-string QBs and RBs have less than 20 attempts. If one game is hard to predict because of the natural variance in performance, than one regular season game's worth of plays won't tell you a whole lot about a team. Then again, skill is skill and should show up to some extent in any game, regardless of its meaning. Depth is also important. So I've decided to examine the validity of my assumptions over the next few articles.

First up, can we use preseason efficiency stats to estimate regular season win totals?


For this experiment, I decided to use yards per rush and yards per pass stats only to keep things simple. Using unadjusted Value Over League Average to predict regular season win totals in 2006, a system with only Off. and Def. Pass and Rush Efficiency stats had a mean error of 1.541 games and an R² of 0.6224. A system using all unadjusted VOLA inputs had a mean error of 1.233 games and R² of 0.77047 in comparison. So even with the reduction in detail, the retrodictive system is still pretty good.

For the preseason efficiency stats, I wanted to stick with the performance of first stringers as much as possible. They're the ones that are going to be playing all season (hopefully). For offensive efficiency stats, this was pretty straightforward. I just took the yards per pass from the QB stats page on NFL.com and the yards per rush from the RB stats page. Clearly this relies on my selection of who's first string. Sometimes the true starter was injured or holding out. If you want to know exactly whose stats I chose, feel free to e-mail me. On defense, I wanted to use first half stats only, but those would have been non-trivial to obtain. In the interest of just getting a rough draft of the idea out there, I just used overall yards per rush/pass. To get a VOLA, I just used the average of the efficiencies as the "league average". Again in the interests of time, I kept that calculation very rudimentary. To predict regular season win totals, I simply pretend that the preseason VOLA stats are the regular season VOLA stats and plug them into the retrodictive system. In other words, we're assuming that the VOLA stats at the end of preseason will be the same as at the end of the regular season, though it's not clear at all that a strong correlation exists. In an upcoming article, I will look at the correlation between preseason efficiency and regular season efficiency.

Because the sample sizes for offensive stats were small, some teams ended up with extremely high or low VOLAs. This did not mesh well with the regression coefficients, which resulted in some teams being predicted to win less than zero games and some to win more than 16 games. Take the actual win totals with a grain of salt. But let's look at how it predicts the divisional standings.

2007 Predicted Final Standings Based on Preseason Stats through Week 3
AFC East

1. New England, 6.9208 wins
   
2. Buffalo, 4.3031
   
3. New York Jets, 4
   
4. Miami, -0.42247
   

New England tops the list thanks to an about average pass offense and pass defense efficiency. Ranks Buffalo too high when it should be last by most opinions.

AFC North

1. Pittsburgh, 14.568 wins
   
2. Cleveland, 9.0493
   
3. Baltimore, 6.5127
   
4. Cincinnati, 2.4849
   

Like Buffalo, Cleveland should probably be last, but the order is otherwise plausible.

AFC South

1. Tennessee, 7.636 wins
   
2. Houston, 7.2683
   
3. Indianapolis, 6.3487
   
4. Jacksonville, 6.161
   

Exactly the reverse order of what it should be. Indy's run defense is 26.255% above league average this preseason. It's pass offense is only 5.8% above average. Think that will last?

AFC West

1. Oakland, 13.317 wins
   
2. San Diego, 8.2215
   
3. Kansas City, 5.0995
   
4. Denver, 4.3164
   

Another case of one team being ranked too high instead of last. Oakland's run offense efficiency is 128.77% above league average. Lamont Jordan has had 8.4 ypc. Interesting that this and the rankings based on actual vs. expected wins in 2006 put Kansas City ahead of Denver, despite the near certainty that Larry Johnson's ACLs will spontaneously combust before Week 4.

NFC East

1. Philidelphia, 23.678 wins
   
2. Dallas, 11.756
   
3. Washington, 9.7259
   
4. New York Giants, 7.0968
   

Based on FO's prediction of Washington being on the rise, these rankings seem totally plausible.

NFC North

1. Detroit, 10.856 wins
   
2. Minnesota, 10.071
   
3. Chicago, 6.7431
   
4. Green Bay, 3.9523
   

Jon Kitna was right! They ARE going to win 10 games! Or not. Another division in reverse order of what it should be.

NFC South

1. New Orleans, 9.145 wins
   
2. Carolina, 8.435
   
3. Atlanta, 8.2839
   
4. Tampa Bay, -0.64007
   

0.8 ypc from Cadillac Williams would do that to Tampa Bay. The rankings in this division seem plausible.

NFC West

1. Seattle, 17.836 wins
   
2. San Francisco, 13.602
   
3. St. Louis, 6.5272
   
4. Arizona, 4.7789
   

Both Seattle and San Francisco benefit greatly from very strong offensive pass efficiency stats, but the ranking is plausible.

I'll refine and recalculate the system at the end of preseason and post the revised predictions. I'll also post predictions using pass and run offense efficiency stats based on entire team performance, rather than on one player at each position. But in addition to sample size issues, the opponent quality is much more varied from team to team, as they do not travel far for road games in the preseason. Expect a lot of noise in the projections because of it. In terms of regular season wins and losses, I don't think this system will be particularly accurate, but in terms of predicting division standings, preseason performance might yield interesting information.

Read More......