Tuesday, November 13, 2007

How Long Does Success Last? Part II The Search for More Money

Tables labeled properly now.

In Part I, I looked at the year-to-year correlations of wins to answer the titular question. In short, the team of this year is not very much like the team of last year. The draft, injuries, aging, free agency, development of players, and plain old luck mean there's always going to be a lot of change from year to year. The team of four years ago is (on average) nothing like the team of today. Take the 2003 and 2007 Patriots for example. Tom Brady is still the anchor of the team, of course. But the talent at the running back position is better (Maroney vs. Mike Cloud, Antowain Smith and assorted parts). The receiving corps is vastly improved (thanks entirely to Just a Wes Welker With Thee). It was part good fortune, part good talent evaluation that all of these moves worked out.

Part II takes a different look at the longevity of success. Instead of wins, which can be highly dependent on luck, this article uses opponent-adjusted value over league average (VOLA) for yards per run and yards per pass on offense and defense. The first table compares to year-to-year correlations of expected win totals based on adjusted VOLA stats with regular win totals.


nX is used to indicate the year-to-year correlations of the expected win totals.



















Correlations of WinsY with WinsY+X
XCorrelationP-valueSample size
10.232810.00020251
1X0.311880.00000251
20.198550.00317219
2X0.162310.01621219
30.130610.07480187
3X0.114910.11733187
40.052940.51295155
4X0.066870.40837155
5-0.028740.75230123
5X0.025440.78003123
60.004560.9656392
6X0.110850.2928392
7-0.057740.6585061
7X-0.027780.8317161
8-0.263210.1599230
8X-0.134680.4779730



As you can see, though the year Y to year Y+1 correlation is stronger for the expected win totals, neither actual wins nor expected wins have use beyond three years in predicting future success. But that's not the interesting part.


Look at what parts of teams are more consistent from year-to-year.











Correlations of ROY with ROY+X
XCorrelationP-valueSample size
10.450530.00000251
20.281030.00002219
30.221360.00233187
40.159350.04764155
50.095660.29256123
6-0.085910.4154892
70.124450.3392961
80.395250.0306430













Correlations of RDY with RDY+X
XCorrelationP-valueSample size
10.377980.00000251
20.328200.00000219
30.233220.00132187
40.210850.00845155
50.135600.13480123
60.143720.1717092
70.221470.0862961
80.272800.1447030













Correlations of POY with POY+X
XCorrelationP-valueSample size
10.500380.00000251
20.337220.00000219
30.248610.00060187
40.149240.06382155
50.111320.22027123
60.102420.3313192
7-0.120300.3557461
8-0.229550.2223730













Correlations of PDY with PDY+X
XCorrelationP-valueSample size
10.346420.00000251
20.327170.00000219
30.303940.00002187
40.318930.00005155
50.284350.00144123
60.250790.0159092
70.210810.1029361
80.134200.4795530



Defense is thought to be less consistent within a season than offense, going in part to explain why offense has a higher correlation with winning. From year to year, however, defense is significantly more consistent with pass defense being the most consistent of the 4 stats. Pass offense quality is similarly longer lasting than run offense. Though I don't have the data on hand to verify any of this, I have some hypotheses as to why it falls out like this. For passing vs. rushing, the results are much more intuitive than offense vs. defense.

Players involved in the pass move around less in free agency and less frequently injured. Running plays involve much harder hitting than passing plays, and RBs are the most likely to suffer season ending injuries. If a RB is really good, the temptation to overuse that player is often not resisted (e.g. Larry Johnson). There's also the idea the RBs are fungible, so they're allowed to leave in free agency. You might be able to prove this by looking at the year to year correlations of Football Outsiders' adjusted line yards for offensive lines. The idea being that the blocking stays more consistent, but the health of the RB plays a large overall factor in the actual result. On the flip side, the defensive linemen's massive bulk might lend itself to greater injuries, leading to less consistency and quicker aging in run defense. It'd be interesting to look at injury rates for different positions in the NFL.

The players involved in the passing offense are also more highly valued and less likely to be thrown aside. Investments in quarterback through the draft are allowed to pan out for at least a few years (often 4-5), and the good quarterbacks stay good and the great ones stay great. There's also a premium on WRs. Look at how Randy Moss transformed the Patriots. So teams are more likely to shell out big money for their FA WRs. Another factor to look at would be the average length of a player's tenure with a team for each position. Similarly, the secondary players (passes defensed, INTs, and opposing receiver yardage) and quick defensive ends (sacks, hurries) have a more distinguishable impact in the game than linebackers (tackles), leading them to get valued more highly.

Perhaps depth might explain why defenses are more consistent from year to year. Within a game, a team might substitute players less frequently in one game than another depending on the sequences of plays (e.g. rotating DTs after several running plays) and how well their offense is performing (i.e. time of possession). Maybe teams spend less money on individual players and value depth over stars more frequently with defense than with offense. A single player can completely change an offense, but it's less likely to happen with a defense. Thus, when that star offensive player gets injured, they can't respond as well, leading to less year-to-year correlation in performance. Anyone have a table on average salary by position?

In conclusion, in terms of wins, success lasts at most 3 years, but the success of individual units lasts at most 5-7 years. By then, drafting, aging, injuries, and free agency will have shifted the balance of power around. It's harder to keep a good offense around than a good defense, and it's harder to keep a good running game than a good passing game. But in any given year, any given team can make a big leap forward or a huge tumble backwards in performance. If there is not hope for the next year, then there will almost always be hope for the year after that.

2 comments:

Brian Burke said...

I think this is very interesting, especially the "y+x" tables showing how year-to-year correlations degrade over seasons.

But I do have one critique. While it's valid to compare correlations of one phase of the game, such as run offense, to itself over time, it's not necessarily valid to compare correlations between phases. Correlation coefficients only have meaning within a context, and the different efficiency stats RO, RD, etc. provide different statistical contexts.

For example, passing defense may suffer from higher levels of natural variance due to luck than running defense suffers. So due to the smaller component of variance that is not luck means that you will likely see a smaller r for pass defense than run defense, even if they were "truly" equally consistent.

You'd need to account for the part of variance that is natural luck, or assume they are equal or negligible, which I doubt they are. But comparing the correlations for different time periods within the same phase is valid, because the luck variance can be assumed to be equal.

Derek said...

Per your critique:
You're quite right, and thank you for pointing it out. The tabels were simply mislabeled. I do not do as much proofreading as I should, and the code I wrote to generate the tables was a modified version of the code used to produce the tables in the first part of this enthralling series.