Actually, I suppose it is merely the latest salvo in a stat war that has been brewing for some time.
Basketball statistics expert and economics professor Dan Rosenbaum (who has consulted with the Cleveland Cavaliers) announced yesterday on the APBRMetrics forum that at a sports statistics conference next weekend he'll be making a presentation arguing, essentially, that professor David Berri's Wages of Wins basketball statistical system is profoundly flawed. Rosenbaum writes:
Dave Lewin and I will be presenting a paper on this issue next weekend at the NESSIS conference at Harvard. The paper is not written yet (and may not be completely written by next weekend), but we will make it available when it is done. And if the paper is not done next weekend, we will make the PowerPoint presentation available. Berri has done a fabulous job of self-promotion with Wins Produced, but it is not good science.
I hope that after this paper comes out, we can just put this whole issue to bed. Let Berri do his thing, but his thing is not about advancing the science. If it was, he would have made a working paper version of his Wins Produced paper (that he has been citing for at least three years) available to the general public or to interested analysts who ask for this paper. That's how good work is done in academics.
Is Rosenbaum right that Berri's work is, esssentially, best ignored?
I am outside of my area of expertise here, but can tell you that I have talked to both Berri and Rosenbaum several times each. Both strike me as really smart, and interested in good science.
Berri for his part, insists 100% of the science necessary to assess his system has already been published, in his book "Wages of Wins." (Furthermore, he says that selling books makes him very little money, and the common accusation that he's hyping his system to line his pockets is less convincing if you know the actual numbers.)
I'll say this: I'm sorry to see so much energy going into intramural attacks in a field that is still finding its feet. Maybe one day Berri and Rosenbaum will be able to help each other, and statistics, in some way. They are both young and, I assume, in this thing for the long haul.
They are also both interested in one of my favorite parts of basketball statistics: trying to measure not just how well a player performs, but how much a certain player's performance contributes to winning.
It's messy and difficult work, there is little consensus about how exactly it should be done -- hence the dispute we're talking about now. It's a lot easier just to slice and dice points, rebounds, and the like. But in the long run, the connection between those points scored or those rebounds grabbed is theoretical. (If you have a guy on your team who averages 40 points per game, is your team better than if you don't have that guy? Under what conditions? By how much? An easy question to speculate about -- surely it has a lot to do with efficiency -- but a tough thing to prove with real numbers over years and years.)
Berri and Rosenbaum are among those who are looking at tons of data to figure out formulas for how much different kinds of performance affects victories. If you sign and play this guy, how many more games can you expect to win than if you sign and play that other guy the same minutes?
Just about everyone agrees that the way NBA teams have historically assessed players overvalues scoring and undervalues things like rebounding and steals -- not to mention things that aren't well measured like, for instance, defense. Be nice to have a clue as to by how much, and what other things are important if you want to win basketball games.
One thing this debate speaks to? The power of Malcolm Gladwell.
Gladwell's writing about Wages of Wins in the New Yorker, and more than once on Gladwell's website, made Wages of Wins the de facto king of rankings systems in many people's minds. I don't think an exhaustive and objective assessment (here's a quickie one) of how this system compares to everything else that's out there, like John Hollinger's Player Efficiency Ranking, Win Shares, eWins, and the like, has yet been done. But I'm pretty sure that time in Gladwell's spotlight is one of the great prizes of the field, and who wouldn't want some of that?
