Pomeroy Rankings- A Significant But Not Final Word

Some readers may remember my inquiry/obsession with the accuracy of the Pomeroy rankings. Specifically, I inquired here as to whether they overvalued reams that play at a very slow tempo or undervalued teams that play at a quick tempo.

Well, apparently I'm not the only one who was asking this question, because the Basketball Prospectus people did some great work on the topic. As you can see from this article unless you are stat-phobic like Hambone, the data suggests that if anything, fast-tempo teams are overvalued relative to tournament performance and slow-tempo teams are undervalued.

I do have one quarrel with the methodology here. My concerns about Pomeroy (I don't think it's accurate to call them criticisms) were that it missed the boat only with respect to the very slowest or very fastest teams, and only with the elite teams who play a lot of garbage time, since that's the only time a disparity in garbage time might be significant enough to affect the rankings.

Therefore, I'm not sure that dividing the tournament field into four tempo "quartiles" really does the trick. Put another way, using this year's tempo numbers: if they did this study on the 2009 tournament, UNC and Davidson would both potentially qualify for the top quartile of tempo teams with 6 or better seeds in the tournament, while Wisconsin and Arizona State would be in the last quartile of tourney teams with 6 or better seeds. My concern was not that a team like Davidson has played a lot more garbage time than Wisconsin, but rather that UNC has played a lot more garbage time than a team like Arizona State. it wasn't the quartiles I was concerned with, because on the whole I think the garbage time comes close to averaging out when you're talking about teams with 21-11 records. It's the teams with the 29-4 records where I think you'd see the disparity. And this is the most important thing to study, since those are the teams you most want to be able to compare to one another.

Anyway, I think they did a great job here and I understand the sample size problems with running these numbers using only the elite teams, but it does leave some lingering questions. And also, the fact that found that the slowest tempo teams were undervalued when they used the 1-6 seeds and divided the field into quartiles suggests that even if they reduced the teams in the analysis to the elite and to the slowest and fastest, there probably still wouldn't be a problem. But it's worth making this point here.

Also, it still leaves open the question of why there's not some easy, automatic cut-off point for the data that goes into the efficiency numbers. Certainly anything that happens in the last three minutes of a 30 point blowout is tainted. I guess maybe it's just too difficult to parse out and exlude that data out of hundreds of games a week. Understandable, certainly.

Finally, a reminder- this is just typical Grover nitpicking. I continue to use and love the Pomeroy numbers alongside the Sagarin numbers, and you should too.