Friday, January 4, 2013

How Accurate are Exit Polls—of the Hall of Fame?

Are you a "big-Hall" person or a "small-Hall" person? How about a "zero-Hall" person? It sounds like that might be the consensus on Wednesday, when at 2pm the 2013 Hall of Fame class will be announced. Yes, the conventional wisdom is now that absolutely no one from an absolutely loaded ballot will lasso the 75% of votes necessary for induction.

This is borne out in Baseball Think Factory's extremely cool exit polling of the Hall of Fame vote. How can you do exit polling on a Hall of Fame election, you ask? Simple—you read the myriad ballot columns that are released at this time of year. As of my writing this post, BBTF had canvassed all 93 ballots that have been unveiled by their voters in a column, out of a likely 570 or so that will be cast (although that hardly seems predictive of this year's total, due to the ridiculously controversial ballot—it could stimulate interest and produce a record-high number of ballots, or so many people could be disgusted or bewildered by their choices that the number takes a substantial hit). According to BBTF's poll, no player has 75% of the vote; Craig Biggio, with 69.9%, comes the closest.

But exit polls, as John Kerry so famously learned, can be wrong. Can we determine how wrong they might be? Not for sure, but we can look at the historical performance of the same types of exit polls.

For the past five years, @leokitty has kept a spreadsheet tallying the percentage of votes each player got in the columns released that year. Here is how the figures compared last year, when the sample size was 114 out of an eventual 573 ballots cast (19.9%):

For the class of 2011, she unearthed 122 votes online out of an eventual 581 (21.0%):

And for the class of 2010, only 92 of an eventual 539 ballots (17.1%) were pre-countable:

Obviously any poll comes with a margin of error. However, there are some players whose vote totals are consistently over- or under-predicted by these surveys, just as political polls tend to undersample minorities and the young. Knowing this can tell us a lot about what we might be able to expect on Wednesday.

Specifically, the broad trend appears to be that the exit polls understate support for "old-school" candidates or candidates whose best case for inclusion is superficial. Such candidates include Lee Smith (with his eye-popping quantity of saves), Don Mattingly (the quintessential "professional hitter"), Larry Walker (looked great, but who doesn't at Coors Field?), and Jack Morris (everyone remembers him for that one dominant World Series start).

Another trend is that the exit polls overstate support for SABR-friendly candidates—or, more precisely, they just really overvalue poor Tim Raines. Interestingly, the polls do not seem biased toward or against any of the steroid-tainted characters.

This paints a picture of an polling sample (self-selected, remember; they are the ones who choose to write columns publishing their ballot for all to see) that is more pro-SABR and, if you'll excuse the editorialization, more forward-thinking than the electorate as a whole. This makes sense—a lot of the old-school voters (their minds calcified in an old way of thinking, like that wins and saves are important statistics) are no longer covering baseball, and thus don't have column space to devote to an explanation of their ballot. Similarly, some of the more closed-minded BBWAA members may not be interesting in sharing their ballot with the world, therefore exposing them to be ripped to shreds in comments sections and on Twitter.

So what does this mean for Wednesday's final vote tally, other than that Tim Raines is going to see yet another big dropoff? In the chart below, I've taken each player's current projected vote share as assessed by BBTF and added an adjustment factor, based on the player's average deviation from the exit-poll results in the past three years. For new candidates on the ballot, I've added no such adjustment factor (they are mostly steroid candidates anyway, and that issue has shown to have little effect on the accuracy of exit polls).

So it does indeed look grim. Even with the adjustments, Biggio remains closest. Can he make it? To be honest, looking at these numbers, I doubt it. He doesn't strike me as an old-school candidate; if anything, he's closer to the subtle greatness of Roberto Alomar and Barry Larkin, who were both overvalued by polls before. (But he is also white, which neither Alomar nor Larkin nor Raines is—could this also be a creeping factor?) Meanwhile, the top candidate likely to benefit from the old-school swing, Jack Morris, simply has too much ground to make up. He's currently polling at 62.4%, and while he is almost certain to get more than that on Wednesday, a 12.6-point bump would be unprecedented. If the old-schoolers directed their love to another, closer candidate—Jeff Bagwell—we'd have a decent shot at an induction, but Bagwell has been on the ballot for two years now and has shown no tendency to deviate from the polls. So all that brings us to the real question:

What if they had a Hall of Fame induction but nobody came?

No comments:

Post a Comment