A followup to my earlier post:
- Nate Silver's column today has two throwaway sentences that reminds me of one more reason I'm cautiously skeptical of the polls:
Some of the consistency in these results may reflect a tendency of polls to converge or "herd" around the polling average close to Election Day. This may occur because some polling firms alter their turnout models or other aspects of the polls so as not to produce outliers -- a dubious practice if the goal is to provide an objective take on the race.
In other words, it's not the case that there are two dozen pollsters that are each deciding independently that Obama is ahead in Ohio. What we're seeing is the most self-confident handful of pollsters standing by their decisions, and less confident pollsters succumbing to peer pressure rather than produce outliers. To have faith in averaging (or even weightedly averaging) these results, one has to believe that the most self-confident pollsters who don't shift their results are also the most accurate and statistically unbiased pollsters. The Dunning-Kruger effect suggests that there is a good chance this might not be so. Thus, all it would take is a handful of obstinate pollsters with statistically-biased likely-voter models to produce a systemic statistical bias in the polls. It's not necessarily the case: the polls that fudge at the end of the cycle might be reducing, rather than introducing, error. But we're not seeing the wisdom of crowds here; the Bayesian probability that there's a systemic bias in the polls is substantially higher than if the pollsters were producing results independent of one another.
- Josh Marshall responds to me and correctly points out that we're working with very very few data points. But that should just reduce our confidence that we know what's going on. Do the 1972 polls tells us anything about the 2012 polls? What about 1992 or 2000?
- Silver looks at recent instances of state polls and finds they got the winner right the overwhelming majority of the time. Of course, as Silver recognizes quietly, that's not the correct question. How often were the polls 3 points off? One has to do the counting oneself, but it was nine times out of 29 in 2008 (albeit less often in closer state races).
- Colby Cosh has the best short piece on the Nate Silver phemonenon I've seen yet.
- Separately, thanks to Nate Silver for a generous tweet. He has a much larger platform than me, and is facing much more prominent critics than me with much stupider arguments; it would have been very easy for him to pretend I didn't exist and simply shoot fish in a barrel, and it's to his credit that he lent his twitter feed to promote my thoughts.