Polling Biases and Their Potential Impacts

by Dan Hopkins on September 26, 2012 · 7 comments

in Blogs

Over at the Weekly Standard, Jay Cost argues for caution in viewing recent polling of the Presidential race.  Polls of swing states like Ohio and Florida are projecting an electorate that looks more like the 2008 electorate than the 2004 electorate in terms of its partisanship, he notes.  In fact, some polls have an electorate that is more Democratic than that in the ‘08 exit polls.  In contrast to recent electoral history, contemporary surveys suggest that Republican partisans are defecting at slightly higher rates than are Democratic partisans.  What’s more, independents appear to be quite evenly divided between Governor Romney and President Obama, an observation which seems incompatible with the sizable lead several polls now show for President Obama.

On the question of independents, John has written plenty about the problem of covert partisans—that is, that most independents act like partisans of one party or the other.  But one thing that has been less remarked upon is that the use of the “independent” label appears to have shifted somewhat in recent years, a fact which has implications for using independents as a bellwether.  Using 7-category partisan identification data from the most recent NBC/Wall Street Journal poll, which identifies covert partisans in a follow-up question, we can estimate that 24% of Democratic identifiers or leaners call themselves “independent” at first.  On the Republican side, the figure is 32%. Nowadays, more of the voters we think of as reliable Republicans are at first calling themselves “independent”—and are classified as such by any poll that doesn’t ask the follow-up question.  And if we look back to the same survey’s data in September 2004, we see some noteworthy differences.  At that time, leaners made up 30% of those who were willing to term themselves “Democrats” in some form, while they made up just 26% of those who termed themselves “Republicans.”  So eight years ago, the situation was reversed, with Democrats a bit more likely to use the “independent” label at first.  My point: independents are a moving target.

Despite declining response rates, live-interviewer telephone surveys have generally proven accurate in recent years.  But it seems to be just a matter of time before we have another Literary Digest moment on a grand scale, and all of the challenges in contemporary telephone polling lead us to systematically misunderstand a Presidential race.  So it’s important to ask: what happens to our Presidential forecasts if Cost is right and if the polls are systematically over-estimating support for Obama, just as the 2004 exit polls overstated support for Senator Kerry?  Emory political scientist Drew Linzer takes on that question in a new post at Votamatic, a site which shows the results of his 2012 election projections. His projections are drawn from a statistical model that combines polling data with a forecasting model.  The site is a must-read from those who want to follow the state of the 2012 race, although don’t expect to see sharp day-to-day swings.  One central feature of Linzer’s forecasts has been their stability.

Linzer’s post considers a number of scenarios, including the possibility that Cost and others have suggested: a systematic pro-Obama lean in the polls.  If that pro-Obama bias is 1 percentage point, if we assume that states swing together, and if we add a good deal of volatility to the forecasting model, Governor Romney wins the electoral college in 20% of simulations.  At a bias of 2 percentage points, that chance improves to 37%. Put differently, according to Linzer’s model, it would take a substantial polling bias alongside a nationwide, pro-Romney shift in the campaign to make the Governor the favorite.  For more, go to the source.

{ 7 comments }

Comments on this entry are closed.

Previous post:

Next post: