Polling Biases and Their Potential Impacts

by Dan Hopkins on September 26, 2012 · 7 comments

in Blogs

Over at the Weekly Standard, Jay Cost argues for caution in viewing recent polling of the Presidential race.  Polls of swing states like Ohio and Florida are projecting an electorate that looks more like the 2008 electorate than the 2004 electorate in terms of its partisanship, he notes.  In fact, some polls have an electorate that is more Democratic than that in the ‘08 exit polls.  In contrast to recent electoral history, contemporary surveys suggest that Republican partisans are defecting at slightly higher rates than are Democratic partisans.  What’s more, independents appear to be quite evenly divided between Governor Romney and President Obama, an observation which seems incompatible with the sizable lead several polls now show for President Obama.

On the question of independents, John has written plenty about the problem of covert partisans—that is, that most independents act like partisans of one party or the other.  But one thing that has been less remarked upon is that the use of the “independent” label appears to have shifted somewhat in recent years, a fact which has implications for using independents as a bellwether.  Using 7-category partisan identification data from the most recent NBC/Wall Street Journal poll, which identifies covert partisans in a follow-up question, we can estimate that 24% of Democratic identifiers or leaners call themselves “independent” at first.  On the Republican side, the figure is 32%. Nowadays, more of the voters we think of as reliable Republicans are at first calling themselves “independent”—and are classified as such by any poll that doesn’t ask the follow-up question.  And if we look back to the same survey’s data in September 2004, we see some noteworthy differences.  At that time, leaners made up 30% of those who were willing to term themselves “Democrats” in some form, while they made up just 26% of those who termed themselves “Republicans.”  So eight years ago, the situation was reversed, with Democrats a bit more likely to use the “independent” label at first.  My point: independents are a moving target.

Despite declining response rates, live-interviewer telephone surveys have generally proven accurate in recent years.  But it seems to be just a matter of time before we have another Literary Digest moment on a grand scale, and all of the challenges in contemporary telephone polling lead us to systematically misunderstand a Presidential race.  So it’s important to ask: what happens to our Presidential forecasts if Cost is right and if the polls are systematically over-estimating support for Obama, just as the 2004 exit polls overstated support for Senator Kerry?  Emory political scientist Drew Linzer takes on that question in a new post at Votamatic, a site which shows the results of his 2012 election projections. His projections are drawn from a statistical model that combines polling data with a forecasting model.  The site is a must-read from those who want to follow the state of the 2012 race, although don’t expect to see sharp day-to-day swings.  One central feature of Linzer’s forecasts has been their stability.

Linzer’s post considers a number of scenarios, including the possibility that Cost and others have suggested: a systematic pro-Obama lean in the polls.  If that pro-Obama bias is 1 percentage point, if we assume that states swing together, and if we add a good deal of volatility to the forecasting model, Governor Romney wins the electoral college in 20% of simulations.  At a bias of 2 percentage points, that chance improves to 37%. Put differently, according to Linzer’s model, it would take a substantial polling bias alongside a nationwide, pro-Romney shift in the campaign to make the Governor the favorite.  For more, go to the source.

{ 7 comments }

IOKIYAR(ight-wing) September 26, 2012 at 12:32 pm

Republicans’ theft of elections is pervasive, deliberate, creative, but not always successful.

Stealing elections is made more difficult by accurate polling.

Republicans are deliberately trying to mask their attempted theft of the 2012 election by questioning the legitimacy of polls.

Republican’s criminality needs condemnation, not enabling excuses.

Dagny October 31, 2012 at 8:34 am

Fascinating delusion. Exit polls always lean democrat because people are physically afraid of the loony left and situations like Black Panther intimidation. Republicans and other actual Americans want voter ID to stop the rampant voter intimidation, illegal activity, recounting until they get the numbers they want, and precinct corruption from the democrats. Not wanting any proof of citizenship isn’t to disenfranchise people it’s to KEEP all of us from being disenfranchised by having our legal votes watered down. There is no republican theft–that’s never happened–it’s democrat theft esp in early voting that’s the terrifying reality. As the leftist Stalin said, it’s not the votes, it’s who counts them. Al Franken’s election is one of the best examples of the incredible corrupti0n found when democrats go wild. Oh, and IOKIYAR, you’re a hysterical example of projection. Go massage Obama’s 16 trillion dollar debt and his refusing help to people dying in Benghazi.

Joel September 26, 2012 at 2:45 pm

I think you’re being a bit too credulous with regards to Cost’s analysis.

Dan Hopkins September 26, 2012 at 2:57 pm

How so?

Jim September 26, 2012 at 4:11 pm

Cost also casually mentions that “elderly people now are overwhelmingly Republican” but looking at ANES data from 2008 I didn’t see evidence of that. My assumption is he’s principally telling his readers/listeners what they want to hear.

meijer September 26, 2012 at 7:14 pm

” Despite declining response rates, live-interviewer telephone surveys have generally proven accurate in recent years. ”

_________

That “proven accurate” claim uses very loose criteria.

A coin toss would do about as well… instead of any complex poll effort.

Picking the winner in a U.S. Presidential campaign just ain’t that hard.

American presidential election-votes are usually very close between two major candidates. It’s a simple binary choice between well known candidates. Even with highly flawed polling procedures it’s often easy to discern who has the edge, especially just before the actual election.

It is relatively simple to predict general elections, as very little of the electorate is up for grabs — maybe 15 percent.
(…pollsters have a very much harder time predicting primaries)

Of the roughly billion-and-a-half ballots cast for Republican or Democratic presidential candidates since 1932 — there’s only hundredths of a percent difference of that total vote count between the two major political parties.

Anybody on the planet, knowing nothing about America or its politics, could flip a coin for predicting the Democrat/Republican winner in a specific presidential election — and be very “close” to a correct forecast.
Add in a little current political news from the American media… and it would be difficult NOT to guess the winner within a 1 or 2 percent error margin.

Pollster election “accuracy” is meaningless in this context.

IOKIYAR(ight-wing) September 26, 2012 at 9:36 pm

Republicans’ theft of elections through various means, specifically the systematic disenfranchisement of legal voters, makes the evidence of the Republican’s election theft from the polls something that the right-wing is currently trying to pre-muddy.

Note: Pollster’s accuracy is what has been used to show that dictators stole elections and led to citizen revolts that have been increasingly successful the last couple of decades.

Bonus: Even if Republicans’ theft of the 2012 election fails, the right-wing is currently creating the false counter belief amongst (some) Republicans that Romney is actually winning (and deserved to win) despite all the evidence to the contrary.

Right-wingers rejection of evidence, science, and the accuracy of the science of polling is itself shown through a growing body of psychological evidence (which in turn is rejected by [many] right-wingers).

“The Republican Brain” provides an easy to understand summation of the right-wing’s mindset of rejecting evidence and science.

Comments on this entry are closed.

Previous post:

Next post: