# How Much Will the Debate Shift the Polls? Here’s One Calculation.

by on October 4, 2012 · 12 comments

In my snap predictions last night, I was wrong about one thing: the post-debate insta-polls definitely suggest that most debate viewers believed Romney won: CNN had it 67-25.  A second poll of uncommitted voters conducted by CBS and Knowledge Networks scored in 46-22 for Romney.  (My second prediction—that the media would reach the same conclusion—turned out better.) One caveat is that insta-poll results don’t necessarily correlate with debate bumps in the polls, as Harry Enten shows.  But still: these results obviously aren’t what Obama wanted.

My ultimate prediction was that Obama’s lead in the polls—now about four points—would narrow by about 1 point.  Does that still seem reasonable?  Here’s another take.  The political scientist Joe Cera looked at the first debate in 2000—one that appears to have moved the polls, as I noted in my piece.  Using a large panel of respondents interviewed before and after the debate, he shows that Bush gained about 3 points, all of which were taken from the pool of undecided voters.  The undecideds shrank from 12% to 9%. 2000 is a useful comparison because roughly similar fractions of people thought Bush had won—57%—as thought Romney won (57% is about what you get averaging the CBS and CNN polls above—with the caveat that they were sampling from different populations).

So let’s assume that the same fraction of undecided voters shift in Romney’s direction as shifted in Bush’s direction.  In 2000, that was 25% of undecideds (3/12=.25).  In 2012, the proportion of undecideds is about 5%.  So 25% of 5% is 1.25 points—about what I predicted last night.  It could end up being a bit more, if you think that Obama has been out-performing the fundamentals of the race, and thus we were due for a course correction.

But if that estimate proves correct—or at least close to it—then as Drew Linzer points out, that’s not enough.  Romney needs to run the table in the subsequent debates, and enjoy favorable news coverage besides (i.e., no more 47% videos).  I tend to doubt that Romney can win the other debates by this same margin.  One of the reasons that the candidates typically fight to a draw in the debates as a whole is that they can often rebound from bad performances with better performances—e.g., Reagan in the second debate of 1984.  We will see.

(NOTE: The post initially described the CBS poll as a poll of debate viewers, which it was not.)

Dan H. October 4, 2012 at 10:29 am

John, just one quick note: the CBS poll (conducted through GfK) wasn’t among viewers but among those not previously committed to a candidate–a different population.

John Sides October 4, 2012 at 10:42 am

Thanks, Dan. I’ve fixed the post.

Erik Voeten October 4, 2012 at 10:40 am

A question: are there good reasons to take the percentage of undecideds at face value? Do we know what proportion of people who claim to have made up their minds at some point end up switching anyway?

Dan H. October 4, 2012 at 10:43 am
pat October 4, 2012 at 10:58 am

Beware of the CNN poll. They didn’t sample anyone under 50, non-white, or not from the South (according to their demographic notes on the poll itself).

Dan H. October 4, 2012 at 11:04 am

Pat, I think those N/As in the cross-tabs indicate that there isn’t a large enough sample size for them to estimate sub-group measures. But since these are people who identified themselves as debate watchers, they are not reflect of the electorate–in the cross-tabs, you’ll see that Romney’s favorables are much better than Obama’s pre-debate, for instance, which isn’t consistent with much of the other polling.

Joshua Tucker October 4, 2012 at 4:25 pm

Pat/Dan H: Ok, maybe there are sample size is small, but, um, not enough people under 50? Not enough people outside of the South to get a “non-South” estimate? I could understand if the debate took place in Atlanta – maybe – but this debate took place in Colorado. How is the South the only region of the country with enough respondents to estimate a % for a sub-group? A bit odd.

Dan H. October 4, 2012 at 10:17 pm

The sample was selected based on people who indicated they’d be watching the debate in a prior survey. So we’re interacting the selection processes of who takes a survey with the selection processes of who watches the debate. And the sample isn’t huge to begin with. The sample leans conservative, but Romney was perceived to win among moderates, and a sizable minority of Democrats thought so too.

Joshua Tucker October 5, 2012 at 3:04 am

Dan, the only groups large enough to get estimates were people over 50, whites, people in the South, and those who had gone to college. Surely that suggests a very odd result from the sampling? I’m not in anyway implying the group was picked for a particular reason, but is it irresponsible of CNN to present this as a verdict on the debate without acknowledging the obvious skew of the respondents? It is clearly heavily weighted towards people who like Romney. Even among Democrats, wouldn’t you think older white southern democrats with a college education are more likely to be favorably inclined towards Romney than other samples of Democrats?

I’m not questioning the bottom line of the poll – there is certainly evidence that all else being equal, a lot of people think Romney did better in this debate, and the “moderated” and “independent” categories are good for estimating this net of partisanship. BUT… the headline numbers are being thrown in to estimates of how big a bump in national polls candidates are going to get, and I just think they may be the wrong headline numbers ( see for example “Nate Silver’s post”:http://fivethirtyeight.blogs.nytimes.com/2012/10/04/oct-3-romneys-electoral-challenge-and-more-on-debate-instant-polls/)

lmd October 5, 2012 at 10:09 am

I agree with Josh that it would have been nice to see more discussion of the fact that this sample is more pro-Romney than the actual electorate (for whatever reason). I don’t think it’s all that surprising that whites and southerners were the only subgroups that could be estimated from a sample of less than 500 people, though. Whites are going to be the largest racial category in any survey and the South is the most populous region. It thus makes sense that these are the categories with enough people for CNN to produce estimates. What is more noticeable is the fact that (as Dan alludes to) Romney had a 54% favorable rating to 42% favorable rating when recent national CNN polls had those numbers at 49% favorable to 50% unfavorable. That to me is the best evidence that this survey is likely to overestimate the perception that Romney won. Of course it seems most people agree he won the debate, just maybe not by a margin of 67%-25%.