Truths and Myths about the 2008 Election, Part II

Much ink was spilled speculating about whether the polls were over-stating support for Obama because people couldn’t admit to pollsters that they wouldn’t vote for a black candidate. I can’t even be bothered to go find a bunch of links. Suffice it to say that when I searched for “Bradley effect” on Lexis Nexis, there were 215 stories in the past 3 months.

We did our part to question the importance of the Bradley Effect. See this post on Dan Hopkin’s research. See also Adam Berinsky’s threepart post.

So what happened yesterday? First of all, Obama’s share of the vote—52% of the total votes cast—mirrored his standing in the polls at the end of the race (52% on Pollster). (The vote share data is obviously incomplete, but I can’t imagine it will change much.)

Second, if you look state-by-state, there is no evidence that he systematically under-performed viz. the polls. David Epstein of Columbia put together this graph over at Reflective Pundit:


He writes:

The projections were more or less spot on. There’s a slight upward tilt to the actual results, meaning that Obama did better than he polled in the states he won, and worse than he polled in the states he lost. On average, for all states, he underperfomed his polling by about 0.8%.
But we’re not necessarily that interested in whether he won New York by 25 or 26 points, or whether he lost Wyoming by 33 or 34 points. In the states where the last polls had Obama within 10% of McCain, in either direction, Obama actually outperformed his polling by about 0.5%.
Of course, this doesn’t mean that some people didn’t lie to pollsters because of being embarrassed to admit they were voting against Obama. It means that, overall, the polls did a great job at estimating outcomes, and there’s no evidence at all of a massive divergence between polled results and actual outcomes.

To be sure, the Bradley Effect cannot be isolated simply by comparing the polls to the election returns. As Daron Shaw pointed out at a Brookings seminar a couple weeks ago, the election returns are also affected by turnout and other factors. That’s why Epstein is right to note that his results don’t mean people never lie to pollsters about their support for black candidates.

Nevertheless, this election does suggest that the Bradley Effect deserves far less emphasis than it has received. Hopefully this will be a lesson carried forward in future elections.

2 Responses to Truths and Myths about the 2008 Election, Part II

  1. Marc S. November 5, 2008 at 1:53 pm #

    I was led to believe that the polls may underestimate Obama’s performance for two reasons:

    1. Most polls excluded cell-phones and thus missed cell-phone only voters who heavily favored Obama. Adjusting for this effect across relevant polls would increase Obama’s expected performance by 2-4% (though average of polls would be affected less).

    2. An excellent GOTV “ground game” is supposedly capable of improving performance over polls by 2-3%. Obama reportedly had one of the best ground games in history, and McCain’s was believed to be terrible.

    If the above two points are accurate, shouldn’t we have expected Obama to reach (conservatively), say, 56% of the popular vote? If so, what kept him at 52%?

  2. BillCinSD November 5, 2008 at 4:08 pm #

    not necessarily Marc. some of the polls that did not use cell phones tried to correct for the effect, so the effect is likely less than 4% and probably less than 2%.

    The presidential GOTV efforts are likely concentrated in battleground states, and the state and local parties also are involved in GOTV. in the past, the state and local GOTV generally favored the GOP, so Obama’s efforts may have only balanced the previous GOP lead.

    Next, this election was one that, for the first time, had large numbers of voters voting early. This would mean one needs to consider how the polls were trending in the 2 weeks to 1 month before the election.

    Finally, the polls average plus or minus 3% or so.