What About Tomorrow’s Election Would Prove Me Wrong? (Plus a Prediction)

by John Sides on November 5, 2012 · 14 comments

in Campaigns and elections

To date, I haven’t made a formal forecast of the presidential election (though I will below). But I want to answer the question in the title of this post first, because it’s one that isn’t asked (or answered) enough.

Political science is more often about testing theories and explanation than forecasting the future per se.  So when I think about tomorrow’s race, I am first and foremost interested in updating how I view key theories, as opposed to whether any particular forecasting model, “mine” or anyone else’s, is “right” (more on that below too).

One interesting question is what tomorrow will say about the role of “fundamentals,” such as the economy, in presidential elections.  Such factors are not the sole determinant of election outcomes, but they do shape whether candidates enter the race, how they campaign, and who wins.  On balance, I have argued that the sum total of economic fundamentals favor Obama.  If he loses, then I will be “wrong” inasmuch as I’ve argued that economic fundamentals do a good job predicting elections. I put “wrong” in scare quotes because the economic fundamentals are always making a probabilistic prediction—so-and-so has a such-and-such chance to win—and so a Romney victory is certainly not impossible.  But when the “less likely” outcome transpires, it would and should give me pause.

Fortunately, from a social scientist’s point of view, either an Obama or Romney win would be instructive.  Regardless of who wins, we can use the 2012 election to help refine the underlying theory that connects economics to elections.  This year, unlike most years, specific economic indicators point in different directions: a forecast based on gross domestic product is more optimistic for Obama compared to a forecast based on disposable personal income, which is more optimistic for Romney. Perhaps tomorrow’s outcome—much as the 2000 election did—will help adjudicate among various indicators as to which best captures the dimensions of the economy that influence voters.

A second issue concerns the polls.  As you will see in a moment, my forecast is nothing more than an educated guess based on state polling averages.  This year’s polls have come in for considerable criticism, so much so that a bunch of links isn’t necessary.  Although I recognize that pollsters face real challenges—declining response rates, identifying likely voters—I have been reasonably confident in the ability of polls to predict election outcomes.  When Nate Silver debuted his pollster ratings in 2010, I noted that the amount of “pollster-induced error” was pretty small.  I’ve also summarized research suggesting that national polls, averaged during the week before a presidential election, are fairly accurate.  This year, the idiosyncratic decisions of individual pollsters appear to create fairly small “house effects.”  Moreover, and most important, there would need to be an unprecedented level of systematic bias in the polls for them to predict the winner incorrectly this year.

That means that a Romney victory would also prove me “wrong” by calling into question my sanguine attitude about polls and suggesting widespread errors that I failed to anticipate.  To be sure, there are a lot of reasons to do polls besides tracking a presidential election horserace, so a 2012 poll-pocalyse wouldn’t put survey research in the grave.  Still, a massive polling failure would challenge my own predispositions and lead me to reconsider them.

A final point, echoing something Dan wrote earlier. No formal forecasting model—that of Nate Silver, Drew Linzer, Sam Wang, DeSart and Holbrook, etc.—can be “disproven” by a single election.  Jay Ulfelder’s post about evaluating forecasting models should be required reading, since after Tuesday there will be a rush to crown (prematurely) the “winners” and “losers” among the models:

…we’re often tougher on forecasters than we should be. Instead of judging forecasters according to the entirety of their track records and comparing those records to a realistic baseline, we succumb to the availability heuristic and lionize or dismiss forecasters on the basis of the last big call they made…

…What we need to understand, though, is that this reflex means we often get worse forecasts than we otherwise might. When forecasters’ reputations can collapse from a single wrong call, there’s not much incentive to get into the business in the first place, and once in, there’s a strong incentive to be as ambiguous as possible as a hedge against a career-defining error. Those strategies might make professional sense, but they don’t lead to more useful information.


Alright, now that I’ve made you sit through that ponderous discussion, here’s my prediction.  My “method” is to rely on state polling averages—not because they’re always perfect, but because I don’t know of a more reliable way to make a prediction.  I could try to insert my own heuristics or rules-of-thumb, but absent systematic data, those could just reflect my own biases and misperceptions.  (I’m not saying that any prediction that doesn’t use polling averages reflects biases or misperceptions, just that I personally don’t know how to make predictions year-in and year-out and avoid those biases without relying on polling data.)

I took the Pollster map and simply assumed that the candidate with the higher chance of winning each state would win that state.  So NC and FL go to Romney and the rest of the toss-ups go to Obama.  That means: 303 Obama – 235 Romney. Here’s the map.  I did this in 2008, predicting Obama would get 378 votes when he actually got 365 (I missed MO and ND).

Incidentally, my gut tells me that Obama will tend to under-perform his polling numbers a bit—that is, I don’t think he will win by 3+ points in Ohio (it’s 49.1-45.8 on Pollster as I write). But I don’t have a solid rationale for that intuition.

Finally, because I agree with Alex Tabarrok that “a bet is a tax on bullshit,” I’ll wager a small sum. I’m not going to bet on my specific prediction, which could easily be off.  But I will bet on the “direction” of the result, which I expect will be more than just a narrow Electoral College win for Obama.  So if Obama gets 277 EC votes or fewer—which would mean, at a minimum, losing CO, NH, VA, and FL —I will donate $100 to the Juvenile Diabetes Research Fund in honor of my sister, who has had Type I diabetes since she was 11 years old.

After Tuesday, stay tuned to The Monkey Cage for analysis of presidential and congressional elections data.

{ 14 comments }

Doug Hess November 5, 2012 at 11:06 pm

You could wager on the popular vote (or the electoral college votes, but not in the way you discuss…i.e., individual states or total EC votes) by buying some shares on the Iowa Election Market.

Nadia Hassan November 5, 2012 at 11:23 pm

Hmm, Professor Sides on the question of economic indicators, didn’t the weighted RDI measure favor Gore? It was election year income growth that was only slightly favorable towards Gore. Nonfarm payrolls, aka monthly jobs report, especially in the middle of the year did not strongly favor Gore either. Seemingly, Main Street was among the first places where the 1990s expansion came to an end.

http://fivethirtyeight.blogs.nytimes.com/2012/05/04/has-obamas-magic-jobs-number-changed/

John Sides November 5, 2012 at 11:35 pm

Bartels and Zaller find that a model based on RDI produced a better 2000 forecast than a model based on GDP. A forecast based on GDP overestimated Gore’s margin.

Nadia Hassan November 6, 2012 at 12:09 am

In pp. 25 of that paper, Bartels and Zaller wrote that they found that RDI over an entire term seems to be less important in light of 2000 and RDI over election year seemed more important as models based on the former tended to be favorable towards Gore. Both are favorable towards Romney, but weighted income growth over the 1st three years of Obama’s term was historically low.

John Sides November 6, 2012 at 11:07 am

The point is that 2012, like 2000, might offer us some limited insight into which economic indicators are better predictors of elections.

Nadia Hassan November 6, 2012 at 11:54 am

I concur.

Simon Jackman November 5, 2012 at 11:31 pm

Cool. I’m flattered that the Pollster model-based averages played such a big role in your final call.

FL still a work in progress. Last (?) polls churning away in model now. We could well land with balance of probabilities favoring Obama, which would take the EV count up to 332. Mark B. (@MysteryPollster) and I are waiting for a run to end, some email to compare notes, and we’ll have 303 or 332 up as the predicted Obama EV number.

John Sides November 5, 2012 at 11:38 pm

Well, maybe I pulled the trigger too soon then! Oh well, I’ll stick with my prediction and, if your final run suggests Obama is favored in FL, I’ll conveniently make up an ad hoc reason why to bet against only one of Pollster’s averages.

Joe from NY November 6, 2012 at 10:31 am

On falsification of probabilistic predictions:
I agree that a Romney win doesn’t falsify Silver’s model. But what could? Since we only have elections every four years, it seems hard to find out whether deviations from his predicted winner are attribuable to a failure of his model or an upset. To some extent, I get the feeling that election prediction hypotheses are untestable… and thus not of scientific value (cf Popper). Thoughts?

Dave Blackburn November 6, 2012 at 11:01 am

There’s elections all the time. Silver is giving odds on like 85 races this year (each state, DC, and all the senate elections), plus he gave odds on the similar number in 2008, and a bunch of races in 2010.

It’s not too hard, then, to see how often the winner is X, based on what the probability of X being the winner was stated to be, and start to get some sense of whether the probabilities line up. So, you get only one draw from each “random variable” but with the underlying model being essentially the same in all of them, you can see if the draws you get line up with what you should get, in theory, if the models are accurate. It’s not perfect, because the model isn’t exactly the same for state races/senate/president/etc. and the model gets refined from year to year, but it’s doable.

Joe from NY November 6, 2012 at 11:44 am

This makes a lot of sense. I was only paying attention to the presidential forecast, so thanks for enlightening me.

Andreas Moser November 6, 2012 at 11:57 am
D. Thompson November 6, 2012 at 2:54 pm

Do your model factor in the impact of voter suppression?

Brice D. L. Acree November 6, 2012 at 3:34 pm

Spot on. Comes in at exactly the same forecast as the Margin of Error (economic fundamentals) model.

Comments on this entry are closed.

Previous post:

Next post: