Cranky Reader Dislikes The Washington Post Election Predictor

by John Sides on April 24, 2012 · 5 comments

in Campaigns and elections

Cranky Reader: Sides, you are not only a moron, but a liar.

Me: You again?  Last time you were insulting me for predicting that Mitt Romney would be the Republican nominee.  How’d that work out for you?

CR: Don’t change the subject.  You said you weren’t an election forecaster, and now you are involved in this forecasting model with Ezra Klein and the Washington Post.

Me: Yes, Seth Hill, Lynn Vavreck, and I helped with that.

CR: Where to begin?  Okay, first, it’s not a very original model.  It looks like other some other forecasting models, like this one by Alan Abramowitz.

Me: It does look like other models.  The purpose of the exercise is not to come up with some Brand New Really Awesome Model.  It’s to give Klein and the Washington Post a plausible model that they could turn into a useful and fun widget.  As I’ve said, there’s no special secret forecasting sauce that this model, or any model, has.

CR: “Plausible model?”  Are any of these models really plausible?  Nate Silver says they “fail” at predicting presidential elections.

Me: I agree with some of Silver’s critique.  But even if some models don’t always do a good job predicting actual vote share, taken together all of the available models do a good enough job predicting the likely winner.  Overall, they have predicted the winner about 85% of the time in the predictions that Silver tabulates.  Ours predicts the winner 75% of the time.

CR: But Kevin Drum says he can predict elections even better just based on whether the incumbent party has served for one term or two, and so factors like the economy and presidential approval must not add anything.

Me: That last point is not quite true (see the aforementioned Abramowitz model).  But it’s fine if there are other models that can do as well or better than ours.  However, this misses the point of the exercise.  Let me pass the mic to Seth Hill:

Our model tries to balance prediction, which would include every variable you could find that correlates to the results in the past, with sparseness and theory. I can express the theory behind our three variables pretty well. We believe that voters respond to the state of the economy, but especially to the recent economy. We also believe that voters respond to a set of other factors separate from the economy, which are nicely summarized by presidential approval. Finally, we believe sitting presidents have a variety of advantages that make them more likely to win than non-presidents.

In other words, the goal is a model that draws on plausible theories about how voters make choices in elections, is simple, and is reasonably accurate.

CR: If your model is so accurate, why does the Washington Post predictor only tell me Obama’s chance of winning?  Why won’t you tell me his predicted share of the vote?  WHAT ARE YOU HIDING?

Me: Any predicted vote share—49.3%, 52.1%, etc.—is going to have significant error or uncertainty associated with it.  Error that arises because the model’s components are measured with error (it’s hard to know exactly how much the economy is growing or shrinking).  Error that arises because the “weights” associated with each component—that is, how much each component affects election outcomes—have uncertainty.  Error that arises because any one model is going to have prediction errors.  This is why we do 1,000 simulations of the model and then calculate the percentage of times that Obama wins.  This allows us to build in some of this error, and it keeps us (and readers) from getting hung up on the exact estimate, which isn’t going to be that precise anyway.  You can read more by clicking “Learn more about the methodology” at the bottom of the widget.

CR: Oh yes, I cannot wait to learn more about your fascinating methodology.  But look: I played with the widget.  It just can’t be right.  If GDP grows 2% and Obama’s approval rating is 50%, he wins 89% of the time!  That sounds too high.

Me: It sounded too high to me too.  And Klein.  This just reflects recent history: incumbent presidents presiding over even modest economic growth and with middling popularity haven’t been defeated very often.

CR: But I can think of lots of reasons this time might be different.  And your model doesn’t take those into account.

Me: Yes, the model doesn’t take many, many things into account.  We’ll see whether those kinds of things end up mattering in November.  Either way, the model and others like it will provide a helpful baseline and guide.  And may also help us ignore some of the distractions.  Like Klein says:

So sure, perhaps this year will be different — that’s what my gut tells me, and the model has me thinking about the ways in which that could be true. But the reality is, everyone always thinks “this year” will be different, and they’re usually wrong. That’s what the model tells me.  I am, however, confident that if this year really is different, it won’t be because of “Dog-gate.”

CR: Well, duh.  It will be because of Mitt Romney’s hair.  It gives us “insight” into “the candidate himself”!

Me: Okay, I’m done here.

{ 5 comments }

Adam April 25, 2012 at 8:17 am

I thought Klein’s explanation of the model was excellent, and shows that we’ve come a long way in our ability to communicate the nuances of forecasting to the public. Compare it to the Washington Post’s 5/26/00 article “Is this any way to pick a winner?” in which the caveats were buried deep in the story, while intemperate triumphalism about our awesome models took center stage. This led to the Meet the Press segment that weekend in which Russert, Carville, and Matalin laughed their tails off at the silly “eggheads.” I have that train wreck on tape, and still show it to one of my classes as a caution about how to (and how not to) interpret forecasting models.

del2124 April 25, 2012 at 11:47 am

Wait, is this is a real reader, or just some straw man?

John Sides April 25, 2012 at 12:08 pm

Ooh, the mystery is building!

Nadia April 25, 2012 at 5:58 pm

Does this mean that Mitt Romney was wrong to say last night, “It’s still about the economy and we’re not stupid” and would have a better chance of winning if he switched to some other signature issue like government or deficits? Lynn likes a nationalism message, but that’s just not Mitt Romney.

I am personally predicting a Romney win, because I think Obama has some very bad campaign strategies like tying Romney to Bush, which will fall flat and fail to meet voters where they are.

mclaren April 25, 2012 at 9:50 pm

Isn’t the first derivative of the GDP a better predictor than a simplistic “are you better off than you were four years ago” absolute level? And what about the second derivative?

In 1940 FDR got convincingly re-elected with higher unemployment and a worse economy than Obama has because the electorate recognized that things were moving in the right direction. The claim that “presidential elections are referenda on the economy” misses the likelihood that in a horse race you don’t want to abandon a horse who keeps finishing closer and closer to first as time passes.

Comments on this entry are closed.

Previous post:

Next post: