I have a theory

by Hans Noel on November 23, 2011 · 9 comments

in Campaigns and elections

I have a theory about the nerd fight over political science forecasting models.

So far, the debate has touched on a number of very interesting issues, but the thrust is whether political scientists are “too confident” in the models that predict the outcomes of elections on the basis of the economy, or whether there is room for the campaign to “matter.” The conclusion, which most reasonable people seem to agree on, is that of course there is room for the campaign to matter, but the fight continues over what that means about the importance of the economy.

But I think the main source of conflict is a different perspective on why one might run these models in the first place. (And some political scientists may share that perspective.) Political scientists are not in the business of predicting the future.  (I never thought journalists were, either.) Forecasting is good because it keeps us honest. Predicting outside the sample ensures you are not inventing an ad hoc explanation. But models built just to predict are not the point.  Generally, political scientists do statistical modeling because we want to test a hypothesis. And we want to do that because we have a theory.

For instance, we might theorize that voters reward candidates or parties who have served them well, and punish those who have not. One way to measure a high-performing president is by looking at the state of the economy. Another is to look at foreign policy failures, especially wars. Theory says we should think about measures that might be felt by ordinary voters, so disposable income is probably better than GDP, and American casualties are probably a better measure than total casualties. These are the variables in the very influential Hibbs’ Bread and Peace Model.


The model performs well. And from that, we conclude that the theory is likely true. We don’t learn who will win in 2012, because that’s not what the model was for. We learn that the theory is likely true. We then go on to test the theory in many, many, many other ways. We start to believe that the fundamentals of the election, things that have little to do with the campaign and everything to do with what has happened in the last four years, matter a great deal.

This does not mean that nothing else matters, because we have not built a model that attempted to maximize the amount of variation we are explaining. We built a model to test a theory. There are other theories, of course. One is that voters vote primarily on the basis of ideology. This theory has generated mixed results. It predicts, for example, that both parties will nominate centrists. and while presidential candidates are rarely very extreme, candidates from all levels of office are also rarely centrist. So while we think voters do punish candidates who are too extreme, we think parties can get away with nominating someone who is not moderate.

And when we go to test the effect of campaigns, the results are again at best mixed. Campaign advertising definitely moves people, but the effect decays rather quickly, and meanwhile, there are a lot of competing messages. Most of the campaign gaffes that fascinate political journalists and political scientists are often completely unknown to most voters. And the most attentive voters are highly partisan, so they filter those events anyway. In short, while no one thinks the campaign is meaningless, there is little reason to believe it can have the kinds of effects that many attribute to it.

One could build models designed just for predicting the future, and some scholars do. If so, one might include, for example, presidential approval as a forecasting variable. That variable “predicts” very well. But it is theoretically uninteresting. The variables we want to test, like the economic variables, will also affect presidential approval, and so including it hinders our ability to estimate the effect of the variables we care about. Such a model might be great at predicting the future, but it’s lousy for social science.

{ 9 comments }

Comments on this entry are closed.

Previous post:

Next post: