The Pro Bowl of Political Science

Over at FiveThirtyEight, Nate Silver and the Monkey Cage’s own John Sides have an exchange about Nate’s evidence on the inaccuracy of Presidential election forecasts based on economic fundamentals.  The exchange puts a lot of valuable points on the table.  Still, I worry about what it might imply to readers outside our discipline about the prominence of Presidential election forecasting within it.

Simply put, the forecasting of Presidential elections is not among the central contemporary projects of political science.  The work forecasting Presidential elections is rather like the NFL’s Pro Bowl: sure, it involves several of the discipline’s most prominent researchers, but it isn’t typically what makes those researchers prominent.  And there is a reason for that.  Since the first election of Franklin Roosevelt in 1932, we have observed exactly 20 Presidential elections.  For statistical modelers, that is not exactly a large number.  What’s more, some of the same candidates stand in multiple years, meaning that elections like 1940 or 1972 aren’t independent observations.  Over this same period, we have seen pronounced changes in our political parties and in our electorate that are the subject of many books, not a single blog post.  All this is to say that forecasting any one U.S. Presidential election is inherently a difficult enterprise—and one that accounts for a small fraction of the work political scientists are doing today.  (This is among Jonathan Bernstein’s many reactions here, too.)

My intent is not to devalue the forecasting literature—if anything, pointing out how challenging this is should do the reverse.  There is tremendous scientific value to making public forecasts, as they force researchers to commit to a specific model before outcomes are known and thus guard against models that are predictive by chance alone.  The forecasting literature seems uniquely committed to this approach.  But nor should the Americanist subfield of political science be judged primarily by its ability to forecast Presidential elections.  Since starting graduate school, I have been to somewhere north of 400 research presentations by political scientists.  Of those, the number that were on forecasting Presidential elections?  Two.

4 Responses to The Pro Bowl of Political Science

  1. Andrew Gelman April 2, 2012 at 9:08 am #

    Dan:

    Steven Rosenstone and Doug Hibbs are prominent political scientists, and election forecasting is a major part of both of their work.

    In addition to election forecasting itself, the existence of accurate election forecast implies interesting things for politics, as King and I discussed in our 1993 paper. So I wouldn’t be so quick to dismiss the enterprise.

  2. Dan Hirschman April 2, 2012 at 10:47 am #

    “What’s more, some of the same candidates stand in multiple years, meaning that elections like 1940 or 1972 aren’t independent observations.”
    Are you implying that if candidates only served one term and didn’t run for re-election, the results of each race would be independent? It seems like the choices made by the winner of an election at time 1 affect the chances of candidates in the election at time 2. The next candidate may well be the VP or Secretary of State of the past winner, or their spouse. Either way, the incumbent party runs a somewhat different campaign, even if the candidate changes. So, don’t the failures of the independence assumption run deeper than just seeing the same candidates multiple times?

  3. Jonathan Ladd April 2, 2012 at 11:50 am #

    Andrew,

    I don’t certainly don’t think forecasting results should be dismissed. The fact that we can predict the popular vote margin without knowing anything that happens during the actual campaign tells us something very important about elections and voting.

    But I also think that it would be unfortunate if outsiders got the impression that forecasting is what modern quantitative political scientists mainly do. There is relatively little new ground being trod in forecasting right now. You don’t see ABDs doing job talks based on new forecasting models and you don’t see papers on forecasting models in the top-3 political science journals. You did in the 1980s, but not now.

    Most people agree with the main results from the forecasting literature, but there are a lot of other ways to also study economic retrospection, spatial issue voting, and the other theoretical issues implicated in the forecasting literature. Forecasting models are one way to test theories in these important domains, but there are many others as well. Most of the recent advances on this important topics have come by using methods other then forecasting models. There is much more to political science than election forecasting.

  4. Dan Hopkins April 2, 2012 at 1:39 pm #

    @ Andrew — Thanks for the comment. I think Jon’s comment captures the thrust of my response, which would be to say that I don’t dismiss the forecasting literature at all, and I agree that has been quite important in shaping the field. But outside the discipline, I think the prior belief is that forecasting is a central component of what we do–and I want to nudge that prior a bit. (Also, I made sure to say “typically” to allow that yes, there are folks who are centrally forecasters.)

    @ Dan — Thanks for the comment. I was just giving an example of non-independence, not suggesting that it was the only such source.