The latest rounds of “nerdfight” — Seth Masket vs. Sean Trende, Nate Silver vs. Ron Klain vs. Nate Silver vs. Brendan Nyhan, and also Harry Enten — are instructive in many ways. I want to weigh in with some points that I feel are getting short shrift.
A lot of the discussion has centered on the value of particular forecasting models — like Silver’s, those in political science (e.g., Alan Abramowitz), those by economists Ray Fair and Douglas Hibbs. Silver has raised concerns about whether some models are overfit and whether forecasters are underestimating the uncertainty in their predictions — in part because some of the factors in these models, such as different measures of the economy, are measured with error. I think those concerns are justified. At the same time, I also think there are defensible ways of averaging models so that we aren’t too dependent on any one of them. Nyhan and Jacob Montgomery do this here; see also this paper (pdf) by Larry Bartels and John Zaller. Silver dismisses this as a “game show” but I think that’s a little unfair. After all, the average provides a more accurate forecast than individual models. And it should also be possible to design an averaging scheme that down-weights models that do not abide by good forecasting principles– e.g., by overfitting.
But this focus on forecasting models is obscuring some more important points. Let’s begin by leaving aside the specific task of forecasting future elections and just think about explaining past elections. My starting point on this blog, virtually from Day 1, was this: Much reporting and commentary about American elections, and presidential elections in particular, overestimates the impact that campaigns have and underestimates the role of structural forces, such as the economy. I agree with Masket that progress has been made, but not enough. Why? In part, to get to the title of the post, because of 3 underemphasized points:
1) Many people still misunderstand how the economy matters. As best we can tell, trends in economic indicators matter more than the actual levels of these indicators. This is something Klain gets wrong in his discussion of the 1984 election when he wonders how the economy could have helped Reagan if unemployment was 7%. Specifically, it is the trend in the year before the election seems the most consequential. Silver’s post corrects Klain’s mistakes.
2) Then there is this underappreciated fact: evidence that the economy matters does not just come from forecasting models. It also comes from survey data, which often demonstrate that people’s perceptions of economic conditions are correlated with the vote choice. I use the term “correlated” deliberately, because economic perceptions are not divorced from people’s partisanship and thus it is difficult to prove that these perceptions cause vote choice. But nevertheless, multiple strands of evidence — from aggregate economic data, from survey measures of economic perceptions, from forecasting models — confirm that the economy is related to election outcomes. Any flaws in forecasting models don’t make this finding go away.
3) People treat the economy and the candidates and campaign as if they are separate factors, which gives a lot of these debates an unhelpful “either/or” framing. This is mistaken for two reasons. First, the characteristics of candidates and campaign strategies themselves depend on the economy. Better candidates will challenge incumbents when the economy makes those incumbents vulnerable. Whether candidates campaign on the economy will depend on whether they will get credit or take blame for economic conditions. Second, as Andy points out, factors like the economy often come to matter precisely because they are emphasized in the campaign. That is to say, the campaign “makes” the forecasting models come true. Klain asserts that campaigns must matter more today than in the 1970s simply because more money is being spent, and he holds this up as evidence that the economy matters less. But in fact, all this additional campaign spending might make the economy matter more.
In short, the economy is bound up with campaigns and elections in may complex ways. You cannot simply predict election outcomes with change in gross domestic product, conclude that the economy explains, say, 30% of variation in election outcomes, and then declare that the majority of variation is explained by campaign spending and strategies. There is no simple way to separate the total effects of structural forces like the economy and the total effect of the campaign itself.
So, confronted with debates about the economy vs. the campaign and the challenges inherent in separating their effects, what’s the average politically attentive person to do? Here’s a very practical question: Between now and Election Day 2012, should people be more skeptical about claims that the economy affects elections or claims that campaigns affect elections? They should be more skeptical about the latter. Whatever the problems with specific forecasting models, the former rests on solid evidence. There is also evidence that campaigns matter, but the problem is that most claims about campaign effects in political journalism and commentary provide little to no evidence whatsoever. Changes in poll numbers before and after a discrete event can be suggestive and perhaps even persuasive. But many other kinds of campaign strategies — e.g., television advertising — contain few discrete events. So their impact can rarely be assessed as the campaign itself unfolds. That is why I am so often bird-dogging claims about campaign effects.
When the 2012 election is over, there will be time to unpack how much economic conditions hurt or helped Obama and how much the campaign may have affected the outcome. In the meantime, however, we don’t have to debate straw men or get lost in the weeds of forecasting models.