Sam Wang has criticized our model of House elections that predicts a one-seat gain for the Democrats this year. (The title of his post is “Monkeying around with House models.” Get it? Get it?) He compares our model unfavorably to his own, which predicts a Democratic majority of 16 seats, for a gain of roughly 30 seats.
His main concern seems to be the uncertainty in our estimate, which he says makes it “meaningless.” He notes that:
[M]odels based on “fundamentals” (GDP growth, previous seat count, and so on) are research tools that set a range for what might happen before an election season starts…[But] at this point the best indicator of opinion is…measurements of opinion.
We don’t disagree on this point. We were very clear about our intent, which was less to offer a prediction per se, and more to see how far the fundamentals of congressional elections could take us. We also were very open about the uncertainty in our estimates: as Wang acknowledges, we devoted an entire post to it. Moreover, we readily acknowledged that a forecast based on polls or expert ratings might ultimately be more accurate:
Models that take into account the ongoing dynamics of individual races—either through spending, polls, or general handicapping—might do a better job of picking up [other developments].
So we’re not really interested in turning this into another round of forecasting model nerdfight. It’s absolutely worth trying to forecast House elections via polls, as Wang does or as do political scientists Joseph Bafumi, Robert Erikson, and Christopher Wlezien. Both of their forecasts draw on the generic ballot polling data. Instead, we want to talk about how these models differ, why they differ, and what one therefore needs to believe about 2012 in order to trust one or the other.
First, how do they differ? The predictions are roughly as follows:
* Our forecast: the Democrats will win 49% of the votes and gain one seat.
* Wang: the Democrats will win 51% of the votes and gain 32 seats.
* Bafumi, et al.: Dylan Matthews shows that the Bafumi et al. model predicts Democrats will win about 51% of the votes. This translates into a Democratic seat gain of 25 seats, based on the relationship between votes and seats since 1996, or an even larger gain (47 seats) based on the votes-seats relationship from 1946-2010. However, Bafumi, et al. have not, to our knowledge, run these numbers yet themselves, and since they have a complicated process of translating votes into seats that Matthews, who was doing something quick and dirty, doesn’t use, we’ll call this the Matthews, Bafumi, Erikson, and Wlezien estimate (or MBEW, for short).
The difference between these models isn’t really vote share: our 49% prediction is pretty close to the 51% of Wang and MBEW. Instead, they differ radically in terms of seat gains. We say a one seat gain for Democrats; Wang and MBEW expect dozens.
Why do they differ? We predict the district vote share, assigning each district as a win for one party and summing up all the results to a single national number. Wang and MBEW predict the national vote share. This requires them to find some way to translate their prediction into seats. They assume this translation–often called the “swing ratio”–will look like it has over a certain number of elections in the past. Wang uses the last six, while MBEW alternates between the last eight and the last 33.
The challenge is that the swing ratio has varied quite a bit over time. While it has been strong in the last three elections, it has frequently been much weaker, especially in the last few decades. This is evident from the wide range of predictions that Wang and MBEW get when the swing ratio is based on different election years.
Swing ratios are weak when a lot of races are uncompetitive. Apart from the fact that some districts are inherently more Republican or Democratic, the principal reason why races are uncompetitive is because incumbents are difficult to beat. This advantage protects a lot of incumbents when the broader political climate would otherwise sweep them from office.
In fact, this incumbency advantage appears to be the most important difference between our model and the other two. If we run ours with the generic ballot, we get roughly the same result as before: 49% vote and 45% seats. (The comparison here is a little complicated, since the generic ballot question has only been asked with frequency in recent years. We tried it over a variety of election years and the result was always about the same.) But if we take the same “generic ballot” model and pretend that there are no incumbents running for reelection and so no incumbency advantage, we predict that the Democrats will win about 50% of the vote and 50% of the seats. That’s closer to what the MBEW 8-election prediction would expect.
This brings us to the last question: what does this mean for our expectations in 2012? I think the lesson is simple: to believe that the Democrats can take back the House, you must believe incumbency will be weak this year. Is that reasonable? Well, it was certainly weak enough in 2006, 2008, and 2010, when dozens of incumbents lost reelection in the midst of broad national tides. Why not this year? Couldn’t voters be fed up with Tea Party Republicans and itching for a course correction? Absolutely. After all, our model performed pretty badly in the big swing years of 1994 and 2010 (though also really well in the big-swing years of 1974 and 2008.) Moreover, we could be missing a bunch of incumbents who are unpopular in their districts even if the broader political climate is largely status quo.
But some context here is important. The 25-seat gain Democrats need to take back the House would be historically anomalous. In the last 60 years of House elections, we have never seen more than three in a row with a net change of more than 20 seats. Nor have we ever seen a vote shift of the size (11%) that Wang estimates (though 2010 came close). Incumbency tends to rear its head eventually, and things settle down again until the next big surge.
It’s also worth considering some other forecasts. Pollster suggests that if the Democrats win every toss-up, they will gain 11 seats. We can also look at expectations of handicappers, which are often quite accurate. For example, take Larry Sabato’s House forecasts. If we assume that the Democrats win every seat that is solidly, likely, or leaning Democratic as well as half of the 14 toss-ups, then they would control 196 seats, for a gain of 3. If they win every toss-up, they would gain 10 seats. The Cook Political Report suggests something similar. If the Democrats won all of the seats they are favored to win, as well as every toss-up, they would gain 14 seats.
So while Pollster, Sabato, and Cook can suggest larger Democratic gains than our model predicts, they currently suggest little chance that the Democrats will take back the House. Moreover, their largest gains assume a sweep for Democrats of all the toss-up races; without that clean sweep, their predictions are entirely in line with our own.
So this time could be different. The public could be in a mood for yet another year of massive change. Lots of specific incumbents might be weak. Incumbency itself might have been permanently weakened after the turmoil of the last three elections. We don’t begrudge anyone the right to make those claims. They just don’t make for a safe prediction. The outcomes the generic ballot models currently consider most likely would be quite extraordinary. There is nothing wrong with a poll-based estimate, and we expect such an estimate may ultimately prove more accurate than our own. But a model that downplays incumbency is probably too optimistic for Democrats.