We have gotten a lot of feedback about “this post”:https://themonkeycage.org/2010/11/did_controversial_roll_call_vo.html on the effect of four controversial roll call votes–TARP, stimulus, health care, and cap-and-trade–on Democratic performance. Much of it questioned the decision to combine all the roll calls into an index, an approach that assumes each vote had equal impact. Is that a fair assumption? It’s an important question. John and I have explored it together, so we wanted to do a follow-up post that addressed the issue.
Some methodological details to start. We think it’s very important to limit things to contested Democratic incumbents. It’s meaningless to look at the effect of roll call votes on open or uncontested seats, since in either case the voters can’t punish the incumbent (the incumbent might have retired after casting a controversial vote, but that would suggest an even larger effect for these votes than we’ve already found). Moreover, with the exception of TARP, these four bills received very little Republican support. Running a regression on all races with a control for incumbency or party is asking a handful of Republicans to carry a lot of statistical weight. Finally, we treat any incumbent who didn’t vote either way on a bill (either because they abstained or because they were not in office at the time) as a “no” vote, since the real issue is whether you actively supported the bill. The ones who weren’t around to vote for it lucked out–they got the result without having to take a stand.
What if we run a model with these restrictions but with the four bills separated out, instead of as an index? (For the moment, we avoid conditioning on the competitiveness of the district.) Only TARP is small and insignificant in this model. The other three roll calls suggest meaningful vote losses for those who supported the bill: 2.8% for the stimulus, 2.1% for cap-and-trade, and 4.5% for health care.
What if we drop TARP and then condition the effect of each of the remaining three roll calls on the competitiveness of the district? (Conditioning TARP on competitiveness doesn’t change anything–it’s still not important.) The stimulus seems to hurt everyone. But health care and global warming show the effects one would expect: they matter more as a district becomes more Republican. If we look at the most Republican district represented by a Democratic incumbent who voted for health care–West Virginia’s 3rd, represented by Nick Rahall -the model suggests a loss of 4.7%. The same sort of Democrat for cap-and-trade–Ike Skelton of Missouri’s 4th–is predicted to have lost 5.7% for his support of that bill.
How does all this affect our prediction about seats won by Republicans? If we make every Democrat who lost reelection cast a “no” vote for all three bills, the estimated effect is about the same–a gain of 35 seats for the Democrats–but the margin of error around the effect is much larger:
The margins-of-error (MoE) for each estimate just cross, but because the MoE for a difference is slightly smaller than the sum of the separate MoEs, this is probably a statistically significant result.
A case can be made that the Democrats had no real choice but to support TARP and the stimulus–the economy might have been even worse if they hadn’t. But they clearly had the _political_ option to vote against both health care and cap-and-trade. What if we assigned “no” votes only on those two bills? The result is below:
Here the effect is smaller (about 24 seats) but more precisely estimated. This scenario suggests majority control would remain barely with the Republicans, though not with 95% confidence.
What can we conclude from all this? First, our earlier finding definitely stands: controversial votes hurt the Democrats, and vulnerable Democrats especially.
However, the effect was not consistent across votes. Instead, TARP had no effect, perhaps because it had been proposed by a Republican president (Bush) and had received a substantial number of Republican votes. There was some political cover for that one. But the other three all had important effects, and health care and cap-and-trade both hurt vulnerable Democratic incumbents more than others.
On balance, there’s good news and bad news for the Democrats here. The good news is that none of our estimates, including the first cut, suggests roll call votes definitely cost them the House, and if anything, this new analysis suggests an even weaker effect.
The bad news is that these roll calls definitely cost them votes, and probably quite a few seats. Moreover, it doesn’t help if the effect is limited to the purely “elective” bills–health care and cap-and-trade. Our estimates suggest Democrats would probably still lose the House without those two controversial votes, but the damage would have been smaller.
We should note that we still don’t control for a member’s partisanship (as opposed to ideology, which we tested and which didn’t add much). Jamie Carson, Greg Koger, Matthew Lebo, and Everett Young have some recent “research”:http://onlinelibrary.wiley.com/doi/10.1111/j.1540-5907.2010.00449.x/abstract (gated) that suggests party loyalty, not ideology, is the real issue. So these bills may just be capturing broader loyalty to the party, as opposed to these specific votes. Party loyalty scores were harder to get a hold of, so that analysis will have to wait until another day.
How do we gibe this result with the somewhat different results others have found? “Seth Masket”:http://enikrising.blogspot.com/2010/11/most-damaging-roll-call-vote.html finds that health care mattered for the most vulnerable incumbents but the stimulus and cap-and-trade did not. Likewise, “Josh Rosenau”:http://scienceblogs.com/tfk/2010/11/climate_change_legislation_did.php finds that health care had an effect but cap-and-trade did not. We can’t say for sure what produces the difference, though we would argue for the exclusions we described above–that is, limiting the analysis to contested Democratic incumbents–as a starting point for any comparisons. Both Seth and Josh use different exclusions. Beyond that, Seth’s “proposal”:http://enikrising.blogspot.com/2010/11/most-damaging-roll-call-vote.html that we drink about it seems wise. Heck, let’s drink even if we all agree.