Interpreting a before-after change as a causal effect: campaign spending and the Wisconsin election

John quotes Seth Masket’s claim that it didn’t matter that the Republican outspent the Democrat by 7-1 in the recent governor recall election in Wisconsin. Masket writes:

Walker and Barrett faced each other less than two years ago. Walker beat Barrett by five points back then, after raising $11 million to Barrett’s $6 million. That is, Walker raised 65% of the funds raised by the Republican and Democratic candidates that year and he won 53% of the two-party vote. This week, Walker raised about 88% of the funds raised by the two candidates and he won—wait for it—54% of the two-party vote.

So there’s your money effect, folks. Go from a 2:1 money advantage to a 7:1 money advantage, and it could increase your vote share by a full percentage point! Woo hoo!

I really really really don’t like this sort of snappy “woo hoo” reply. Nor do I like the subtle minimization of the effect (note how Masket writes “it could . . .”, thus implicitly taking the 1 percentage point as an upper bound rather than an estimate of the effect).

There’s been a lot of research showing that money matters in campaigns, but more so in nonpartisan contests such as referenda and less so in highly partisan contexts. I think that’s the way to address such questions. Not by taking a single before-after comparison and treating it as a causal effect. That’s just sloppy.

That said, the numbers from a rematch election are relevant, as long as you don’t step off the cliff by taking the before-after difference and interpreting it as a causal effect. Masket criticizes a news report for saying that “this is the biggest story in the campaign so far: Money matters.” We already know that money matters, so you could argue that the news reporter is overinterpreting this one election—-but I really don’t like this idea of directly interpreting a before-after difference as a causal effect. As political scientists, I think we should be a bit more careful with our snappy rejoinders.

13 Responses to Interpreting a before-after change as a causal effect: campaign spending and the Wisconsin election

  1. meh June 7, 2012 at 4:29 pm #

    Incredibly lazy analysis. Totally ignores the fact that Walker’s numbers were in the toilet and a hypothetical rematch showed a tie BEFORE the incumbent went on his 5 month run of air superiority.

    If you are going to be snarky, at least be right.

  2. Seth June 7, 2012 at 5:36 pm #

    Sorry you didn’t like the snappiness, Andrew. I certainly wasn’t trying to suggest that two data points could give us an accurate estimate of the overall effect of money on elections. My main point was to object to the idea, which I’ve heard from multiple sources, that Walker “bought” the election. I think that if you’re going to claim that, you need to give some idea of what Walker’s vote share should have been if not for his spending blitz. That’s obviously hard to say, but his vote share against the same opponent two years ago is probably not a bad baseline.

    • Andrew Gelman June 7, 2012 at 6:30 pm #


      Yes, I agree that a rematch is a good comparison point, I just wasn’t so happy with the apparent certainty that the effect was no more than one percentage point.

      • Brian Schmidt June 8, 2012 at 1:54 am #

        More fun with numbers – if Barrett had had equal money instead of being outspent 7-1, then Walker would still have won with a little over 52% of the vote. If Barrett had a 3-1 advantage instead of 7-1 disadvantage, Walker would still have won with over 51%.

        I can do this all day. Of course some might consider this analysis to be a little deficient.

    • MtnCaddys June 8, 2012 at 7:15 pm #

      “…his vote share against the same opponent two years ago…” is comparing apples to rotten oranges. Walker won 53% of the vote BEFORE he pulled all of his union-busting stunts (the “rotten oranges”) Do you seriously believe that he didn’t need $30 million to retain his Governorship?

  3. Peter June 7, 2012 at 6:41 pm #

    I would love to see the Monkey Cagers devote some more attention to the Wisconsin recall. There were a few bright spots in yesterday’s media coverage, but they were lost in the rush to describe the race as a portent for November or as bad news for Obama. Neither seemed right to me given the exit polling that 70% of voters thought a recall just wasn’t appropriate. Are there take away lessons we should be learning here?

  4. Snark June 7, 2012 at 6:52 pm #

    If we can agree on one thing, though, it is that Citizens United forces Political Scientists to make unwarranted snap judgments about big money in politics with insufficient evidence, and pits them against each other. If it weren’t for the influence of big corporate donors, nobody would bring down The Monkey Cage with this stuff. In this post-Citizens United world, this kind of flare up will come to define the discipline, and this blog. It is all Citizens United.

  5. John Sides June 7, 2012 at 10:26 pm #

    If you factor in the outside group spending, Walker outspent Barrett by more like 2-1, not 7-1:

  6. Stephen Baird June 7, 2012 at 10:47 pm #

    I would like to determine when both the Barrett and Walker campaigns spent most of their money. The first Marquette University poll conducted after Barrett became the official challenger shows Walker with a 8 point advantage, 51 to 43%, with 3% undecided. If both campaigns allocated most of their funds in May, it would be difficult to claim that Walker “bought” the campaign, since Walker finished with a 7 point lead over Barrett.

  7. Chris June 7, 2012 at 11:53 pm #

    “The spending story stems from a release by the liberal Center for Public Integrity, which took based its analysis on data from the Wisconsin Democracy Campaign (ostensibly nonpartisan, but a past recipient of Soros money and the sort of group whose director told CPI the spending was “outrageous and wrong”). However, according to that data, when you combine the spending of the candidates and their supporting groups, the gap shrinks to 2-to-1.

    Moreover, it is a fair bet those figures do not include all of the money spent by left-leaning groups on all candidates in the recall.

    Indeed, it should be underscored that the left/media here is focused entirely on spending in the gubernatorial recall, when this election was just one of many the left attempted to turn into referenda on Gov. Walker’s public-sector collective bargaining reforms. If once considers the total amounts spent during the Days of Cheesehead Rage on state senate recall elections, Supreme Court elections and so on in 2011-12, the gap shrinks to roughly 1.5-to-1.

    Lastly, these figures only account for sums legally required to be publicly reported (and assumes those sums are properly quantified). Rutgers University economist Leo Troy has estimated that actual union political spending is likely several times higher than generally reported. There is no reason to think otherwise in this case.”

  8. RacineAl June 8, 2012 at 12:13 am #

    Is the goal of this article (a) to open debate on rebutting the claim that money didn’t matter here, or (b) to just complain about some snark, or (c) to undermine subtly the money didn’t matter claim by complaining about the snark, perhaps subconsciously? (e.g., my side lost, so I’ll pick at the margins of some guy touting something for the other side that I know doesn’t hold up to rigorous analytical standards so I can feel superior? I have as much dislike of that behavior as I do sloppy statistical analysis)

    (b) seems pretty picky, given that it could be directed far more accurately at the ridiculous 7-to-1 claims running around, which the author interestingly didn’t choose to do.

    So I’ll go out on a limb and pick (a) or (c) … (c) isn’t a topic worth discussing. So on to (a) I will go …

    IMO, besides GOTV money (which I argue is not an issue when not issued in the form of a bribe, which I will assume hopefully is relatively rare when it comes to direct voting)

    (1) Why Money (after a certain level) May Not Have Mattered – One poll cited 88% of voters had already made up their mind at the start of the recall process (I don’t have the exact details of the poll, so let’s consider it hypothetical vs. factual, I can’t do any source verification).

    If true, then most of the spending was over shifting 12% of the vote, let’s say within a band of 7.5% / 5.5% for-against to 5.5% / 7.5% for-against (because money isn’t going to be a clean sweep of undecideds, it never is). Influential yes, but not an “election buy”. Now, it could create bigger swings — but there weren’t a lot of issues or ad buys or “shady” election-surprise ads that influenced things that I saw.

    It could be that it bought get it out the vote, but if it didn’t buy votes (and I really have seen no evidence of that), then I think supporting voter turnout is a good thing. I can’t see anyone being against participation in principle, even if its partisan turnout support.

    (2) 88% is plausible, IMO – I think this is plausible, because (a) these two already ran a high-profile campaign against each other (no big surprises in a re-run), (b) these issues played out nightly for over a year, through a recall campaign and a proxy recall election as well on the same issues (everyone knew what was going on).

    Further, as a hypothetical point, the recall wasn’t predicated on the performance of the economy, so I can plausibly see the Walker jobs kerfuffle discussions at the end as not being a real mover vs if that was a driver behind the recall … hence the money spent on ads around that issue couldn’t move voters (but could in another election). I have any evidence to support this, no – feel free to dismiss … but there was hard evidence on how Wisconsinites felt about the recall, and that says to me to stray from the core issue of the recall was going to have a more muted effect than in a general choice election. No hard evidence, but not a huge leap of faith.

    Also, frankly, I’m surprised only 88% had already made up their mind. (Yes, I’m being snarky, shame on me). My informal polls showed 100% commitment among my circle of overinformed friends. (worthless survey, I know)

    (3) DailyKos (hardly a hotbed of tea party conspiracy) published an analysis on why they think outside money did not buy the 2010 house for the Republicans this week.

    The analysis has no relevance to Wisconsin except for one point. (And they don’t say money doesn’t matter, they said it matters less than we think).

    The most important thing to take away is … you can’t move forward if you don’t learn the right lesson about why you lost. And that a lot of people are running around the Democratic party with this meme, and have been for a while.

    That, I get. Citizens United was a great unilateral disarmament tool to preserve union power (but goes only on one side of the aisle) but disarm corporate power, which can go either way (but perhaps goes more towards the other side of the aisle).

    But don’t confuse tactics & memes with the truth of the election drivers.

    And when I was taught root cause analysis in applied statistics (for industrial operations), you used many principles, from statistical analysis, fishbones, and “The 5 Whys”, which was the simplest. And that one stated you never asked “Why” got an answer, and stopped. You had to ask Why 5 times, peeling the onion back further and further, before you had a chance of understanding what was really going on. I have found it is generally useful to keep asking Why? and not stopping when you get to the answer that confirms your bias.

    Why? Money! End of Story. It feels like Dug and the ubiquitous “Squirrel!” line in Up. We ain’t going to get better if we keep falling for that one.

    (4) Plausible Other Reasons Exist – Non-union voters might have actually liked the outcome of the reforms – you can express your support for union rights all you want, and still vote your pocketbook in the booth. One poll showed over 25% of union members voted for Walker. The reforms were in place long enough to have an impact and keep either side from distorting the record a whole lot on the reforms credibly. Maybe the reforms might actually be a bit more popular than AFSCME wants to admit, since they lost something like 65% of their members (it was not a uniform effect – some unions lost vastly more than others, so there are some union specific drivers of continued maintenance, but I have no idea what they are).

    More compelling possibly, people were sick of this and didn’t feel the recall should proceed on a basis of non-misconduct related stuff.

    No one of these has to be the answer. 15% for reason A, 23% for reason B, add it up and you have a classic multi-driver causal analysis ready for you.

    Turnout was very high, so apathy by one side is not a really plausible alternative.

    (5) 7-to-1 is a canard – The 7-to-1 nonsense is a dangerous myth. If you ascribe the answer to why you lost to a non-existent root cause, then you won’t ever know the truth. Or face up to the change you need to win the next election, in this case. If anyone honestly believes the 7-to-1, get out a bit more. Read both sides-of-the-debates articles openly. Only then do you get to the root … The sober analyses I have seen show a substantial edge of 2 to 1. Key, but not a runaway. Ed Schultz does not represent serious Blue Team analysis.

    Given the money spent, anything over a few million spent on ads was probably wasted IMO, in the UNIQUE circumstances of this election — except for making their side feel like they are doing something, doggone it! (GOTV investments, though, I might think about differently – some of those have an ongoing value).

    Now again this analysis is unique to this election so I’m not making a care for or against money in elections, or for or against money’s influence in other elections, etc etc. But governor recalls are very rare, so any analysis should be unique anyway!

    My Best Guess – In this case, I think if Democrats run around blaming money HONESTLY believing that is the issue, they are deluding themselves, and it will cost them in the future (not that it doesn’t matter – just for this recall).

    If some analysts are (as I think) running around pushing the issue (a) to influence discourse around Citizens United and money in elections in general, and (b) rev up the base by nurturing a sense of David vs. Goliath and injustice, then OK, I get it. Its a useful tool.

    But let’s not use a point-of-persuasion for other purposes as the starting point for a serious analysis.

    Feel free to pile on.

  9. Alan Abramowitz June 11, 2012 at 11:20 am #

    Andrew is certainly right that a simple before-after comparison is too simplistic. But there are other reasons to think that money was not the key factor in this result. First, spending by incumbents generally has less payoff than spending by challengers. Second, spending by incumbents or challengers has diminishing marginal returns. Third, the electorate in Wisconsin was deeply divided with few swing voters available for persuasion. Fourth, without a presidential race on the ballot, turnout was tilted toward the GOP as it usually is. And fifth, a good number of voters just didn’t agree with the idea of recalling a sitting governor in the absence of evidence of misconduct in office, not just policy disagreement. So we can’t say that spending had no effect on the outcome, but the effect was probably fairly small.