Obama’s Voter Mobilization Was Barely More Effective than Romney’s

by John Sides on May 7, 2013 · 15 comments

in Campaigns and elections

As part of our mini-symposium on campaign effects in 2012, we welcome this guest post by Ryan Enos and Anthony Fowler.  The first post in this sequence, by Michael Franz, is here.

*****

Does the 2012 Obama campaign deserve the hype?  Many journalists have praised the technological and strategic advantage of Obama over Romney’s campaign. The 2012 Obama campaign is often described as an extension and improvement of the 2008 effort, which has been credited as one of the first campaigns to make extensive use of rigorous academic findings about effective Get out the Vote (GOTV) and persuasion techniques.  Yet, as some academics have pointed out – the final vote margins were very close to predictions that would be obtained with no effect of the campaign, indicating that, perhaps, the new campaign techniques had minimal effects or that Obama’s advantage was neutralized by an equally savvy Romney campaign.   As part of an on-going research project, we use individual-level data to quantify Obama’s campaign advantage.

The data indicate that campaigns have large voter mobilization effects. By examining the individual behavior of voting-eligible citizens in the 38 media markets that span both battleground and non-battleground states (more than 42 million voters in total), we can isolate the effect of presidential campaigns.  These voters are subject to the same media exposure, but they differ in the amount of mobilization received from the campaigns.  Only voters in the battleground states were subject to the massive mobilization efforts of the Romney and Obama campaigns, thus allowing us to isolate the effect of the campaign.  And this difference is quite apparent in citizen behavior: registered voters in battleground states were, on average, 4.5 percentage points more likely to vote than their non-battleground counterparts in the same media market.

In many respects, this is an impressive aggregate effect of mobilization.  However, given the massive investment by both campaigns, some might have expected a greater effect.  Part of this effect may also be attributed to the psychological reactions of citizens in battleground states to the closeness of the election (the “p” term in the calculus of voting), which would be present regardless of the campaigns.  However, previous research suggests that the direct contribution of election closeness to turnout is small.

On the other hand, this 4.5-point result is an underestimate of campaigns’ reach because it includes voters that campaigns don’t attempt to mobilize. While registered Democrats and Republicans in these media markets (7.5 million and 7.0 million individuals, respectively) were heavily targeted by the Obama and Romney campaigns, others may not have been.  Among these select groups, we estimate massive campaign effects of 15.4 percentage points for registered Democrats and 13.8 percentage points for Republicans.   These numbers suggest impressively effective mobilization efforts by BOTH the Obama and Romney campaigns

Despite all the celebration of the Obama campaign’s technological and other superiority, their campaign only had a 1.6 percentage point advantage over Romney in turning out party registrants.  Of course, we don’t know whether Democratic registrants are inherently more difficult to mobilize than Republican registrants (although we find similar effects in 2008 before the Obama campaign had adopted the technological innovations of 2012, casting further doubt on the importance of these innovations).  Also, keep in mind that the 1.6 percentage point difference that we detect may have been enough tip the election in a very close state like North Carolina in 2008, so the Obama mobilization effort may have helped to pad Obama’s victory.  Nonetheless, these preliminary results suggest that the praise for Obama’s 2012 campaign may be overblown.  Both campaigns appear to have been effective in mobilizing voters, and the 2012 Obama campaign was not dramatically more effective than Romney’s campaign or Obama’s 2008 campaign.

Do these findings align with the expectations of academics and other people interested in politics?  In order to quantify just how large the Obama mobilization advantage was generally thought to be, we surveyed students, academics, political journalists, and campaign professionals by circulating a survey through our professional networks.   Our intention was to capture a convenience sample of people that are very interested in politics.  (We think we succeeded; we asked three political knowledge questions from the American National Election Study, and 94% of our respondents got all three correct.)  Our respondents were convinced of Obama’s superior mobilization, 95% thought the Obama campaign did a better job than the Romney campaign.  We asked respondents to quantify how much they thought Obama and Romney mobilized their supporters.  A clear pattern emerges: most respondents thought that Obama was far better than Romney in mobilizing his supporters, with a median estimate of a 4.9 percentage point advantage for Obama.

We summarize the responses of some groups in the table below (note that these groups are not mutually exclusive).  Political journalists and campaign professionals in our survey were the best at estimating the difference between the campaigns, but their estimates of the size of each individual mobilization efforts were too small.  Students and academics better estimated the size of the mobilization effects, but overestimated the size of Obama’s advantage.  Our personal predictions (before estimating the actual effects) were closer to the journalists’ predictions.

enosfowler


Interestingly, Republicans have both the largest estimate of Obama’s mobilization effort and the campaigns superiority over Romney.   This is, of course, not a representative sample of Republicans, but one interesting interpretation of this result is that Republicans are motivated to attribute Obama’s victory to superior campaigning, rather than Democrats having an advantage in underlying party or ideological closeness with voters.

Overall, the survey results confirms our suspicion that the vast majority of experts and political observers believe that Obama’s campaign was significantly more effective than Romney’s at turning out supporters.  Our empirical evidence suggests that even though presidential campaigns are very effective in this regard, Obama’s campaign – the most technologically advanced mobilization effort in American history – was not dramatically more effective than Romney’s.

{ 15 comments }

Michael McDonald May 7, 2013 at 9:57 am

This is a key line of the piece:

“Of course, we don’t know whether Democratic registrants are inherently more difficult to mobilize than Republican registrants (although we find similar effects in 2008 before the Obama campaign had adopted the technological innovations of 2012, casting further doubt on the importance of these innovations)”

The correct counter-factual is the baseline turnout propensity absent mobilization. And to respond to the parenthetical, since 2004, the emerging age-gap between Democratic and Republican voters may be an important confound to consider, since a baseline turnout propensity gap may be getting larger.

Let’s assume the analysis is correct: its difficult to evaluate since there is no link to any underlying data or analysis. So what? Obama may have needed a larger mobilization effort — which was directly measurable in the disparate resources the campaigns invested in their ground games — because Democratic supporters are more difficult to register and to mobilize to vote. I tried to tamp down expectations in the media. Here’s one of many pre-election quotes on this topic from me (this one in The Hill):

“The Obama campaign needs to have those offices because their supporters are the type of people who are lower-propensity voters,” said Dr. Michael McDonald, an early-voting and turnout expert at George Mason University. “The Romney campaign does not need as much of an organization as the Democrats do, because a lot of their voters are going to take care of themselves.”

Read more: http://thehill.com/homenews/campaign/251405-obama-holds-ground-game-edge-as-early-voting-begins-#ixzz2Sc9ddiwp

Furthermore, there is a missing piece of the ground game that is unaccounted for in the analysis here. Part of what the Obama campaign did was targeted persuasion, and the target included pockets of Romney supporters that the campaign micro-targeted. County-level data shows the Obama campaign was able to increase their performance over 2008 in some Republican areas of the country, like southeastern Ohio. Those targeted persuasion efforts may have been critical to the outcome.

Nadia Hassan May 7, 2013 at 10:19 am

Could that counter-factual be explored by contrasting battleground states versus non-battleground states?

Obama’s voters were also more likely to describe themselves as enthusiastic in 2012 than Romney voters in 2012, and probably compared to Democrats and Democratic leaning independents in 2008.

Michael McDonald May 7, 2013 at 10:50 am

This would be one logical way to explore the correct counter-factual: to compare those with and without the stimulus. Enos and Fowler suggest that this is their approach, by comparing media markets that span battleground states. I would need to know more about their data and methodology to know how well their research design was able to account for mobilization effects. For example, is this survey or partisan registration data (if the latter, I hope Ohio is not included in the analysis since many have erroneously thought Ohio has party registration)? Which states/media markets are in the universe (they mention North Carolina, and I wonder how much of the effect is attributed to this state since the campaigns withdrew)? How did they control for other organizations’ mobilization efforts, both within and outside the battleground states?

The reason I am skeptical is that we know there are now large age differences between Democratic and Republican voters, and we know age is among the strongest correlates to turnout propensities. This analysis suggests that the baseline turnout propensities of registered Democrats and Republicans is only different by 1.6 percentage points. That doesn’t pass a face validity test. Maybe independents are excluded from the analysis, but then wouldn’t those be important people to target with mobilization efforts?

Adam Schaeffer May 7, 2013 at 10:33 am

Michael . . . had many of the same thoughts as well.

Some other things to consider . . . Ohio and Virginia don’t have good Party id data, so those states are essentially missing from the registrant analysis (save for the small number of primary voters, who would be extremely high prob turnout in an election year).

Your note on a combined persuasion/turnout effort I think is potentially a very important part of this story.

Related to that, it is possible that the Obama campaign’s targeting of voters who aren’t registered with a major Party and of Republicans who are more likely to be cross-pressured or defectors was better. If so, they could have had much greater success turning out voters who prefer Obama, but this would not be visible in the analysis described here.

Voters who aren’t obvious Dems or Reps are also more likely to be marginal voters, and therefore the ones we’re most likely to see a big boost with during a very high salience election like 2012.

Michael McDonald May 7, 2013 at 11:02 am

There are many confounds if they analyzed registration data — not to say survey data is without confounds, too. You’ve mentioned two critical states — Ohio and Virginia — that would have to be excluded from a party registration analysis. And if they analyzed registration data, they are likely using the Democratic firm, Catalist. There are many issues with using those data that are not understood well by academics, or really anyone for that matter. While I’ve published previously on some of the issues with registration data, I’m now analyzing some rather unique data that I’ve collected from election administrators that, for example, reveal purging patterns between the time of the election and when vote history data are released.

Andrew Sprung May 7, 2013 at 10:17 am

Granting that George Washington probably couldn’t have beaten Obama, given what we now know about the economy’s performance in 2012, two caveats:

1) Is a 15.4 — 13.8 win as insignificant as you make it sound? Perhaps it’s roughly proportional to the extent to which Obama outperformed the fundamentals.

2) Might the Obama GOTV effort not have offset some Republican voter suppression measures?

Nadia Hassan May 7, 2013 at 10:21 am

George Washington probably could have beaten Obama without much trouble. By several measures, Obama wasn’t as advantaged by context as Gore in 2000 and Bush managed to squeak it out in the electoral college. Eisenhower also won by a larger margin than measures of economic performance in 1956 alone would have suggested.

Michael Smith May 7, 2013 at 10:53 am

Andrew– we have done work on the hypothesis that suppression laws and mobilization work in opposite directions. It is the Smith, Anderson, and Rackaway paper that you’ll find here: http://conference.mpsanet.org/Online/Search.aspx?session=3384

We are assembling out county-level dataset to test this further. Suggestions welcome!

bp May 7, 2013 at 11:11 am

The ostensible baseline of non-battleground residents that share the same media market with battlegrounders is not really a mobilization-free baseline. Presidential campaigns and the groups that support them attempt to mobilize individuals even in non-battleground states, especially if there are important down ballot races. Take the media markets that span New Hampshire (battleground) and MA (non-battleground), for instance. Do we really believe MA residents, who were not in the presidential battleground but certainly were in the Senatorial battleground, weren’t subject to intensive mobilization efforts? These confounds make it virtually impossible to “isolate the effects of presidential campaigns.”

Adam Schaeffer May 7, 2013 at 11:20 am

That’s a very good point, especially considering that the GOTV techniques used by the Obama campaign were actually pioneered by Progressive interest groups and Unions who would be active even where the campaign may not be. And many state-level Progressive groups have adopted the same methods as well.

Bear Braumoeller May 7, 2013 at 12:38 pm

My sense was that media accounts suggested, not that Obama’s campaign was more effective in terms of percentage or number mobilized, but in terms of targeting—that is, that they were more effective at figuring out *who* to mobilize. Is that not the case?

Dave May 7, 2013 at 5:39 pm

In Colorado, at least, I am not sure that a metric of turnout by registered Republicans vs by registered Democrats is totally dispositive as to effectiveness of turnout. Here’s why.

A. In 2008 Obama registration efforts appeared to be quite successful in closing the gap with Republican “active” registration numbers (which had enjoyed a large advantage). In 2012, while the initial gap in “active” registration numbers was smaller, Obama registration campaigns did not manage to narrow the gap to the degree they had in ’08.

B. Registration efforts by Obama’s 2012 team were focused on demographic groups, and they seemed to care less about getting new registrations to actually affiliate with the Democratic party than just getting them to register. A clear focus on registering college students, younger women and minorities was obvious in overall registration numbers, but less obvious when measured by Democratic registration gains. Anecdotally, for example, the usual grocery store registration table workers would shout out to urge minorities to come over and register, but an Anglo walking by would have all faces down, looking at paperwork on the table … with no effort made to get them to come over and register.

C. On election day the large number of “inactive” voters who re-activated by voting was quite evident. Places like Gunnison County (with a small college) and inner city Denver and Aurora (both with large numbers of minority voters) had long lines on election day. Again, it wasn’t the number of Democrats who appeared, but rather the number of target demographics who came out to vote.

D. A final way to see turnout effectiveness (Democrats vs Republicans) would be to compare the active voter registration edge. On the close of registration in October 2012, there were 40,744 more Republicans than Democrats. On 1/1/13 (by which time almost all activations/inactivations would have been received by the Secretary of State), there were 40,685 more Republicans than Democrats. Almost no change at all in which party was leading in active voters (almost all of whom retained “active” status by voting in 2012). Yet, despite this “no change” registration, Obama won with some degree of comfort in Colorado. That, to me, confirms that Obama-side registration/turnout efforts had gone beyond counting Democrats’ noses alone.

GOP May 8, 2013 at 2:04 pm

A win is a win. Romney and his team sucked.

Bob Kubichek May 9, 2013 at 12:57 pm

Let me preface this comment by saying I think this is overall a fantastic article. I stumbled across this piece while doing some research on inactive voter mobilization and I think it shines light on a few things the media really missed.

During the 2012 election I was actually the state director of Wyoming for the Obama campaign. So we were certainly not a battleground state. The thing I think this article misses (and really could not have known) is the way our border state program worked.

OFA Wyoming was not focused on winning Wyoming, because that would have been impossible and therefore an enormous waste of resources. Our work was entirely focused on mobilizing volunteers in Wyoming to make targeted phone calls into Colorado and to organize weekend canvassing trips to Northern Colorado. So I think looking at these things in terms of media markets may be a bad way of approaching the analysis. Put simply, there should not have been spillover effect from individual level targeted direct voter contact at the doors. I am walking out on a limb here, but none of our models indicated that such a thing would occur.

While traditional campaign media through TV, radio, and newspapers played a role in the persuasion work, we did not craft a strategy based in it working as an effective tool for voter turnout operations. Our GOTV operations were almost entirely door to door and phone based.

I think it may also be helpful to clear up some of the questions around the data we used. We did not use Catalist data, because Catalist data is mostly designed to be used for civic engagement work. I believe our data vendor was in fact Target Smart (or targetpoint I cannot remember now). We used NGP VAN’s VoteBuilder interface and trained every volunteer who was capable of turning on a personal computer how to use the system.

Also, the targets themselves were based on our 2012 Obama Support model rather than simple voter registration information. While I cannot go into the specifics of how this model was developed, I can say that it provided us with a much larger target universe than party identification would have.

I want to add that we actually expected this election to be even closer than it ended up being. We knew the Romney people were very smart and talented political organizers. So our goal was to build a machine capable of overcoming that challenge along with the natural challenges the President’s campaign faced.

Finally, I want to note that we did rigorous analysis on the effectiveness of our contacts and media work throughout the cycle. Our strategic decision making was based in data driven results, so the strategy itself was heavily steeped in a scientific method approach. Not that this in anyway validates our strategy, I just thought it may be helpful to note that our decision making process was not just based on media market importance.

Anyway, I think this is a great look at the work done during the 2012 cycle. I did voter behavior analysis in college and actually left a master program to go work in the campaign so I appreciate the work you folks produce.

Andrew June 6, 2013 at 9:21 pm

As Bob said, the only real way to evaluate this is by knowing who was targeted by each side. And even that’s not apples-to-apples because you’d have to control for baseline turnout of each group which is likely going to vary due to demographic factors like transience.

The closest you can come from data that’s been made publicly available would be to do state-by-state comparisons of how each side’s final vote totals compare to predictive models like the ones OFA published to demonstrate their accuracy compared to Nate Silver, et al: http://www.businessweek.com/articles/2013-06-06/obama-campaign-says-it-was-42-percent-more-accurate-than-nate-silver

Those margins could then be compared on battleground vs. non-battleground bases. That’s going to be complicated because not all non-battlegrounds were treated equal.

In all of the analysis it’s going to be difficult to use an after the fact study to draw accurate conclusions. Tough to control for paid media, direct mail, digital media, and persuasion tactics versus the GOTV tactics. And it’s especially difficult to account for the demographic differences demonstrated by each side’s voters.

Comments on this entry are closed.

Previous post:

Next post: