In the spirit of John Sides’s selfless offer of himself and David Park as regression-runners and findings-interpreters for New York Times columnist David Brooks (HERE), I hereby volunteer John and David’s services to ABC News political analyst George Stephanopoulos as well. (I myself am much too busy blogging about cats to become involved in such an undertaking.)
This offer is prompted by the following exchange between anchorman Charles Gibson and Stephanopoulos on May 6, 2008, the evening of the Indiana and North Carolina primaries:
bq. GIBSON: And joining me now is our chief Washington correspondent, George Stephanopoulos, who is here in New York tonight. …
bq. STEPHANOPOULOS: … Let’s look at those numbers. … We did ask a question I know in the exit polls about Reverend Wright, Barack Obama’s former pastor and whether that was influencing voters. What did we find? Right down the middle. About half said it’s important, about half said it was unimportant. Of those who said it was important, look at this in Indiana, 70% went for Senator Clinton. Of those who thought it was unimportant, again right down the middle, 65% for Barack Obama. So what you thought about the importance of Reverend Wright basically determined your vote.
Now, George Stephanopoulos, who graduated summa cum laude from an Ivy League school (Columbia), is no dummy. The concluding sentence of his summary of the survey results is so fundamentally flawed, though, that one can only wonder what they were teaching in all those political science courses that he took (he was a political science major) or whether the time he spent in the Clinton White House so addled him that it’s all gotten jumbled in his head.
The most basic lesson one is supposed to learn in one’s very first research methods course is — repeat after me, everybody — “Correlation is not the same as causation.” Now, with that thought in mind, go back and re-read the offending sentence: “So what you thought about the importance of Reverend Wright basically determined your vote.” Do you begin to see the problem?
The causal spin that Stephanopoulos put on these survey results assumes, in effect, that heading into the Indiana and North Carolina primaries most voters weren’t committed to one candidate or the other. Then the Reverend Wright issue emerged, and — presto! — it split the Democratic electorate into two groups, Clinton voters and Obama voters.
Instead, let’s begin with a decidedly different, and altogether more plausible assumption. Let’s say that heading into these two primaries, which after all occurred very late in the primary season, most voters were already pretty well committed to one or the other of the two candidates. Then the Reverend Wright issue emerged. For many of those who were already intending to vote for Clinton, that issue reinforced their existing inclination; certainly it wouldn’t have transformed them from Clinton supporters into Obama supporters. For many (presumably most) of those who were already intending to vote for Obama, though, the Wright issue wasn’t sufficient to persuade them of the error of their ways and move them into the Clinton column.
Then on election day along came an exit poll interviewer asking them how important the issue had been to them. Naturally, many of the Clintonites cited it as a reason for voting for Clinton rather than Obama — because it had been consistent with that intention and also because it had also emerged late enough to still be retrievable from memory. But how many Obama supporters would cite it as a reason for having voted for Obama? As the issue played out, that simply wouldn’t make sense; it was a one-way or “valence” issue, and its valence was anti-Obama. So Clinton supporters were more likely to cite the Wright issue as underlying their vote choice than were Obama supporters — the same statistical finding that Stephanopoulos reported but with a very different interpretation.
The problem, stated more generally, is that when pollsters ask voters which issues were most important to them, what the voters tell them can’t be taken at anything approaching face value. People just aren’t reliable reporters of their own mental processes. (The classic statement of this point is Nisbett and Wilson’s “Telling More Than We Can Know: Verbal Reports on Mental Processes,” Psychological Review 84 (May 1977), pp. 231-259.) In the context under consideration here, this unreliability typically takes the form of ex post facto rationalizations: Those who were predisposed to vote for Candidate X cite the issues that X had raised as the decisive ones, and those who were predisposed to vote for Candidate Y cite Y’s issues as decisive. Look back over several decades of ANES post-election surveys and you’ll see this pattern time and time again. Or save yourself the trouble and hunt down a copy of Wendy Rahn, Jon Krosnick, and Marijke Breuning’s “Rationalization and Derivation Processes in Survey Studies of Political Candidate Evaluation” (American Journal of Political Science 38 (August 1994), pp. 582-600), the bottom line of which is that “Voters’ reports of the reasons for their preferences were principally rationalizations.”
My point isn’t that the Reverend Wright matter was wholly inconsequential. It may well have figured into the decision calculus of some late-deciders in Indiana and North Carolina. But that’s a world away from concluding based on the data that Stephanopoulos had in hand that “What you thought about the importance of Reverend Wright basically determined your vote.”
Columbia has an excellent political science department. Perhaps as a public service they should bring back their erstwhile star student for some refresher courses in research methods and electoral behavior. Or maybe Stephanopoulos will decide to take me up on my sincere offer of John and David’s assistance.