We have a new book on the 2012 presidential election, The Gamble, that provides one model for public engagement. The book was designed to be an accessible academic account of the election, written in real time and published within a year of the election itself — standard timing for books focused on the general public, but an unusually short time frame for a scholarly book. Together with our publisher, Princeton University Press, we structured the project so that we could enter into the ongoing public discussion about the election alongside pundits and journalists — via continuous analysis and writing, serializing the process of peer review, and accelerating the final mechanics of publication.
Our experience writing this book suggests to us that there are underutilized opportunities for both scholars and their publishers to innovate on traditional modes of academic writing and thereby bring scholarly research to a much larger audience. We joked over the past two years that part of “the gamble” was simply writing the book itself. We believe that this gamble has paid off, and we offer our story in hopes that it might encourage others to roll the dice. We think this sort of project can benefit scholars, publishers, and the broader public alike.
As the criticism of our current state of journalism and the current state of journalism education mounts, we ask a simple question: Could political science graduates do a better job of providing political reporting than graduates with journalism degrees? Although we do not test this question empirically, a review of the extant literature suggests that political science departments and curricula have the potential to foster graduates that have a high level of political knowledge, political judgment, and critical thinking skills. These skills are essential for political reporters if they are to wade through political spin, manipulation, and misdirection.
Secondly, if political science graduates are indeed more qualified to provide political reporting to US citizens and to reenergize American democracy, would media executives be willing to hire such graduates absent a degree in journalism? This question we do examine empirically. With a survey of current media executives utilizing a battery of questions assessing their willingness to hire nonjournalism graduates, we are pleased to find an openness on the part of media executives to hire political science graduates to do their political reporting, even if such graduates do not possess a degree in journalism.
That is from a new article (ungated) by Matthew Manweller and Ken Harvey. Their account gels with growing interest in journalism schools in teaching subject area expertise—see, for example, this Carnegie initiative as well as the Columbia Journalism School’s curriculum for the M.A. in journalism and for M.A. students concentrating in politics. It also gels with (and cites) my piece with Brendan Nyhan on how political science can help journalists and Greg Marx’s piece on “embracing the wonk.”
To be sure, the majority of media executives in Manweller and Harvey’s survey do not see subject area expertise in political science as a prerequisite to political reporting: about 4 in 10 agree that a political science degree is needed to do political reporting. Indeed, more of them appear to value practical experience in politics or government. But clearly there is a sizable group that desire subject-specific knowledge in political science. This one quote from a Kansas newspaper editor was particularly striking:
Mass communications students I’ve employed seem to lack a general knowledge of how government works. They, at first, seem overwhelmed by the detail they must quickly assimilate in order to adequately report what is happening on their beat. It often takes as long as two years to get a new reporter up to speed on how to cover a government beat. I don’t expect a mass com grad to know it all, but when I have to explain the difference between a city manager and a mayor, I begin to doubt a reporter’s ability to make sense of the complicated beat government can be.
Just wanted to note a quick and surely incomplete census of scholars providing some heft to “scandal” coverage that badly needs it over the weekend.
- While we have been told with increasing breathlessness that the Obama administration’s scandals are “worse than Watergate,” a story in this weekend’s Washington Post, by Matthew Dallek, points out the problems with the Nixon analogy (which, as I pointed out a long while back, would have to ramp up rather dramatically to be close to true. Though I do hope there are White House tapes.)
- Another Washington Post piece noted the oddly dichotomous simultaneous critiques of the president—at once he is tyrannical and disengaged. Will Howell makes a cameo here and gets a nice book plug too.
- And, this a.m. NPR’s “Morning Edition” hits the trifecta: quotes fr0m Brendan Nyhan and Donna Hoffman on the purported “second term curse,” while another featured Frances Lee on “why Congress has reasons not to be bipartisan despite various rose-colored pipe dreams about the post-World War II ‘consensus.’”
Please add other similar links in the comments section.
… because someday someone might write this about you:
But as we all eventually learn the hard way, Nyhan ALWAYS COLLECTS HIS MONEY, HONEY.
That’s Jason Linkins at the Huffington Post writing about Dartmouth College political scientist and blogger Brendan Nyhan. The topic in question? Nyhan’s research about scandals and US presidents—he blogged about it here— which, if I’m not mistaken, was a somewhat big topic yesterday in the US media.
And yes, for those keeping track, Nyhan did have a scandal probability forecasting figure:
One of the central goals of our blog is to improve communication between political journalists and political scientists. From one direction, we want to make journalists aware of important and relevant scholarly research. From the other direction, we want to encourage political scientists to write for general audiences.
As part of our goal, we spend a lot of space on promotion, encouraging journalistic work that we like and spreading the word about new political science research by ourselves and others.
Sometimes, though, we’re critical. The usual targets of our barbs are:
– Scholars who attempt popular writing outside their areas of expertise, don’t check their facts, and make silly mistakes (for example)
– Scholarly work that could be fun if presented as such but is overhyped and oversold in its real-world implications (for example)
– Innumeracy in political reporting (for example)
Other times, we link in our posts to relevant news articles or analyses published by journalists, not with the purpose of criticizing but rather to anchor our discussion in the context of current political debate. In these sorts of links, we are acknowledging journalism as part of the “real world,” and we are connecting our political-science work to the larger public discourse.
The trouble is, sometimes the mockery we do (and here I’m speaking first about what I do, but also more generally of bloggers in general) establishes a general adversarial tone that seems to stay in the air, even when we’re doing straight references to journalists. That’s what happened here the other day.
Following up on an earlier post I’d done on our research on the potential effects of changes to presidential election vote counting (in particular, national popular vote or a congressional-district-based electoral college system), I posted some simple calculations on the 2012 election. In that post, I gave a brief parenthetical link to a relevant recent article by Nate Cohn in the New Republic.
Cohn commented on my post and showed great annoyance that, as he saw it, I’d imputed some foolish positions to him. In response, I rephrased to clarify that I had no problems with what he’d wrote.
But I think the real problem was not what I’d posted in this case but rather the general expectation of animosity, based on years of interactions between journalists and bloggers. There’s just the expectation that, when a blogger links to a news analysis, it’s to criticize, to mock, to “fisk,” if you will.
I’ve seen this happen once or twice before, when I’ve linked to a news article that I liked, but the journalist in question got angry at what he perceived as snark. No snark was intended in these cases, but it’s notoriously difficult to convey intonation in typed speech, and if you’re expecting snark, you’ll likely see it.
So, to step back, my fault was not anything particular right here, but rather more generally in establishing an occasionally adversarial position in the past.
It goes like this: I mock John Yoo, I mock Gregg Easterbrook, I even mock Niall Ferguson (whose earlier work I’m on record as liking a lot). Then I give a straight link to New Republic writer Nate Cohn and it’s perceived as a slam. If I hadn’t established a pattern of mocking, maybe my recent post wouldn’t have been misread.
And, really, there’s no reason to mock, not if my goals are those of the Monkey Cage, to help make political science more relevant and journalism more science-based. Mocking can be fun, and it can make the occasional blog post more fun to read, but I don’t see it serving the larger goals at all.
So, from now on, no mockery. Here’s an example where I actually wrote the blog post in two ways, first to mock, then in a completely sincere tone. If I keep this up for a few years, maybe the journalists who I write about will accept my links in their intended spirit. But, for now, the ball’s in my court.
P.S. Here’s the American Film Institute’s list of the top 100 movie quotes. They all seem right to me, except for #21. I think all the other quotes on the list are better and more memorable (with the exception of #88, I suppose). What were they thinking??? Overall it’s an excellent list. I wonder what are the best movie quotations that didn’t make the cut?
As Barack Obama enters his second term, some already are wishing it over. Not just the House Republicans, either – but those who want to locate Obama’s place in The Annals of Presidential Greatness, deciding whether he is the 7th, 17th, or 37th best president of all time.
In a new 538 post around this topic, Nate Silver spends a lot of energy proving the unsurprising: that presidents who serve longer, and win larger re-electoral margins, are better regarded by history—or at least by historians. I will forgive his use of Franklin Pierce—Bowdoin ‘24!—as the poster child of loser one term presidents. My beef is really with the larger enterprise of the rankings themselves, which I would argue fall into the irresistible-but-mostly-useless category of political science data.
Silver’s raw material is—as it has to be, really—the iterated surveys of presidential performance taken by Arthur Schlesinger Sr. and Jr. (in 1948, 1962, and 1996), as added to by others who sought to redress what they saw as the Schlesinger samples of respondents’ Democratic bias (as with the Wall Street Journal and Federalist Society surveys, as well as Alvin Felzenberg’s more idiosyncratic take) and the wider recent efforts of C-SPAN. (The top ten from Schlesinger’s ‘96 survey are in the montage above.)
But a number of issues arise. It’s worth noting that presidential incumbents are, almost to a person, outliers – located on the far positive reaches of any scale measuring American political aptitude and skill. This creates a problematic bell curve when they are isolated into a single population. We might see the top and bottom as clearly differentiable, but is there any meaningful gap in performance between someone ranked #15 and someone ranked #25 in such a small set of observations?
Further, the rankings themselves change over time. Harry Truman is the classic case: a president widely unloved by the electorate at the close of his term but one whose stock has risen steadily since. (It doesn’t hurt to have David McCullough write a book about you.) Dwight Eisenhower, too, has seen his rankings rise over time – in his case, to match his extant public popularity—as a fuller internal record of his presidency became available. Richard Neustadt, for one, downplayed Eisenhower’s executive skills, but early judgments can mislead, even within a single individual’s tenure in office. Indeed, the assessment of the George W. Bush legacy immediately after his reelection – as observers lauded the Republican majority realignment apparently achieved – contrasts rather sharply with the picture four years later.
The deeper problems with rankings, though, lie in the nature of the enterprise itself. They are inherent in the difficulty in choosing the right standards for measurement and in assigning credit or blame to isolated individuals in the separated system of American governance. As Donald Rumsfeld famously briefed, “Stuff happens” – but the fact of its happening during a president’s term doesn’t mean the president made it happen. There is a wide range of governmental outputs, and outcomes, not all of them attributable to presidential action – even if he did in fact prefer that outcome. What outcomes has Barack Obama personally effected? Which of those should receive more weight in our retrospective assessment? Can we give credit for a good decision, even if it had a bad outcome?
And: is a decision or outcome “bad” if it conflicts with a later code of morality or with a latter-day judge’s ideological preferences? The institution of slavery and the treatment of Native Americans, for two, loom rather nastily over the earlier presidents. The comparative salience of even less normative issues, as abetted by most raters’ imperfect knowledge of American history – a quick read of past State of the Union messages suggests an array of basically-forgotten crises—also makes a difference for our accuracy in judging the past. Along these lines (cf. Stephen Skowronek’s work on “political time”) we must also recall that different presidents enter office under distinct political circumstancesthat expand or constrict the options available for presidential achievement. Bill Clinton moaned to his advisers that no one could be a great president without a national emergency, a thought channeled by Obama chief of staff Rahm Emanuel a decade later when he urged his boss to “never waste a crisis.”
How does all this get played into the mechanics of the rankings? The early surveys ranked presidents as “great,” “near-great,” etc., on the grounds that – like pornography – you know greatness when you see it. C-SPAN, in its 2009 rankings, asked sixty-five scholars of the presidency to grade the forty-two individuals to have served as president at that point on no fewer than ten “attributes of leadership.” These cut across a wide range of areas encompassing not only skills and policy arenas but perceptions: public persuasion, crisis leadership, economic management, moral authority, international relations, administrative skills, relations with Congress, vision/agenda setting, “pursuing equal justice for all,” and broad “performance within context of the times.” This makes some sense in that it invites us to consider the multiple skills required for success in the job and the multiple dimensions to any presidency. (How do we feel about personal behavior, for instance, as opposed to the content of that person’s public policy?) Providing a series of categories allows presidential raters to make some distinctions along these lines, allowing us to praise Lyndon Johnson’s commitment to voting rights while decrying his decision to escalate in Vietnam.
Yet when translated into scores, those assessments across a long list of categories are compacted, equally weighted, into a single score—which assumes that each category is of equal importance, both across a presidency, and across time. Does “performance within the context of the times” have the same value as “administrative skills” or “relations with Congress” (indeed, does the former subsume the others in any case?) Do “economic management” and “moral authority” count simply as two equivalent questions on a multi-part exam? Should a bad television presence cancel out a good nuclear crisis? It’s a mess.
In the end, then, Silver is right to wish there was an historical equivalent of the VORP (value over replacement player) statistic in baseball. What we want to know is the “value added” of having a specific person in the Oval Office, controlling for context? We can put the question in intuitive terms without casting too far back in history: comparing Gore v. Bush and the aftermath of 9/11, certain parts of history would have gone along similar lines, but others would likely have diverged significantly. The variance represents (some of) the Bush difference. Still, we’re stuck with counterfactuals when slugging percentage and fielding range would be much more satisfying things to know.
All this should encourage us to seek nuance, to be as cautious about our causal claims and evidence as we would be in a more obviously empirical exercise. I’m not hopeful that the second term punditry will toe that line. But in the meantime, Marc Landy and Sidney Milkis have a short and sweet assessment of greatness: whether the president transforms how Americans view their government, winning a “struggle for its constitutional soul.” You know that, presumably, when you feel it.
This, between Brendan Nyhan and Politico reporter Alex Burns is worth reading as another chapter in the vexed relationship between political science and horserace journalism. The underlying disagreement here is over which kinds of knowledge count. On the one hand, Burns suggests that the collective wisdom among well-connected DC politicians is strong evidence of what is actually true in national politics. On the other, Nyhan wants polling data or similar. What’s interesting is that this conversation wouldn’t have been likely to have happened a few years ago. Someone in Burns’ position would not have felt obliged to defend their form of specialized knowledge a few years ago, because it would have seemed self-evidently unassailable. The rise of an alternative set of norms about which knowledge is appropriate to settling this kind of dispute, appealing to very different kinds of evidence is an important sociological change in journalism (which has perhaps been pushed on a little by blogs like ours, but which is probably more directly driven by an increased willingness of prominent journalists to engage directly with data, efforts to figure out which causal relationships are plausible, which implausible and so on).
[updated to bring the underlying argument out better]
David Lazer has an interesting piece on the topic.
What is important is how well pollsters did in the face of increased obstacles to doing a good job: response rates to surveys have plummeted, and increasing numbers of individuals rely exclusively on (hard to reach) mobile phones. Despite these challenges, in aggregate surveys are more accurate than ever, almost spot on in 2012. How is this possible? This is worth far more reflection than a blog entry can offer, because not all communities face challenges like these so effectively. … Here I will simply speculate that it reflects three things. The first is that there is real world feedback as to the effectiveness of methods to address these challenges. … Here I will simply speculate that it reflects three things. The first is that there is real world feedback as to the effectiveness of methods to address these challenges. Third, there is a collective process of sifting through best practices. While there is certainly some desire to keep the secrets to success private, in fact there is a certain necessary degree of transparency in methods; and this is a small world of professional friendships where knowledge is semi-permeable, allowing a certain degree of local innovation providing short run advantage, while allowing good practices to disseminate. That is, there may be (as I have written about elsewhere) a good balance between exploration (development of new solutions) and exploitation (taking advantage of what is known to work) in this system.
…The system of pollsters might be contrasted with that of pundits. Do you expect a Darwinian culling of the right leaning pundits who missed the outcome? The answer is surely not. Nor will there be an adjustment of practices on the part of pundits who largely served up a mix of anecdotal pablum to their readers. … And how did the right get it so wrong? How could the Romney campaign of successful political professionals, in part embedded in the same epistemic community as the broader set of pollsters, not have seen an Obama victory as a plausible (put aside likely) outcome? This was not a near miss on their part. Consider: at last count, you could have subtracted 4.7 points (!) from Obama’s margin in every state and he would still have won … . Romney’s campaign, and many commentators on the right, were living in a parallel world, one with fewer minority and young voters than in ours. Again, I don’t know the answer to this question. Likely key ingredients: an authentic ambiguity in how to handle the aforementioned challenges; a strong desire to see a Romney victory; an informational ecosystem today that provides the opportunity for producing plausible sounding arguments to rationalize any wishful thoughts one might have; and the relevant subcommunity was small, centralized, and deferential enough so that a few opinion leaders could trigger a bandwagon.
As David is suggesting, this is a specific case of a more general problem – how does one build forms of collective cognition that generate useful information rather than garbage? The only thing that I would add is that it might be useful to think very slightly more explicitly about the incentives that these different communities have. As he notes, there is likely a fair degree of intellectual exchange happening among professional pollsters, producing something that roughly approximates the kinds of exploration-of-different-alternatives-by-actors-communicating-with-each-other that he has studied through simulations, and that Mason and Watts work on through experiments. There may be some tendencies towards isomorphism, but they look to be relatively mild. In contrast, professional pundits are in the business of entertaining, and producing counter-intuitive claims rather than being right. As @jimcramer rather revealingly describes the perceived incentives he faces, ” No one will recall who picks Obama by 10 electorals if it turns out to be 150 margin. Believe me.” Such pundits are indifferent between wild guesses that are wrong, and safe guesses that are right – neither is likely to be remembered. Hence, they have strong incentives to make wild guesses rather than sober ones – there’s no downside to being wrong, and much upside to being right. Finally, the problems for pollsters in a campaign don’t only have to do with wishful thinking, and the bandwagoning power of a few leaders. They also likely have to do with commmenters’ desire not to be seen as deviating from the collective consensus among their ideological community. Their problem is precisely the opposite of professional pundits – deviants and iconoclasts from the prevailing wisdom are likely to be cast out if they are wrong, whereas both those who are wrong, and those who are right, are likely to continue to be employed (and to have reasonable employment chances in other campaigns) as long as they do not stray from the herd.
In short, professional pollsters have (most of the time), good incentives to be right. Professional pundits have good incentives to guess wildly, regardless of whether they are wrong or right. Political hacks have good incentives to guess safely, regardless of whether they are wrong or right. And that, arguably, is why we are where we are.
Update: Also this, from Cosma Shalizi way back in 2005:
When political scientists, say, come up with dozens of different models for predicting elections, each backed up by their own data set, the thing to do might not be to try to find the One Right Model, but instead to find good ways to combine these partial, overlapping models. The collective picture could turn out to be highly accurate, even if the component models are bad, and their combination is too complicated for individual social scientists to grasp.
Admit it. You wish you could have seen a figure like this on your TV last night.
I make a longer case for why John King should have been showing us scatterplots at Mischiefs of Faction.
Political science proclaims, “debates don’t matter.” After this election, we may need to retire a lot of political science.
So that’s just wrong. In my piece about debates, I was careful to separate two things:
1) Whether debates move the polls enough to determine the winner of the election. This may have happened in 1960, although the polling data is thin and so any inference is tenuous. This may have happened in 2000 as well.
2) Whether the debates move the polls at all. They can! Not always, and not always by a lot, but they can. Look, I even posted the data.
Yes, political science finds a smaller role for presidential campaigns than reporting and commentary often suggests or implies. So our position gets caricatured as “campaigns don’t matter.” But that’s just a convenient straw man, and it’s not what the political science research says.