bq. Presidential ranking polls communicate far more than an ordered list of names; they communicate the leadership qualities our nation values. Given this, the results of presidential ranking polls have been a source of contention. One recurrent concern is that the academic raters surveyed in the polls, who tend overwhelmingly to be partisan Democrats, may favor some presidents over others. This study looks for evidence of a partisan bias in the ranking polls. Concentrating on the modern presidency, we find that presidential partisanship is a potent predictor of rank; academic raters consistently rank Democratic presidents ten places higher on average than Republican presidents. We also compare the rankings from academics to rankings from non-academics and show that academic raters favor Democratic presidents more than non-academic raters. Our findings suggest, in accordance with previous literature, that partisan attachment affects the subjective judgments that presidential ranking polls inherently require.
From a forthcoming paper by Joseph Uscinski and Arthur Simon. This is based on a study of presidents from McKinley through George W. Bush. See the paper for why they focus on these more modern presidenets.
Uscinski and Simon show that academics surveyed by C-SPAN rate Democratic presidents more favorably, and Republican presidents less favorably, than a sample of C-SPAN viewers (see table 2 of the paper). Similarly, a 2005 Wall Street Journal survey of academics, which weighted the results of self-identified liberal and conservative academics equally, and found less evidence of any pro-Democratic bias (see the Appendix Table 2). They suggest that both of these findings could be due to partisan bias and, yes, they do discuss some alternative explanations.
Here are a few of my initial thoughts:
* Uscinski and Simon examine both ratings and rankings. I’m glad. I’ve posted before on why ratings are better than rankings.
* Given all of our posts on partisan bias — and the evidence that even “experts” are not immune to such biases — I am fully prepared to believe that academics who rank presidents are similarly afflicted.
* I am a little concerned that even relatively knowledgeable C-SPAN viewers may simply be using somewhat different criteria than academics to judge presidents — and not simply because of differences in partisanship. What if C-SPAN viewers simply have different perceptions of, say, the actual powers of the president? Or of specific historical cases? I am thinking here of Bryan Caplan’s research. Here’s a quick summary:
bq. Compared to experts in American politics, the public greatly overestimates the influence of state and local governments on the economy, the president and Congress on the quality of public education, the Federal Reserve on the budget, Congress on the Iraq War, and the Supreme Court on crime rates. The public also moderately underestimates the influence of the Federal Reserve on the economy, state and local governments on public education, and the president and Congress on the budget. While we are open to the possibility that non-cognitive factors explain observed belief gaps, controlling for demographics and various measures of self-serving and ideological bias does little to alter our results.
* Of course, Caplan’s findings don’t necessarily mean that partisan bias isn’t operative here. It’s just challenging to separate it from other biases.
* Looking at all the American presidents, and all the extant expert rankings, I’m struck by how generally similar they are. Wikipedia has a table here. Again, there could still be partisan bias present, but there is a general consensus about many presidents.
Ultimately, I remain somewhat dubious about the entire enterprise of ranking presidents, but that is not material to this paper. I’d encourage C-SPAN to do a lot more to improve the way that it measures presidential greatness, and thinking about partisan bias should be part of that.