Rating the Presidents


As Barack Obama enters his second term, some already are wishing it over. Not just the House Republicans, either – but those who want to locate Obama’s place in The Annals of Presidential Greatness, deciding whether he is the 7th, 17th, or 37th best president of all time.

In a new 538 post around this topic, Nate Silver spends a lot of energy proving the unsurprising: that presidents who serve longer, and win larger re-electoral margins, are better regarded by history—or at least by historians. I will forgive his use of Franklin Pierce—Bowdoin ‘24!—as the poster child of loser one term presidents. My beef is really with the larger enterprise of the rankings themselves, which I would argue fall into the irresistible-but-mostly-useless category of political science data.

Silver’s raw material is—as it has to be, really—the iterated surveys of presidential performance taken by Arthur Schlesinger Sr. and Jr. (in 1948, 1962, and 1996), as added to by others who sought to redress what they saw as the Schlesinger samples of respondents’ Democratic bias (as with the Wall Street Journal and Federalist Society surveys, as well as Alvin Felzenberg’s more idiosyncratic take) and the wider recent efforts of  C-SPAN. (The top ten from Schlesinger’s ‘96 survey are in the montage above.)

But a number of issues arise. It’s worth noting that presidential incumbents are, almost to a person, outliers – located on the far positive reaches of any scale measuring American political aptitude and skill. This creates a problematic bell curve when they are isolated into a single population. We might see the top and bottom as clearly differentiable, but is there any meaningful gap in performance between someone ranked #15 and someone ranked #25 in such a small set of observations?

Further, the rankings themselves change over time. Harry Truman is the classic case: a president widely unloved by the electorate at the close of his term but one whose stock has risen steadily since. (It doesn’t hurt to have David McCullough write a book about you.)  Dwight Eisenhower, too, has seen his rankings rise over time – in his case, to match his extant public popularity—as a fuller internal record of his presidency became available. Richard Neustadt, for one, downplayed Eisenhower’s executive skills, but early judgments can mislead, even within a single individual’s tenure in office. Indeed, the assessment of the George W. Bush legacy immediately after his reelection – as observers lauded the Republican majority realignment apparently achieved – contrasts rather sharply with the picture four years later.

The deeper problems with rankings, though, lie in the nature of the enterprise itself. They are inherent in the difficulty in choosing the right standards for measurement and in assigning credit or blame to isolated individuals in the separated system of American governance. As Donald Rumsfeld famously briefed, “Stuff happens” – but the fact of its happening during a president’s term doesn’t mean the president made it happen. There is a wide range of governmental outputs, and outcomes, not all of them attributable to presidential action – even if he did in fact prefer that outcome. What outcomes has Barack Obama personally effected?  Which of those should receive more weight in our retrospective assessment? Can we give credit for a good decision, even if it had a bad outcome?

And: is a decision or outcome “bad” if it conflicts with a later code of morality or with a latter-day judge’s ideological preferences? The institution of slavery and the treatment of Native Americans, for two, loom rather nastily over the earlier presidents. The comparative salience of even less normative issues, as abetted by most raters’ imperfect knowledge of American history – a quick read of past State of the Union messages suggests an array of basically-forgotten crises—also makes a difference for our accuracy in judging the past. Along these lines (cf.  Stephen Skowronek’s work on “political time”) we must also recall that different presidents enter office under distinct political circumstancesthat expand or constrict the options available for presidential achievement. Bill Clinton moaned to his advisers that no one could be a great president without a national emergency, a thought channeled by Obama chief of staff Rahm Emanuel a decade later when he urged his boss to “never waste a crisis.”

How does all this get played into the mechanics of the rankings? The early surveys ranked presidents as “great,” “near-great,” etc., on the grounds that – like pornography – you know greatness when you see it.  C-SPAN, in its 2009 rankings, asked sixty-five scholars of the presidency to grade the forty-two individuals to have served as president at that point on no fewer than ten “attributes of leadership.” These cut across a wide range of areas encompassing not only skills and policy arenas but perceptions: public persuasion, crisis leadership, economic management, moral authority, international relations, administrative skills, relations with Congress, vision/agenda setting, “pursuing equal justice for all,” and broad “performance within context of the times.” This makes some sense in that it invites us to consider the multiple skills required for success in the job and the multiple dimensions to any presidency. (How do we feel about personal behavior, for instance, as opposed to the content of that person’s public policy?) Providing a series of categories allows presidential raters to make some distinctions along these lines, allowing us to praise Lyndon Johnson’s commitment to voting rights while decrying his decision to escalate in Vietnam.

Yet when translated into scores, those assessments across a long list of categories are compacted, equally weighted, into a single score—which assumes that each category is of equal importance, both across a presidency, and across time. Does “performance within the context of the times” have the same value as “administrative skills” or “relations with Congress” (indeed, does the former subsume the others in any case?)  Do “economic management” and “moral authority” count simply as two equivalent questions on a multi-part exam?  Should a bad television presence cancel out a good nuclear crisis? It’s a mess.

In the end, then, Silver is right to wish there was an historical equivalent of the VORP (value over replacement player) statistic in baseball. What we want to know is the “value added” of having a specific person in the Oval Office, controlling for context?  We can put the question in intuitive terms without casting too far back in history: comparing Gore v. Bush and the aftermath of 9/11, certain parts of history would have gone along similar lines, but others would likely have diverged significantly. The variance represents (some of) the Bush difference. Still, we’re stuck with counterfactuals when slugging percentage and fielding range would be much more satisfying things to know.

All this should encourage us to seek nuance, to be as cautious about our causal claims and evidence as we would be in a more obviously empirical exercise. I’m not hopeful that the second term punditry will toe that line. But in the meantime, Marc Landy and Sidney Milkis have a short and sweet assessment of greatness: whether the president transforms how Americans view their government, winning a “struggle for its constitutional soul.” You know that, presumably, when you feel it.

10 Responses to Rating the Presidents

  1. Andrew Gelman January 23, 2013 at 9:55 pm #

    I’m not complaining about this particular post, but more generally I agree with James Loewen who wrote in the classic “Lies My Teacher Told Me” that presidents as a whole are overrated as a topic of study. Sure, Lincoln etc are huge, no doubt about that. But, as Loewen points out, U.S. history textbooks seem to feel the need to have something on every president, even while skipping various more important historical figures.

  2. Manny January 24, 2013 at 12:57 am #

    Don’t forget Mark Taylor’s piece in the October PS. Digs up some historical economic indicators to line up FDR, Harding (!), Hayes (!), McKinley (!), and Fillmore (!) as the top five.

    Jokes aside, I thought it was a clever article.


  3. Joe January 24, 2013 at 9:20 am #

    For a review of the literature on presidential rankings, see:
    or http://onlinelibrary.wiley.com/doi/10.1111/j.1741-5705.2012.03991.x/abstract (gated)

  4. Andrew Rudalevige January 24, 2013 at 10:27 am #

    I’d agree that any given individual president may not be worth studying – in a way, in fact, that’s a lesson of the rankings enterprise. But the *presidency* is well worth study; indeed, as the size and scope of the national government grows, the intersection of the presidential branch with the executive branch becomes even more crucial to understanding policy outcomes. The best presidency research gets around the n=1 problem to look at institutional behavior and repeated presidential functions or tasks (appointments, executive orders, legislative proposals, speechmaking, etc.)

    Manny – Fillmore is overdue for a break! Still, I’m not sure the article is able to establish much in the way of causality between presidential actions and economic performance. (After all, even if a president favored certain policies, he will rarely have put them into place single-handedly…)

  5. adano January 24, 2013 at 11:52 am #

    I’m not recognizing president #9 in the photo. Who is that?

    • Andrew Rudalevige January 24, 2013 at 9:14 pm #

      Howdy – that’s James Polk.

      • Mitch January 25, 2013 at 10:22 am #

        That’s a pretty emo Polk. He’s got a fauxhawk going on. He’s working at the Nashville Starbucks.

        • Mitch January 25, 2013 at 10:24 am #

          “I was for the Treaty of Guadalupe Hidalgo before it was cool…”

  6. chris January 24, 2013 at 4:43 pm #

    ISTM that several of the pictured presidents are getting by partly on their non-presidential achievements — particularly Washington, Eisenhower, Jefferson, and possibly Jackson.

    Other than that, Clinton seems to have been clearly right about the need for crisis — Lincoln, Wilson, and Truman all had their presidencies defined largely by the crises that occurred during their terms, and FDR, thanks to his long term, is credited with the response to *two* major crises (or maybe one and a half, since he has to share WWII with Truman). And Eisenhower is made great mainly by virtue of his role in a national crisis that occurred *before he was even President*.

    On the other hand, George W. Bush seems to prove that crisis is a necessary but not a sufficient condition for a great presidency.

  7. David Karger January 30, 2013 at 6:19 pm #

    I think presidential rankings are interesting, not because the presidents are actually rankable, but because they provide a great framing device for someone who wants to make an argument about the role of the presidency.