Political scientists tend to be lousy political prognosticators. We speak in circumlocutions, carefully couching our every utterance in a web of qualifications. Our favorite answer to “what’s going to happen?” kinds of questions tends to be “It depends,” followed by a lengthy recitation of the reasons why things are just too complicated to be predictable. We prefer to do our “prediction” after the fact, when things are, after all, more predictable. To be sure, a few political scientists — Tom Mann, Larry Sabato, and the inevitable Norman Ornstein come immediately to mind — have mastered the art of speaking ostensible truth to the great unwashed, but the rest of us are far less skilled insofar as conveying our interpretations of current events and our expectations of future ones to the general public.
When most practicing political pundits are called upon to forecast what’s going to happen, they typically answer so forcefully and self-assuredly as to produce a sense of certainty in the listener, whose attention span, in turn, tends to be so short that he or she never remembers to check up a few days later on whether the pundit’s prediction was borne out.
Could it be that pundits are themselves dismal forecasters? Well, yes. That, at least, was the conclusion that Jarol Manheim, Susannah Pierce, and I reached several years ago in a study titled “Inside Dopes? Pundits as Political Forecasters” (abstract here). In that study, we assessed the accuracy of the predictions offered by members of TV’s noisiest shout-show, “The McLaughlin Group.” Those predictions turned out, in large measure, to be either untestable or, if testable, wrong.
Where does this leave us? For one thing, it leaves us wanting to know more about the bases of expert judgments, and in that connection I know of no better source than Philip Tetlock’s book, Expert Political Judgment.
For those who are interested enough to log a bit of time doing armchair fieldwork of their own on the punditocracy, there’s a new resource that could make things much easier. The folks over at CampaignCircus.com recently announced what they’re calling the “Pundit Accountability Project.” To keep tabs on what various pundits have been saying, all you have to do is click here and you’ll get a drop-down menu with the names of various pundits. As of now, 28 pundits are listed, but the intention is to keep adding pundits and clips as circumstances warrant. Click on a pundit’s name and you’ll get clips of his or her on-air statements that have some predictive component. The press release mentions that CampaignCircus.com plans periodic releases of clips by the pundit who made the most consistently wrong predictions — though it’s unclear what sort of “fact-checking” technique they’re going to use to determine who wears that crown. For example, the press release says “Howard Fineman is currently leading the pack” in predictive inaccuracy, but how that determination was made isn’t clear. In any event, this compilation of clips should at the very least be a fun entry to add to your “Favorites” menu, and it might even turn out to be a worthwhile source of data on the conventional “inside the Beltway” wisdom as the campaign plays out.