The Politics of Pinocchios

by Henry Farrell on October 10, 2012 · 3 comments

in Media

Both left wing and right wing commentators complain frequently about the rise of journalistic ‘factchecking’ columns arguing that they substitute personal (and perhaps biased) opinions for purportedly neutral evaluations of politicians’ factual claims. A more sophisticated version of the critique suggests that factcheckers face their own sets of incentives – i.e. for example, that they may need to appear ‘balanced’ by criticizing both left and right reasonably evenly, even if one side (depending on your viewpoint) is systematically more mendacious than the other. Which makes this “defense of factchecking”:http://mediapolicy.newamerica.net/sites/newamerica.net/files/policydocs/The_Rise_of_Political_Fact-checking.pdf.pdf by the founder of the Washington Post‘s factchecking column, Michael Dobbs, particularly interesting.

What is striking about Dobbs’ account is the apparent inconsistency in its defense of ‘Pinocchios’ – the pseudo-quantitative score that the factchecking column awards (from one to four Pinocchios) to dishonest political statements (the more Pinocchios, the more dishonest the statement is supposed to be). On the one hand, Dobbs suggests that the Pinocchios are opinions, not dicta:

bq. While I agree that reporters should not be partisan, I see no reason why we should not draw on our experience and plain commonsense to sort out the truth from fiction. Reporters should be allowed to sift the evidence and reach conclusions. I see nothing wrong with that, as long as the conclusions are based on the evidence, rather than reflecting some kneejerk political position. A Pinocchio is just that, a conclusion, expressed in a colorful way. It is the evidence-based opinion of a reporter who has examined the facts, and tried to determine the truth as best he can. Readers, and politicians, are free to agree or disagree.

On the other, he and his successor think that the Pinocchios are what get the attention, and seem pleased with that attention.

bq. Kessler defends the Pinocchio device as a way to whet the interest of readers and attract the attention of the candidates, some of whom are “obsessed with Pinocchios.” He has received calls from senior politicians wanting to know what they need to do in order to avoid receiving more Pinocchios. “I have learned my lesson,” said one errant member of the
House of Representatives. “I am not going to utter another word on this subject without having my staff check it over.” … While they are quick to ridicule fact checks aimed
against them, campaigns often gleefully jump on fact checks aimed at their opponents. “Four, count ‘em, Four Pinocchios,” the Obama campaign crowed when I exposed Hillary Clinton’s Walter Mittyish fantasy about “coming under fire” at Tuzla airport while on a visit to Bosnia. I knew that “Pinocchios” had entered the political lexicon when I tuned into a Republican presidential debate in late 2007 and heard Mitt Romney berate one of his Republican rivals: “You earned three Pinocchios for that.” (He did not feel the need
to explain exactly what a ‘Pinocchio’ was.)

So, on the one hand, Dobbs is telling us that Pinocchios are just an expression of a specific journalist’s considered and evidence based opinion, which readers and politicians are free to agree or disagree with. On the other, he seems to be suggesting that it’s great that so many people take them so seriously. Something there seems to me not to compute. This is more worrying still when you throw in Dobbs’ admission that he sometimes felt out of his depth in writing the column.

bq. I often felt overwhelmed by the deluge of misinformation on the campaign trail. While I was well versed in some topics, such as foreign affairs and history, I was a neophyte on others, including the complexities of the federal budget and the environment. In order to adjudicate factual disputes, I had to become an instant expert on a wide range of issues. There were times when I felt I was the only reporter researching the policy proposals of the candidates, rather than their day-to-day maneuvering and their standing in the opinion polls. Producing my unofficial quota of one fact check a day was sometimes a struggle. Looking back, I feel that I did more distinguished work during the 2004 campaign, when I was able to focus on a few issues, such as the Swift Boat controversy, than in 2008, when I was spread very thinly.

This suggests that Pinocchios are likely to be a pretty unreliable measure of anything except for obvious lies or truths. When a single person is trying to make judgment calls on very different topics every day, most of which he or she has no subject-matter expertise on and then is expected to score these judgment calls using a semi-arbitrary ranking scheme, the final product is likely to have a pretty low information content. Not only that, but it is likely to have a particularly pernicious mix of apparent authoritativeness and actual unreliability. Readers are likely to pay a lot of attention to it, because it looks quantitative and definitive, without thinking much about how it was arrived at. It might be interesting to see media scholars doing experiments, in which they varied the number of Pinocchios awarded as their treatment, while keeping the verbal description of assumed dishonesty fixed, to see exactly how much the quantitative frame shaped individual perceptions.

Dobbs’ suggestion (if I read him correctly) that the factchecking column is a poor man’s substitute for substantive journalism on the content of policy is also very interesting. However, that’s a different topic for a different post (most likely on a different blog).

{ 3 comments }

LFC October 11, 2012 at 1:01 am

I looked at the Pinocchios score for a particular statement on only one occasion, as I recall, and never had the inclination to do it again. I also rarely read the Post’s factcheck column. I don’t really see the point of focusing so intently on single statements, which is what the Pinocchios score and the column both seem to do. I’m more interested in whether a candidate is generally talking sense or generally not talking sense, and most of the time I would prefer to rely on what you call “substantive journalism on the content of policy” to help me determine that. And I don’t esp. like it that the Post seems to have shunted a fair amount of that job onto Ezra Klein and his staff (for domestic policy, at least). They’re pretty good, I think, although I don’t read them all that often, but I’d prefer it if they had more of their reporters doing that stuff on a regular basis. There are some who do it, e.g. Walter Pincus, Dana Priest, and some others, but not as many as there might be.

Bob October 11, 2012 at 6:36 pm

There was a good article in The Atlantic about ‘reality checking,’ as opposed to fact checking. I recommend it: http://www.theatlantic.com/politics/archive/2012/09/better-than-fact-checking-an-ohio-reporter-speaks-truth-to-power/261995/

David Karger October 11, 2012 at 11:04 pm

I’ll take issue with your conclusion that a relative non-expert will not be able to reliably check our politicians. It seems to me that somewhere along the way our politicians decided that their statements don’t need to have the slightest connection to truth (see e.g. http://www.tnr.com/blog/plank/108457/wsj-declares-liar-out-bounds). It’s generally quite easy to flush out each falsehood. The challenge, and the reason these fact-checkers are so useful, is that there are just so many of them, nobody who has a full-time job can spend the time checking each one. These fact-checkers don’t need any specific skills. All they need are two qualities that pervade journalism: a sense of objectivity, and time to exercise it.

If you’re going to question the value of fact-checkers, I’d use a different attack: given that we *know* everything the politicians say is false, it isn’t clear there’s any benefit to having someone tell us so.

Comments on this entry are closed.

Previous post:

Next post: