Thoughts on Groseclose book on media bias

by Andrew Gelman on July 29, 2011 · 27 comments

in Media

Respected political scientist Tim Groseclose just came out with a book, “Left Turn: How Liberal Media Bias Distorts the American Mind.” I was familiar with Groseclose’s article (with Jeffrey Milyo) on media bias that came out several years ago–it was an interesting study but I was not convinced by its central claim that they were measuring an absolute level of bias–and then recently heard about this new book in the context of some intemperate things Groseclose said in a interview on the conservative Fox TV network.

Groseclose’s big conclusion is that in the absence of media bias, the average American voter would be positioned at around 25 on a 0-100 scale, where 0 is a right-wing Republican and 100 is a left-wing Democrat. (Seeing as the number line is conventionally drawn from left to right, I think it would make more sense for 0 to represent the left and 100 to be on the right, but I guess it’s too late for him to change now.) Groseclose places the average voter now at around 50, so, by his assessment, media bias is a huge deal.

After some further blog discussion, I became curious enough to head over to the local Barnes & Noble and take a look at the book.

Before I get to the details, let me again link to Brendan Nyhan’s thorough methodological discussion of the Groseclose and Milyo article from 2004.

Journalism as one of many institutions that affect politics

Stepping away from the data for a moment, it makes sense that the news media would generally fall on the left side of the political spectrum. Whether this is good or bad is another question, but newspapers generally seem to position themselves on the side of the underdog. Consider, for example, the slogan that a newspaper should “comfort the afflicted and afflict the comfortable.”

Another way to think about media bias is to think about the communication industry as a whole. I imagine that “journalism,” taken as a whole, leans left, while “public relations,” taken as a whole, leans right.

And if most of the news media in a country moved from journalism to public relations (as in Berlusconi’s Italy), I could imagine it could make a difference in the country’s politics.

The question, “What would public opinion be like if journalists expressed views comparable to the average American?”, while interesting, could perhaps be combined with similar questions such as, What would U.S. politics be like if there were no public relations industry? Or, What would U.S. politics be like if campaign contributions were given equally to the left and the right? Or even, What would U.S. politics be like if religious leaders were, on average, in the political center?

My point is: Thinking of all the different institutions affecting political attitudes, it makes sense a priori to suppose that journalism in particular is on the left. Traditionally, journalism and universities have been on the left, and business and religion have been on the right. There are lots of exceptions, from Martin Luther King to Silvio Berlusconi, but my first guess would be that journalism is one of the left-leaning institutions in the U.S.

It makes sense for Groseclose, as a conservative media analyst, to want to shift journalism to the right, just as, from the other direction, a liberal businessman might want to persuade businesses to move in the other direction.

Multidimensional

Policy and politics are multidimensional. For example, Slate magazine notoriously polled its staff a few years ago and found 55 out of 57 supporting Obama. On the other hand, a Slate writer (who I’d guess is an Obama supporter) wrote this:

If we can find other ways of overcoming the simmering resentment that naturally accompanies wage cuts, workers themselves will be better for it in the long run.

The “we” at the beginning of the sentence does not seem to be the same as the “workers” at the end of the sentence. This is just an anecdote (n=1, and not a randomly sampled n=1 at that) but I do think it reflects a general attitude in the big media, to by default take the perspective of the employer or rich person rather than the employee or lower-income person. (Think of all those famously obnoxious NYT or WSJ lifestyle pieces.)

The multidimensionality of political attitudes should not discourage us from studying bias but it’s worth keeping in mind.

In the U.S. context I think there’s asymmetry in political bias, with Democratic reporters–a survey a few years ago found that twice as many journalists identify as Democrats than as Republicans–biasing their reporting by choosing which topics to focus on, and Republican news organizations (notably Fox News and other Murdoch organizations) biasing in the other direction by flat-out attacks.

I’ve never been clear on which sort of bias is more effective. On one hand, Fox can create a media buzz out of nothing at all; on the other hand, perhaps there’s something more insidious about objective news organizations indirectly creating bias by their choice of what to report.

But I’ve long thought that this asymmetry should inform how media bias is studied.

What is media bias?

Before getting to measures of media bias, let’s think of some ways that a news organization could be politically biased:

  • Reporting that the unemployment rate today is twice as high as it was during Bush’s presidency . . . or reporting that it doubled during the two years leading up to July 2009 and has slightly declined since then.
  • Reporting economic news more frequently when the economy is improving with one party in power or declining when the other party is in power. (See here and here.)
  • Reporting on the sex scandals of David Vitter and Newt Gingrich . . . or reporting on the sex scandals of Anthony Weiner and Al Gore.
  • Interviewing an attractive person who’s out of a job and has no health insurance . . . or interviewing an attractive person who had to shut down a business because of high taxes.
  • Running a warm-and-fuzzy human interest story about an up-and-coming conservative Republican politician . . . or running that same story about a liberal Democrat.
  • A clip of a cute same-sex couple getting married . . . or footage of scary-looking people at a gay pride parade.
  • A feature on tax-subsidized corporate jets . . . or on the big salaries paid to union bigwigs.
  • Linking John McCain to an official bio while linking Barack Obama to mocking cartoons . . . or vice-versa.
  • Coverage of political rallies etc.
  • Giving Democrats or Republicans, or liberals and conservatives, opportunities to address the audience directly in op-eds, softball interviews, etc.
  • Flat-out endorsements.
  • Using politically-loaded terms such as “Star Wars” for the missile-defense program or “death tax” for the estate tax.

Some of these biases have been studied. For example, a Media Matters report found that the Sunday talk shows (Meet the Press, etc) had more Republican than Democratic panelists in the 1997-2005 period.

And Gentzkow and Shapiro implemented a generalization of Groseclose and Milyo’s method to evaluate what they call the “slant” of different newspapers based on how often they use phrases that distinguish the two parties (for example, Republicans talkes about “personal accounts” for Social Security, while Democrats used the term “private account”).

An indirect measure of bias

Setting aside the methodological criticisms raised by Nyhan and others, my big problem with the Groseclose and Milyo estimates of media bias is that they are indirect.

Consider first some simple, direct measures of bias: Counting how many Democrats and Republicans, or liberals and conservatives, appear on op-ed pages or TV interviews. Tabulating the frequency of political sales terms such as “death tax” or “Operation Freedom” or “Affordable Care Act” (as compared to “estate tax” or “Iraq war” or “the Obama health care plan”). Measuring the prominence of positive or negative economic stories (is the bad news always on page 1 and the good news on page 14?).

With some thought we could probably come up with more. My point is that all the above are direct measures of bias. If you do more of X and less of Y, you’re biasing the news in one direction or another–you’re directly sending out a message that can influence people.

In contrast, the Groseclose and Milyo measure–citations of research and advocacy groups–is almost entirely indirect. Sure, if you quote the Family Research Council, you’re likely to be making a conservative point (or providing the conservative perspective in a he-said, she-said story). But it’s not directly an influence; rather, it’s indicative that the news organization might be taking the side of this source.

Here’s another example: the Poole and Rosenthal measures of legislators’ positions. You can argue about their methods, but their estimates are ultimately based on votes in Congress. The way a congressmember gets placed on the left or the right is by voting on the left or right in actual votes. Similarly with various ideal-point estimates on the Supreme Court. The methods aren’t perfect but, again, they’re based on actual votes. Or Bafumi and Herron’s estimates (featured in Red State Blue State) that most voters are less extreme on the issues, compared to congressmembers. These positions are based on how survey respondents say they would have voted on particular issues. You get placed to the left or the right based on actual political positions.

The Groseclose and Milyo estimates aren’t like that. This is not to say they’re definitely wrong, just that any interpretation of them is inherently much more fragile than various more direct measures of bias in communication.

From bias to political outcomes

Groseclose’s model takes several steps:

1. From different rates of mentions of research and advocacy groups, to relative measures of political position of media organizations.

2. From relative to absolute positions: thus, it’s not that Fox News is to the right of CNN, it’s that Fox is near the center and CNN is biased to the left.

3. From absolute positions to the effects of hypothetical changes (for example, how would Americans vote if all their news media had the political slant of Fox News?).

4. From effects of changes in the media environment to the inference about the Americans’ true political positions that they would have if the media reflected their views.

In Groseclose’s endgame, a balanced media might include some TV networks promoting the view that abortion should be illegal under all circumstances and subject to criminal penalties, whereas others might merely hold that Roe v. Wade is unconstitutional; some media outlets might support outright discrimination against gays whereas others might be neutral on civil unions but oppose gay marriage; and on general politics there might be some newspapers that endorse hard-right Republican candidates (0 on Groseclose’s 0-100 scale) whereas those on the left would endorse the positions of Senator Olympia Snowe).

I don’t have much to say about steps 3 and 4 above; as noted earlier, I find it plausible that a Berlusconi-style media environment could shift U.S. politics far to the right, but given the effort it would take to maintain such a system (in Italy, Berlusconi has the power of the government but still has continual struggles with the law), it’s hard for me to think of this as an equilibrium in the way that it is envisioned by Groseclose. This just seems like a counterfactual that would require resources far beyond what was spent to set up Fox News, the Weekly Standard, and other right-leaning media properties.

It is more difficult for me to evaluate Groseclose’s steps 1 and 2. For the reasons expressed by Brendan Nyhan, I am skeptical about the steps used to get the estimates, and for the reasons given above, I worry that the estimates are so indirect that if anything goes wrong in the model, there is no reason to believe them at all. On the other hand, it seems completely plausible that the news media lean left on many issues. I think the most useful approach might be to consider positions on an issue-by-issue basis.

{ 27 comments }

Bob July 29, 2011 at 11:28 am

The irony of course is that Groselclose’s book is biased in the inflammatory title alone. I have a hard time considering the man “respected” after publishing this book.

Andrew Gelman July 29, 2011 at 11:42 am

Bob:

Groseclose’s rhetoric and crowd-pleasing rabble-rousing aside, his book is a serious work of political science. But as I noted above, he relies very heavily on a theoretical model that seems to me a few steps removed from actual media bias. There’s a long tradition in social science of theoretical models, but we have to be careful when making claims about their application to reality.

Paul Waldman July 29, 2011 at 12:54 pm

Sorry to engage in some horn-tooting, but I’d encourage people to read what I wrote about Groseclose’s study here: http://prospect.org/csnc/blogs/tapped_archive?month=06&year=2011&base_name=tomorrows_bogus_liberal_bias_c. Short version: Groseclose’s study on media bias is a methodological train wreck. He has always seemed quite taken with himself for coming up with the indirect method of measuring bias Andrew mentions above. But what should be explained is that the method produces results that are self-evidently absurd. For example, it codes the RAND Corporation and the Council on Foreign Relations as “liberal,” and the ACLU as “conservative,” and so if a newspaper quotes RAND more than the ACLU, it has displayed “liberal bias.” That sounds like a joke, but it’s actually what the study does. At the risk of repeating what I’ve written elsewhere, if a student in his first semester of research methods designed a study that came up with that result, you’d tell him that his instrument had failed the test of external reliability, and he ought to go back and redesign it.

Perhaps I’m biased (!) because my own training is in the field of communication, but I think Groseclose’s problem stems from the fact that he seems unaware that there even is such a field. If someone explained that to him, he might open some communication journals, where he’d learn that there have been a couple of hundred peer-reviewed studies on media bias, utilizing various methods of measurement. But he has always spoken as though he was the first person to really examine this question systematically, which only demonstrates his ignorance. His original study has a half-assed lit review that cites a couple of right-wing web sites, but not a single scholarly article on the question he purports to be addressing. There’s a reason he published his study in an economics journal, and not in a communication journal, or even a political science journal. The reviewers in the latter two would have been likely to understand what the problems were (in my experience, political scientists who work on American politics, electoral politics, etc. are reasonably well aware of the work that goes on in political communication).

There are many different ways one could go about measuring media bias, and some might produce results demonstrating a liberal bias. But if anyone has devised a less accurate, less reliable method than the one Groseclose chose, I haven’t seen it.

Andrew Gelman July 29, 2011 at 10:33 pm

Paul,

Let’s set the ACLU aside–I agree that this organization doesn’t fit so well on a left-right scale. But I can well believe that Rand falls on the left side of contemporary policy debates. Rand is technocratic, they’re all into cost-benefit analyses. That seems to fit in much better with Democrats than Republicans nowadays, at least on issues such as public health, crime control, and military spending.

matt w August 1, 2011 at 10:26 pm

“Rand is technocratic, they’re all into cost-benefit analyses. That seems to fit in much better with Democrats than Republicans nowadays, at least on issues such as public health, crime control, and military spending.”

Isn’t that saying that Rand is left-of-center because they are interested in facts, and liberals are more likely to cite facts?

Erik Nisbet July 29, 2011 at 1:44 pm

As a political communication scholar I would like to echo Waldman’s thoughts about Groseclose ignoring the entire field of communication and the great deal of scholarly social science work, including on media bias, that has been produced over the last 20-30 years. Unfortunately, it is a sin that is widely repeated across political science where research from top communication journals is rarely referenced or cited – whether it the topic media bias, media effects, public opinion, political discussion and social networks, etc., etc.

Jestak July 29, 2011 at 1:51 pm

Brendan Nyhan’s critique covers some, but not all, of the serious methodological problems with both the original Groseclose/Milyo paper and in Groseclose’s book. One big problem he doesn’t cover concerns the way Groseclose identifies the “center” of the electorate. Anyone with even basic training in statistics knows that, if you’ve got a group of people that are ranked in some sort of statistical order, the way you identify the “center” or midpoint of the group is to use the median.

Groseclose, however, doesn’t do this. He defines the “center” of the electorate by first figuring a mean (not median) ADA score for Congress over the period he’s studying (finding a value of about 50), and then assuming that the “center” of congress will precisely reflect the center of the electorate. What he should have done is find a *median* ADA score. I’ve done a few back-of-an-envelope estimates, and it looks to me like the median ADA score for Congress is about 70, maybe a little lower.

If the “center” of the electorate is defined as having a “political quotient,” which is really an imputed ADA score, of 70 instead of 50, all of Groseclose’s big conclusions go out the window. At a minimum his results have a severe sensitivity problem.

anonymous coward July 29, 2011 at 2:41 pm

Even using a median there seems very, very dodgy.

A better way would be to use survey data that included approval or feeling-thermometer scores for people with ADA scores to map from individual respondents’ ideology (or ideal point estimated from some set of questions) onto the ADA. No way could I do this myself, but it doesn’t seem that taxing for people smarter than me — you know that someone has X ideology or Y ideal point, and that they offer a vector Z approval or FT scores for some set of actors with known ADA scores. It should be possible to then back out the respondent ADA score that is most consistent with this pattern.

Then just aggregate. I would bet you my house that you will get an answer that differs strongly from what you get from imputing the center of the American political space by assuming that the mean or median House member = the median American. Given how many steps are between the median American’s attitudes and even the median House member — and worse, how many politically endogenous steps like districting are in there — that assumption is completely untenable.

Bob July 29, 2011 at 4:50 pm

Andrew,

Really what I’m saying is that the title indicates that Groseclose isn’t interested in engaging in serious debate. As a political scientist, he should be ashamed of that title. He’s cast his lot in with the Ann Coulters when he uses an inflammatory title like that. I’d consider this a low point for political science, as I always thought we were better than that.

Andrew Gelman July 29, 2011 at 6:29 pm

Bob:

I disagree. Conditional on Groseclose believing his model, I think it’s quite reasonable for him to pitch it that way. I don’t have the same trust in his model that he has, but that’s a separate question.

I do, however, agree that he stepped over the line in his Fox TV interview when he accused Media Matters of violating “the spirit of the law” for taking the same tax break that lots of other advocacy organizations, left and right, use.

GabbyD July 29, 2011 at 7:24 pm

hi andrew, long time lurker.

you say that his model makes the title conscionable.

but how does his model explain how the left “distorts” the american mind?

does ALL media bias “distort” the mind. what does “distort” mean anyway?

Justin H. Gross July 30, 2011 at 12:19 pm

I will add, having read the book and article, and finding myself quite critical from a methodological perspective, that the tone of the book is far less intemperate than the title would suggest. My bet is that the publisher/editors suggested the title for marketing purposes. Of course, the author takes responsibility for the final product, as well as for the red meat he has tossed to folks at Fox and right-leaning bloggers, but it would be incorrect to imply that that he comes off as a firebrand throughout the book. One could even argue that if the title gives him credibility with a typical right-wing listener of talk radio, such a reader might walk away somewhat more thoughtful about media bias than if he or she had not read the book.

That said, the attempt by right-wing media to pitch this book as “Statistics and Science have spoken: the media are biased!” should raise hackles.

Andrew Gelman July 29, 2011 at 8:44 pm

Gabby:

Under Groseclose’s mathematical model, each voter has a latent true ideological position and also an actual position, which is a weighted average of the media position and the voter’s true position. To the extent that the media are on one side or another, they are (under his model) distorting voters from their true positions.

GabbyD July 29, 2011 at 9:30 pm

ah, so thats what “distort” means.

i guess, i found it inflammatory too. distort has a negative connotation, it supposes subterfuge and manipulation. it has a moral component.

wouldnt “influence” be better? for example, in economics prices “distort” our preferences. i’d love a ferrari; but the prices makes me not want to buy it. but we normally dont use the word “distort” to the function of prices in helping us make consumption choices.

or to take another example: nurture vs nature in social outcomes. would it be right to say that the way we are raised “distorts” us?

frankcross July 29, 2011 at 9:45 pm

Andrew, I disagree about the title and approach. Groseclose is a very serious scholar but he is spinning this to sell books. To make money and maybe get some influence in politics. In person, he does not present even remotely the way he presents in this book. To me, this is selling out scholarliness for a mess of pottage.

Andrew Gelman July 29, 2011 at 10:29 pm

Frank:

Could be, but I have to assume that he believes his model. In the book, he may be cutting corners by not always admitting the potential holes in his argument, but it’s hard for me to believe he would flat-out writing things he doesn’t actually believe are true.

Quack (Ph)Doctor July 29, 2011 at 10:52 pm

I’d like to further echo Waldman and Nisbet’s comment above, and ask Andrew whether, if the book is really a “serious” work of political science, it bothers to cite even one of the hundreds of political science or mass communication studies on bias, as the 2005 article failed to do?

(In fact, the first public-domain version of the article that I saw — a 2004 conference paper — not only failed to cite any bias studies, it dismissed the entire canon with a snotty, derisive analogy to quack doctors. It was the most unprofessional paragraph I had ever seen in a publicly available work by a top scholar.)

I don’t care as much about the title of the book. But I’m concerned about the precedent set by calling something a “serious” piece of science if it turns out that it wholly fails to engage the extant accumulated knowledge (which is why I ask, because I haven’t seen it yet).

Andrew Gelman July 29, 2011 at 11:02 pm

Quack:

I did a quick search and did not find a place where I referred to the book as a “serious” work of political science, so I think your argument here is not with me. I referred to Groseclose as a “respected” political scientist, which I think is correct. His specialty is theory, though, not empirics. That’s one reason I suspect he’s taking his own theory a bit too seriously in his book.

On the other hand, Steven Levitt really is an empirical researcher, and he endorsed Groseclose’s book. But Levitt has a history of endorsing just about anything that his friends say, so I wouldn’t count his endorsement as meaning much.

Quack (Ph)Doctor July 29, 2011 at 11:06 pm

Andrew,

Quoth you, in the second comment above: “Groseclose’s rhetoric and crowd-pleasing rabble-rousing aside, his book is a serious work of political science.”

Andrew Gelman July 30, 2011 at 6:26 pm

Ya got me there! I do think that the arguments in the book have the form of serious social science. Groseclose just believes his model a bit more than I do.

Augie July 30, 2011 at 7:56 am

Hi Andrew,
Did you really just write, “Rand is technocratic, they’re all into cost-benefit analyses.”? Have we really reached the point where being empirically and practically rigorous is an exclusively LIBERAL domain? God help us all. We are really involved in a race to the bottom.

Dave July 30, 2011 at 9:50 am

I don’t get all of the hoopla over the name of the book. First, I don’t think the discipline is better off by having titles that are esoteric and uninteresting to non-academics. Second, it’s not much different than titles of books by some academics on the other side of the political aisle. Take, for example, some recent books by Hacker and Pierson:

Off Center: The Republican Revolution and the Erosion of American Democracy

Winner-Take-All Politics: How Washington Made the Rich Richer–and Turned Its Back on the Middle Class

Justin H. Gross July 31, 2011 at 11:56 pm

In terms of marketing, more troubling than the title is the alternate subtitle on the author’s homepage: “Is our media biased? A leading political science professor provides scientific proof.” It will be fun to see whether commentators who may dismiss evolution as “just a theory” are willing to accept “scientific proof” of something as vague as media bias, based upon a sequence of assumptions and indirect measurements.

Andrew Gelman August 1, 2011 at 7:41 am

Wasn’t there some Fox News personality who denied that the moon causes the tides?

Steve Masy August 1, 2011 at 11:32 am

That would be, I believe, Bill O’Reilly. Not a big fan of science, Bill.

Russell B August 1, 2011 at 2:47 pm

Before I defected and switched my emphasis from American Politics to International Relations, I was interested in the question of media bias and wrote one of my graduate qualifying papers on it. It wasn’t a particularly sophisticated paper, or even a necessarily good one, but at least I can pat myself on the back for having used a “direct” measure — I simply counted the number of times the first 3 grafs of a story devoted more time to the D or R side of a political issue. I chose that “method” after a powerful insight was shared with me by the famous newspaper editor, Walter Burns:

Walter: That’s lousy! Aren’t you going to mention the Examiner? Don’t we take any credit?
Hildy: I’m putting that in the second paragraph.
Walter: Who the hell’s going to read the second paragraph?

(Ben Hecht and Charles Mac Arthur, “The Front Page,” Act III)

I found some slant, though considerably less since 1968 than before, but ultimately I sort of threw my hands up and got stuck on a problem with the entire media bias research agenda, one that I couldn’t resolve to my own satisfaction: The entire thing hangs on a counterfactual.

Groseclose’s hypothesis, like that of all those on the right who gripe about left-wing media bias, is essentially that things would “different” if only the media were “different.” For the life of me, I could never find any evidence that this would be the case — even in the late 19th/early 20th-century period when newspapers were blatantly slanted towards one side or another, there isn’t a significant degree of variation in the politics — there’s never a period of unambiguously clear Democratic or Republican party domination, let alone one that can be causally linked to the tone in news stories. You’d have to really believe that people would consume more media, and consume it well, in the absence of slant — and for good or for ill, I just can’t bring myself to believe that would be the case.

Barry Hardek September 12, 2011 at 10:23 pm

Unburdened by either a political science or communications degree, I am attempting to measure media bias myself, and have developed a crude bias measurement system of my own. My methodology is simple, far different than Professor Groseclose’s, but is equally worthy of criticism.

Not unlike your “R” and “D” scheme, my system amounts to parsing news program transcripts into distinct statements, then having respondents rate whether they are politically-slanted or neutral. An aggregate scoring system then produces a “Bias Index” rating. I’m in the VERY early stages of testing, so feedback is appreciated.

My interest in developing such an index grew from discussions I have had with friends and family, as well as some professional exposure to the media measurement business. On a personal level, I became curious about how intelligent people formed dramatically divergent opinions, and often noticed that they (and I) parroted sound bites from the news programs that they favored. One can debate whether viewers watch a particular new program to develop an opinion or to reinforce one, but repetitive messaging is bound to influence viewers to some degree.

I may be biased, but I think there is a need for such an index. Like nutritional labels on food packaging, patterns of behavior may change with heightened awareness.

Comments on this entry are closed.

Previous post:

Next post: