Rassmussen Reports has a lead story on its website trumpeting the fact that 43% of the country thinks Obama’s trip to Copenhagen is a bad idea. Fair enough, but my immediate suspicion was that the vast majority of the survey respondents holding that opinion would be people inclined to see anything Obama does as a bad idea. So I set off to try to find some cross-tabs for this question (being particularly interested in partisanship and region), only to find the following announcement:
Crosstabs and ( sic ) are available to Premium Members
Premium members, it turns out, are those willing to pay $19.95 a month, which immediately raised two questions in my mind. First, is there anyone out there who is willing to pay $19.95 a month for cross-tabs from Rasmussen? Second, how bad must the rest of their business be going if they have been reduced to selling cross-tabs on line?
More seriously, what is the responsibility of polling organizations that provide marginals that will clearly be quoted in news stories to also provide cross-tabs so we can begin interpret exactly what those marginals mean? Let me then add my voice to other bloggers following public opinion (here and here) calling for survey firms to make more of an effort to provide pertinent cross-tabs with their survey reports (and especially cross-tabs for partisanship and region).
{ 10 comments }
You might be interested in Paul Burka’s blog (http://www.texasmonthly.com/blogs/burkablog/?m=200909) on another Rasmussen poll with ‘hidden’ results. Apparently Rasmussen reported the race between Sen. Hutchinson and incumbent Gov. Perry for the Texas Republican gubernatorial nomination to be a dead heat. Rasmussen reported the Perry was leading Sen. Hutchinson by more than 10 points early this summer. So what accounts for the closeness of race? Unreported in the news accounts of the second Rasmussen poll, and only available to subscribers of the service, is that Rasmussen conducted two surveys, one of likely Republican primary voters and another of ‘other’ voters. These ‘other’ voters appear not to be likely Republican primary voters, but account for the closeness of the primary election. Of course all of this information was not provided in the news release from Rasmussen and again is only available to those who subscribe to the Rasmussen service. I wonder who is at fault here? The news outlets that printed the story without inquiring about the sample or Rasmussen?
Get one of your universities to run a poll with all the x-tabs for you.
These survey research firms are in business to, duh, make money. They don’t do charity for blogs, or academics, or the press. If the press quotes them, it doesn’t mean they owe the press anything. It means the press needs desperately to fill space (air time, column inches, etc.) No obligation is owed the press for a quote.
So stop griping about their charges, or get a real job so you can pay the fee.
Forget the cross-tabs. I want raw numbers.
… as to survey firms being businesses. Of course. No obligation is owed to the press or anyone else, but no obligation is owed to the survey firm, either. Survey firms love the coverage, which is why they send out (lots and lots of) press releases. All this post is saying is don’t trust anyone who only gives you half the numbers. If their business model is to only give marginals, they are delivering a lousy product, which is how it should be treated. If their business model is to give the numbers to only those who pay, then they should get out of the business of providing numbers to journalists for publication.
Hans, What complete nonsense. You’re obviously a political scientist in academia.
The whole idea behind advertising is to get people to /buy/ the product.
You put some incomplete numbers out there precisely to attract attention, and get people to buy. It’s a sampling to generate interest and curiosity.
It’s up to those reading whether they want to use those numbers in a story, or quote them. Of course the idea is to get press attention and coverage. That’s what generates business.
We wouldn’t expect other businesses to provide their product for free.
Suppose we were in Crate and Barrel and the clerk gave me a small sample of something to eat or drink. It tasted great, or I’d at least like to try some more. I wouldn’t then jump up-and-down and demand a bottle of the stuff for free! I’d go and pay for it. That’s the whole idea.
Some of these posts give political science a really bad name.
Or another, better, example:
A political scientist wouldn’t write one of their silly little books for a print run of 300 copies, have it reviewed or excerpted in /Perspectives/ or in some other journal, and then be expected to hand out free copies when readers complain that they hadn’t seen enough from the excerpt.
Political scientists are in the business of producing information too. Now it may be the case that only 200 people (university librarians) are interested in buying one of these poorly written books. But if only the table of contents is shown on Amazon, and someone wants to see more, they wouldn’t piss and moan on a blog about having to pay Amazon to actually buy the book. That would be considered petty and ridiculous.
So here we have firms in the business of producing information of a different sort — surveys — and the political scientists on this blog are saying they should be forced to fork over the entire product for nothing?!
Outrageous!
I’m sure Hans can take care of himself very nicely here, but I’ll simply point out the distinction between news coverage and advertising. If Rasumussen wants to buy an ad in a newspaper, it can say pretty much what it wants, within the bounds of legality, taste, etc. If it wants to claim that it has something to say that a newspaper should want to publish, then it should be open to providing further inquiry. I’m not blaming Rasmussen here so much as the media folks who take what the pollster feeds them (the marginal) and ask no questions that would make that information interpretable. Refusal on the pollster’s part to provide such information should make the media shy away from whatever bits and pieces the pollster is putting out.
The difference between the poll data and the academic book example is that the poll data may be heavily biased and without the cross-tabs no one will discover this. Many other kinds of partial information will not be complete, but the information that you do have is interpretable without the hidden remainder.
The proper analogue to the book example is if the cross-tabs were available, but only a subset of all of the asked questions were free. Then the free info would be usable.
How complete is “complete?” Someone can say, gee, I’d like to see another table, but then complain that that still isn’t enough. So they demand more, and more, and more detail. That potentially places an incredible burden on a business that is supposed to make money from a product it sells — information.
Moreover, @ Lee, the press is itself a paying customer. Blogs, networks, major newspapers, all pay for data — routinely. The clients can evalutate for themselves whether the product is bad, biased, substandard.
For all major media, there are budgets for data purchase, data collection, and research. If there is a demand for more detail, it’s the press that should pay for it, and then release those details (or not). If the press doubts the veracity of the sample tabulations that are put forward in publicity releases, they shouldn’t publish them in the first place.
Ultimately, bad pollsters are exposed by clients who have been ill-served. The marketplace works.
As a former journalist, I have to think there’s a misunderstanding about how most polls are reported in the news.
Newspapers conduct their own polls, and they also _report_ on other polls, because those polls are news. Usually this is not paid. Whether the journalist should pay for the “whole” story is a red herring, but as a general practice, journalists rarely pay sources.
The only claim anyone is making here is that a single poll number reported by someone else — for free — should not be regarded as news without more context. If the journalists fail to get the context, they are not doing their jobs. More broadly, the world would be a better place the more we knew about the polls.
I think political scientists like to whine and bellyache because their “expertise” is rarely called upon, and they are mostly irrelevant to politics, political debate and political newsmaking.
This discipline writes poorly and obscurely for tiny audiences, partly because it doesn’t attract very smart people, and partly in order to guard its feeble claims to special skill and knowledge. But then it turns around and complains that its expertise is irrelevant and goes unconsulted in news reporting and newsmaking.
This is a boneheaded contradiction that will not be resolved soon.
So let the whining continue, I guess.
Comments on this entry are closed.