Roger Simon’s Ignorance about Polling

by John Sides on December 22, 2011 · 4 comments

in Public opinion

Roger Simon, the Chief Political Columnist for Politico:

I have never been called by a political pollster and don’t know anybody who has, but I know some pollsters, who assure me they don’t make the numbers up, and I believe them.

From George Gallup’s mock Q-and-A in A Guide to Public Opinion Polls, which he wrote in 1948:

“Why haven’t I been interviewed?  Why have I never heard of anyone who has been interviewed?”

These questions come up frequently in connection with modern public opinion surveys, because people do not understand how it is possible to get an accurate measurement of public opinion when only a small part of the total population is interviewed…In this respect, modern surveys merely apply to public opinion research certain well-established procedures which have been used for years in the fields of engineering, medicine, education, and all the social sciences.  When an engineer wishes to judge the quality of ore in a mine, he examines a few “samples.”  From these samples he makes a highly accurate estimate of the amount and quality of ore in the mine.


Simon again:

We are a nation of nearly 313 million people. So how many people did the pollsters actually speak to? If you have extremely good eyes, you can find the answer in tiny type at the bottom of a chart: The Post-ABC poll was conducted by phone “among a random sample of 1,005 adults.”  That represents 0.0003 percent of the nation at large.

Gallup:

“How many persons have to be included in a poll to obtain reliable results?”

Size and accuracy are inextricably linked in the minds of most laymen.  Invariably the first question that is asked by persons who examine the results of a public opinion survey is: “How many persons were included?”  Actually, the size of the sample (the number of persons interviewed) is far less important as a factor in achieving reliable results in modern polling than several other factors, among the most important of which are the accuracy with which the persons chosen to be interviewed mirror the total group, the wording of the question or questions used to develop the information and the accuracy and lack of bias or influence in the interviewing procedure itself.


Apparently, we haven’t come that far in the 60+ years since Gallup wrote his book.  Again, Simon is the Chief Political Columnist for Politico.

For more, see Alex Lundry, Charles Franklin, Nate Silver, Seth Masket, and quickmeme.

{ 4 comments }

Andrew Gelman December 22, 2011 at 3:59 pm

Hey John. The funny thing is, someone sent that article to me too, but I posted my mockery at the sister blog because I worried it was too silly for the Monkey Cage!

Matt Singer December 23, 2011 at 12:53 am

I have been polled for state politics media polls at least once, and I was just polled earlier last week by a pollster whose client appeared to be a teacher’s union. A couple years ago I got polled by a company evaluating appeals Walmart could use in doing site expansions (one of the highlights of getting polled is to figure out the models and manipulations being used).

But I think this might just capture is: http://dialogic.blogspot.com/2010/01/phd-comics-dear-news-media-when.html

Melis February 10, 2012 at 8:08 pm

Its a mksatie to push it anytime. Cap and trade is a total scam. All this scam is waiting for is for government to allow it to occur.

Brett Gall December 24, 2011 at 7:22 pm

Today, Matt Yglesias tweeted, “Increased political polarization is a straightforward consequence of a better-educated, better-informed electorate.” As a former student of yours, I suspect you would strongly disagree. Care to comment or write a post on it?

Comments on this entry are closed.

Previous post:

Next post: