Panel Discussion on Falling Response Rates

by John Sides on April 23, 2013 · 3 comments

in Public Events,Public opinion

Policy Makers & Businesses Need Reliable Information and Data: The Impact of Falling Response Rates to Social Surveys and What Can Be Done.  Sponsored by Senator Jay Rockefeller and Congressmen Chaka Fattah and Charles Dent. Friday, April 26, 10:00 AM to 11:30 AM, Capitol Visitors Center room SVC 209-08

Organized by AAPSS in collaboration with the Annie E Casey Foundation, the Russell Sage foundation, Sage publications and the NRC’s Committee on National Statistics, the briefing will review the state of survey research and its role in the federal statistical system, assess the nonresponse challenge and discuss alternate approaches for providing more reliable data at less cost.

Douglas Massey, President of the AAPSS and Roger Tourangeau, Vice President of Westat, will provide a review of the state of survey research nationally and the nonresponse threat to survey validity. Then a panel of experts will discuss the implications of their presentation for federal, state and local policy and for business development. The panel includes Kenneth Prewitt, former director of the U.S. Census Bureau, Paul Emrath, National Association of Home Builders and David McMillen, formerly of the Census Bureau. Discussion will be moderated by Chicago Tribune columnist Clarence Page.


More information here.  And here is a report by the American Academy of Political and Social Science: The Nonresponse Challenge to Surveys and Statistics.

{ 3 comments }

RobC April 24, 2013 at 1:12 am

Perhaps this is an appropriate post in which to raise the question whether social scientists should disclose in their publications the response rate to surveys they’ve conducted. A similar question is whether public opinion polls should disclose the response rate. My sense is that response rate is rarely disclosed. If I’m wrong about this, I hope you all will educate me.

It seems to me that this information is desirable to help make a judgment about how much faith to put in the survey. The only reason I can think of not to make the disclosure is that it can be a little embarrassing to show a low response rate and may tend to undermine confidence in the findings. That seems like a reason that’s contradictory to the principles of scientific inquiry. Again, if I’m missing a valid reason not to make such a disclosure, I’m eager to hear it.

John Sides April 24, 2013 at 7:20 am

Most public polls do not report their response rate. At least one scholarly journal, Public Opinion Quarterly, requires that any published article report the response rate. You’re probably right that reporting it would undermine confidence in the result. That said, a low response rate does not imply any bias in a survey:

http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/

caulden April 24, 2013 at 12:19 pm

Response Rates are at a crisis level, not merely “falling”.

Social Scientists don’t seem alarmed; published data generally seems more important than truthful survey research data.
A comfy “panel discussion” approach to the issue indicates how seriously it’s taken.

Thirty years ago a 70% Response Rate was generally considered the minimum acceptable level for academic survey research; researhers today are lucky to get half that rate. Commercial pollsters/surveyors are worse. The 2012 Pew Research Methodology Study pegged typical Response Rate at 9% (year 2012), steadily down from 36% in 1997.

Of course pollsters are highly embarrassed and fearful of disclosure. They claim to deliver scientific probability-sample based data — but actually deliver data of unknown quality/accuracy based on convenience-sampling.

Comments on this entry are closed.

Previous post:

Next post: