The Critics of Social Science Are Sometimes the Best Defenders of Social Science

Jun 14 '12

Washington Post columnist Charles Lane’s opposition to National Science Foundation funding of the social sciences elicited various rebuttals on this blog and elsewhere (e.g., here, here, and here).  Yesterday Lane elaborated his critique here.  Lane’s critique centers on his beliefs that social science research is not rigorous enough to warrant federal funding.  As he put it in his original column:

bq. Though quantitative methods may rule economics, political science and psychology, these disciplines can never achieve the objectivity of the natural sciences. Those who study social behavior — or fund studies of it — are inevitably influenced by value judgments, left, right and center. And unlike hypotheses in the hard sciences, hypotheses about society usually can’t be proven or disproven by experimentation. Society is not a laboratory.

In his post yesterday, Lane emphasizes that he finds the social sciences admirable and worth paying attention to:

bq. I am an admirer, and, as a journalist, a frequent consumer, of quantitative analyses of social data ranging from election results to consumer auto buying habits.

I want to follow up on that point by delving a bit into Lane’s work.  I thank a Monkey Cage reader for suggesting this.

In 2010, Lane published a book about the death penalty, Stay of Execution: Saving the Death Penalty From Itself, that argues for keeping capital punishment but also for radically reforming it — “shrinking the death penalty in order to save it,” as one reviewer summarized.  In making this case, Lane draws on social science and indeed apparently learned from federally funded social scientists.

In the acknowledgments, Lane writes:

bq. Other researchers who graciously shared their expertise include Professor Theodore Eisenberg of Cornell University Law School, Professor Scott Phillips of the University of Denver, Professor Nancy King of Vanderbilt Law School, Kent Scheidegger of the Criminal Justice Legal Foundation, Professor William J. Bowers of the State University of New York at Albany’s Capital Jury Project, Professor James Alan Fox of Northeastern University, and Professor Christian Boulanger of the Humboldt University in Berlin.

Professor Eisenberg has been the recipient of NSF funding.  As has Professor Phillips.  Professor King has received funding from the Department of Justice.  Professor Bowers received the initial NSF grant that led to the Capital Jury Project.  According to the Project’s website, it has received other NSF funding as well.  Finally, Professor Fox has received federal funding in many instances (see his vita), such as from the Department of Justice and the Department of Health and Human Services.

In the book, Lane cites social science research in various places.  Here are some examples:

* This 2004 article by John Blume, Theodore Eisenberg, and Martin T. Wells in the Journal of Empirical Legal Studies.

* This 2002 article by Eisenberg in the Cornell Law Review.

* This 2005 article by John Donohue and Justin Wolfers, which Lane describes as an “excellent review.”

* This 2008 article by Scott Phillips.

* This 2000 article by James Liebman, Jeffrey Fagan, and Valerie West.

I chose those articles because they look like garden-variety social science — hypotheses are formulated and then tested with quantitative data, etc., etc.  The book also cites other studies, copious government statistics, and public opinion polling — all of which often rely on findings from social science as well, although its contributions are often behind-the-scenes (e.g., work in survey methodology).

Now, to be clear, the specific studies cited by Lane were not federally funded as far as I can tell.  In the studies above, scholars often thank their home institutions for providing support — a means of funding the social sciences that Lane approves of.  But Lane does thank scholars that have been funded by various government agencies.  Their federal grants no doubt enhanced their expertise, which likely enhanced their contribution to Lane’s book.

Moreover, given Lane’s critique of social science as more “subject to the kinds of political and ideological influences, acknowledged and unacknowledged, to which taxpayers and those who represent them might legitimately object,” it’s fair to ask: does he think that the social science he cites in the book is ideologically influenced?  Does that affect the quality of the argument he is making — which to be sure, is a normative argument, not an empirical one — but nevertheless an argument that uses copious empirical evidence for support? My guess is that he thinks the social science he relies on is objective — not close to “literary criticism,” as he suggests in his post.

Given the political controversies that swirl around the natural sciences — e.g,. the study of evolution, climate change, pollution, etc. — I continue to doubt that the social sciences are more vulnerable to bias than are the natural sciences.  I use the general term “bias” because what threatens scientific inquiry is not political or ideological bias per se, which is Lane’s focus, but any kind of bias that might lead the researcher to manipulate data in support of the hypothesis that he or she favors.  The political scientist Arthur Lupia makes a good point along these lines:

bq. Of course, in most fields of science, a scholar’s view about how the world “should be” affects his or her choice of research question and the answers they hope to find.  The number of scholars working on topics such as global warming and social inequality are cases in point—how many of them hope to increase the rate of warming or inequality?  That people make such choices, however, does not threaten the credibility of science.  The threat begins with a researcher’s ideology substitutes for available data or normal means of scientific inference. The threat is realized if members of the discipline lack the ability to review the relationship between the research methods and the submitted findings.  A discipline that does not require its practitioners to submit their findings to rigorous internal and external reviews is susceptible to becoming an ideological mouthpiece for its practitioners.

Unless social scientists are somehow more dishonest than natural scientists, or the review processes in social science are less rigorous than those in the natural sciences, I do not think Lane’s concerns about political or ideological bias are merited by the available evidence.

Lane’s use of empirical research in his book speaks to another issue: the public value of social science research.  In his post yesterday, Lane writes:

bq. …the natural sciences are more likely than the social sciences to lead to practical applications whose benefits will be both widely shared and widely recognized.

In my email to him, I described research on the factors associated with civil war, which, given the human costs of civil war, arguably has “practical applications” with “widely shared” benefits.  Lane did not reply to that email or mention my point in his post yesterday.  So let me try again, using Lane’s subject matter.  Would a study that seeks to understand why blacks are actually under-represented on death row, relative to their representation among convicted murderers, have a practical application?  Would a study that seeks to estimate the percent of capital defendants whose convictions are overturned because of errors?  Would a study that seeks to estimate the effect of the death penalty on the crime rate?  These are questions investigated by the studies he cites.

Of course, sometimes social science research, like natural science research, does not arrive at a conclusive answer.  Lane cites the study by Donohue and Wolfers, which demonstrates no clear connection between capital punishment and the crime rate.  He seems to believe this finding represents the essential limitations of the social sciences, writing in his post yesterday that “Social scientists are as divided over whether the death penalty deters murder as they were in Cesare Beccarria’s time.”  (Lane suggests that science needs to “settle” questions — e.g., the earth revolves around the sun — to be truly valuable.  If that is his standard, the rest of the NSF, to say nothing of the CDC, the NIH, etc., better watch its back.)  But the point of Donohue and Wolfers is to address those divisions and, ultimately, in this “excellent review” to show the tenuousness of the connection.  And knowing that the connection is tenuous has important “practical applications” as states debate whether to continue to prosecute lengthy and expensive capital murder cases.

Reading Lane’s book, I couldn’t help but be reminded of Senator Tom Coburn, who, about 2 years after he called for the elimination of the NSF political science program, cited NSF-funded political science research.  Once again, the case for the public value of social science research ends up being made, and made convincingly, by its critics.