Federal Funding of Scientific Research Produces Unexpected Successes

by John Sides on September 20, 2013 · 6 comments

in Academia,National Science Foundation,Science


 
Seven researchers, including two Nobel Prize winners, will be honored today at the second annual Golden Goose Award ceremony, celebrating researchers whose seemingly odd or obscure federally funded research turned out to have a significant impact on society.
The awardees will be honored at a ceremony on Capitol Hill, where they will receive their awards from a bipartisan group of Members of Congress.

From the press release.  The Golden Goose website is here. A short item at Inside Higher Ed is here.

Several months ago, I wrote a post called “Why Study Social Science” and said this:

…it’s very hard to determine the value of any research ahead of time.  It’s hard because any one research project is narrow.  It’s hard because you can’t anticipate how one project might inform later ones.  It’s hard because some funding goes to create public goods—like large datasets—that many others will use, and those myriad projects also cannot be anticipated.  It’s hard because some research won’t work, and we can’t know that ahead of time.

The Golden Goose awards illustrate what I meant, as does Robert Putnam’s story.  This is all the more reason why cherry-picking projects that sound “silly” (duck penises, etc.) is not a useful way to evaluate the efficacy of federal funding of scientific research.

{ 6 comments }

mdb September 20, 2013 at 9:53 am

I think you mean the long tail poduces some unexpected results. Federal funding results are rather pathetic otherwise. I won’t even go reproducibilty and other issues with the grant process.

Chris Alcantara September 20, 2013 at 10:15 am

John,

You write that:

“it’s very hard to determine the value of any research ahead of time …. It’s hard because you can’t anticipate how one project might inform later ones. It’s hard because some funding goes to create public goods—like large datasets—that many others will use, and those myriad projects also cannot be anticipated. It’s hard because some research won’t work, and we can’t know that ahead of time.”

If these assumptions are true, then we need to completely change the way grant applications are adjudicated. Either fund every proposal that comes in, or create a process that weeds out the applications that don’t meet certain scholarly standards, and then from that pool, randomly choose which ones get funded until the funds run out. It’s the only fair way to do it given your statements above.

John Sides September 20, 2013 at 10:36 am

Chris: So part of the process that you describe — weed out applications that do not meet scholarly standards — is more or less what is intended by the current process, specifically via the peer review at the heart of federal funding decisions (e.g., at the NSF). Do decisions need to be random after that? I don’t say above that it is “impossible” to discern the future merit of scientific research. Nor do I say that whether projects bear fruit is essentially random. So I lean toward believing that fair-minded peer review can make some headway in determining the future value of research. It’s not a perfect process, of course. But I’m not sure that pure randomness would be an improvement.

Chris Alcantara September 20, 2013 at 11:33 am

Hi John,

True, the task is not “impossible”, but to be fair you do say it’s “very hard”, which means there is still a significant amount of noise and uncertainty in the process. For instance, one of my SSHRC grant applications (the NSF equivalent here in Canada) didn’t get funded until the third time I submitted. The funny thing is, I made very few changes between the iterations. The core methods, theories, concepts, and research questions were entirely the same, as was the trajectory of my research profile!

Given the stakes and goal of producing “valuable” research, I think I would take pure randomness (from the smaller pool) over the current system. At least that way, some of the idiosyncrasies involved in the adjudication process, such as personal preferences and networks, and/or the reliance on reputation and research record, are somewhat mitigated.

Kavner September 20, 2013 at 11:56 am

If some groups strongly favor Federal funding for scientific research… there’s a firm tendency to highlight successful projects in that category, while ignoring the unsuccessful projects, and the overall costs of the program.

The Federal government spends roughly $140 Billion per year on scientific research. Only a small proportion of that amount objectively results in “significant benefits” to the American public paying the bill. And, opportunity-costs are extremely large in the highly subjective selection of projects to be funded. There are also economic alternatives to Federal funding of desired scientific research.

“The root of all superstition is that men observe when a thing hits, but not when it misses.”
— Sir Francis Bacon (Father of the Scientific Method)

BP September 20, 2013 at 2:51 pm

Kavner, until you’ve quantified the benefits that have come from federal funding of scientific research through the decades, your statement is meaningless. Of course “only a small proportion of that amount objectively results in ‘significant benefits’ to the American public” but that small proportion arguably results in massive, massive benefits.

Come back and let us know when you’ve quantified the social welfare that has resulted from all advances that resulted from federally-funded scientific research and then accurately divided that social welfare up amongst the component parts of the research that allowed the specific breakthrough. We’ll be waiting with bated breath.

Comments on this entry are closed.

Previous post:

Next post: