Why aren’t there IRB’s for the development industry?

I recently attended a talk by Ted Miguel presenting results from an  experimental evaluation of a “community driven development” program in Sierra Leone that he conducted with Katherine Casey and Rachel Glennerster.  It’s a terrific and very interesting paper and project, but that’s not what I wanted to post about.

After the talk, a colleague of mine, who was impressed by the paper and who knew that I was involved in a similar randomized evaluation of a “community driven reconstruction” project in Liberia, said something along the lines of “Still, I don’t know, isn’t there something about these experimental evaluations that makes you a little uneasy, kind of like we’re treating these people like rats in a maze?”

I said, in so many words, “yes, definitely.   But we tell ourselves it’s for a good cause [which I think it is – figuring out what sorts of development aid actually produce good results], and in any event it’s nothing compared to what the development industry itself is doing on this score.”   By which I meant that the little experiments and surveys and measurement devices being used in randomized aid project evaluations are like nothing compared to the billions of dollars of attempted social, political, and economic engineering undertaken every year in donor-funded development projects.

For our little experimental evaluations, we have to have our research approved by university “institutional review boards” (IRBs) that assess whether what we are proposing is ethical in terms of treatment of and risks to the subjects.  One often hears grousing about IRBs asking clueless questions or demanding unrealistic procedures for obtaining subjects’ consent for surveys and such.  But no one I know thinks IRBs are a bad idea in principle – quite the contrary.

Why is there nothing like an IRB for development projects?   Is it that aid projects are with the consent of the recipient government, so if the host government is ok with it then that’s all the consent that’s needed?  Maybe, but many aid-recipient governments don’t have the capacity to conduct thorough assessments of likely risks versus benefits for the thousands of development projects going on in their countries.  That’s partly why they have lots of aid projects to begin with.

Or maybe there’s no issue here because the major donors do, in effect, have the equivalent of IRBs in the form of required environmental impact assessments and other sorts of impact assessments.  I don’t know enough about standard operating procedures at major donors like the World Bank, USAID, DFID, etc, to say, really.  But it’s not my impression that there are systematic reviews at these places of what are the potential political and social impacts of dropping large amounts of resources into complicated local political and social situations.

Maybe this is a bad question to ask because we aren’t likely to be able to even guess at all the political and social impacts of specific aid projects, and we don’t want a too-rigorous application of “Do No Harm” to stop us from trying to do anything positive at all.

That could be right, but I’m still struck by contrasts like the following:  For a survey conducted as part of an aid-program evaluation, the IRB process will ask a researcher if any survey questions might cause discomfort or recollection of highly unpleasant events for the respondent, and if so what are the procedures for preventing this and whether the potential benefits outweight the potential risks.  By contrast, a specific aid project could, by introducing a pile of valuable resources, lead to intra- or inter-village competition that might, for all we know, contribute to people getting killed.  Those risks might be  outweighed by the potential benefits for most projects, but are there any institutionalized procedures for asking the question in the development sector?  Should there be?

By the way, leaving aside the question of IRBs for aid projects, the World Bank and other major donors carry out a great deal of research that involves collecting data on, and sometimes manipulating, human subjects.  Again I might be wrong, but as far as I know this research doesn’t need or get something like IRB approval.  If you think it’s a good idea for academic researchers, what’s the argument for why donor research would be different?

One Response to Why aren’t there IRB’s for the development industry?

  1. Ken May 26, 2011 at 10:58 pm #

    I am surprised to hear that there is no IRB-equivalent for RCT interventions. I guess one could say that it is incumbent upon host governments to ensure that projects done by the Bank or other NGOs meet minimum standards of respect for human subjects. That said, I agree with you that there should be controls within academia to not allow academics to work and publish on controversial interventions. I am not that familiar with the RCT literature (the few books and papers that I have read on interventions – mostly in Africa – seemed innocuous) but do you have examples of interventions/studies that you found particularly alarming?