The New York Times has a front page article today devoted to web alternatives to peer review. The most interesting part is a description of a trial by the respected journal Shakespeare Quarterly:
[..] the journal posted online four essays not yet accepted for publication, and a core group of experts — what Ms. Rowe called “our crowd sourcing” — were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal, due out Sept. 17.
This is of course not an alternative to peer review but an alternative way of doing peer review. The system is interesting in that it takes advantage of the speed and accessibility of the web, which lower the cost of engaging in debate. Yet, a web-based system that would allow for broad-based access and anonymous comments would surely make the cost too low, granting academics the opportunity to affect the reputation of others without reputational consequences for themselves.
It seems like an experiment worth trying in political science although I doubt it will replace anonymous peer review, which remains Ralph’s Pretty Good Grocery of academia. There are, of course, opportunities to combine this system with peer review, for example by creating a forum in which scholars can leave signed comments on published articles. This obviously wouldn’t transform the gatekeeping role of peer review but may allow for more lively debate on published pieces.
Another thought is that web-based review may be particularly useful for shorter articles on policy questions. Policy pieces often have an urgency that makes the peer review process unattractive. There are, of course, outlets for publishing policy pieces but these tend not to be academic and count little towards tenure and so on. I could envision an on-line system where policy articles are published and commented on by senior scholars, thus giving the article academic credibility and stimulating debate among academics about policy issues. Economists have something like this with Vox (although the model is somewhat different). any other thoughts on how web-based systems may create alternatives to traditional peer review?