“No Revision” Journal Policy – Thoughts?

by Joshua Tucker on August 29, 2012 · 10 comments

in Academia

Here’s something for all of you political scientists who suddenly have extra time on your hands today to ponder. The journal Economic Inquiry has adopted a “no revisions” option when you submit papers to the journal. Here’s how the process works:

Under this policy, an author can submit under a ‘no revisions’ policy. This policy means exactly what it says: if you submit under no revisions, I (or the co-editor) will either accept or reject. What will not happen is a request for a revision.

The editors and co-editors will ask referees: ‘is it better for Economic Inquiry to publish the paper as is, versus reject it, and why or why not?’ This policy returns referees to their role of evaluator. There will still be anonymous reports. Authors who receive an acceptance would have the option of publishing without changes. If a referee noticed a minor problem and put it in the report, self-respecting authors would fix the problem. But such fixes would not be a condition of publication.

The justification given for the option is that:

Journal time to publication lags has become embarrassing. Many authors have 5 year submission-to-print stories. More insidious, in my view, is the gradual morphing of the referees from evaluators to anonymous co-authors. Referees request increasingly extensive revisions. Usually these represent improvements, but the process takes a lot of time and effort, and the end result is often worse owing to its committee-design. Authors, knowing referees will make them rewrite the paper, are sometimes sloppy with the submission. This feedback loop – submitting a sloppy paper since referees will require rewriting combined with a need to fix all the sloppiness – has led to our current misery. Moreover, the expectation that referees will rewrite papers, combined with sloppy submissions, makes refereeing extraordinarily unpleasant. We – the efficiency-obsessed academic discipline – have the least efficient publication process. The system is broken.

I’m interested in what readers think of this policy. Is this an option you would like to see in political science journals as well?

{ 10 comments }

Andrew Gelman August 29, 2012 at 7:31 am

Josh:

This is not a new policy on the journal’s part. I published a paper there under the no-revision policy a couple years ago. Actually, I submitted 2 papers: one got immediately rejected and one was accepted. Annoyingly, they charge a fee for submitting an article for publication, but I could live with that, given that it’s standard practice in econ journals (although not in stat journals).

Joshua Tucker August 29, 2012 at 8:00 am

Sorry – didn’t mean to imply recently adopted…

Eric L. August 29, 2012 at 9:01 am

This sounds like a policy to avoid Bruno Frey’s publishing as prostitution problem.

Frey, Bruno. 2003. “Publishing as Prostitution? – Choosing Between One’s Own Ideas and Academic Success” PUBLIC CHOICE 116(1-2), 205-223

Tracy Lightcap August 29, 2012 at 9:55 am

I can see up and down sides.

Up side = this will help get rid of the persistent “reviewer # 3″ problem. Often revisions are simply a way for one of the reviewers to deep six a paper over a long reach of time. Today, if I get a revise and re-submit where one of the reviewers obviously doesn’t like the piece, I write the editor back, say I’ll consider the more favorable reviews seriously, describe what, if anything, I’d be ready to do concerning the unfavorable reviewer’s problems, and essentially ask the editor to do her job and decide what she’ll accept. If she doesn’t want the piece, fine; I usually have a second venus set up beforehand. This process saves a lot of time, time I don’t have to waste trying to guess where the goalposts will be moved next by reviewers who, as Lord Keynes once said, simply refuse to understand me. It also makes it more difficult for reviewers to forget what they said in initial reviews. This, for me, is perhaps the most infuriating part of the review process. I have put in resubmissions that meet reviewer’s suggestions only to have the very same reviewer object to the correction he originally suggested! They didn’t re-read their initial review before writing a subsequent one. Editors are also at fault here; they often don’t re-read initial reviews to see how corrections meeting one reviewer’s objections will generate new objections by another. This kind of sloppiness would be over the side with the process described above.

Down side = I’ve found my papers substantially improved by conscientious reviews and revisions. Indeed, even the rejections I’ve had have often proved useful; one of my best pieces was rejected twice until I rethought the theoretical basis for the work. I’d have never down that unless I was forced to and, while the initial work came close to acceptance both times, it ended up being much better for being rejected.

Bottom line = I’m not sure what the final average effect of such a change on academic production would be. Conscientious editors could probably speed up the review process considerably with little effect on the final product. Less conscientious editors would probably be presented with an option I would not trust them to exercise with attendant unpredictable effects on the journals in their charge. But, hey, if people want faster reviews and are willing to take the consequences, this would be one way to get them.

Gina Sapiro August 30, 2012 at 8:39 am

The reviewer #3 problem is probably better dealt with by editors being clearer about which reviews are serious rather than eliminating the possible benefits of blind review advice.

Good example: For a piece of mine a “reviewer #3″ clearly just hated and attacked in one of those “I know so much more than you” ways, my favorite criticism was that I clearly didn’t even know the literature, mentioning the critical pieces I didn’t even mention. Indeed, as I matured I tended not to do a dragnet of all possible tangentially related articles in my footnotes. But in this case, one of the key examples of something critical I obviously “didn’t know” was written …. by me! And while it was in the same field, it wasn’t actually directly related to the point. The editor, of course, would have known that.

But I know some of my work was improved in more than minor ways by reviewers.

Jonathan Baron August 29, 2012 at 10:12 am

If reviewers simply vote yes/no, the argument for the existence of a journal is weak. The alternative is to have a place on the Web where articles are posted, with tallies of “like/dislike” votes. Perhaps we could have a separate tally from members of a selected group (analogous to a board of reviewers). Such a procedure would save enormous amounts of money (mostly paid by university libraries) and some amount of time.

As editor of Judgment and Decision Making, I consider the main purpose of reviews to be to help produce better papers. I try to avoid sending a paper to reviewers when it is likely to be rejected. I do not consider reviews as votes, and I solicit only two, so that reviewers do not feel that their efforts will be diluted. Reviewers are thus playing a similar role to members of a PhD thesis committee. (Theses are rarely rejected but often revised.) Although I am certainly biased, my impression is that this process works well.

Pippa Norris August 29, 2012 at 10:41 am

Nowadays, probably like 99% of colleagues, I never read ‘a journal’ from cover-to-cover, or even care that much about the status of the journal where something is published; I use the Web of Science SSCI to trawl for ALL relevant articles and I prioritize those which are most highly cited by the sub-field. Thus there is already a filtering mechanism for quality control. It comes from what colleagues regard as important once an article has been published. The traditional editorial process is hopelessly out of touch with how scientific journals are used today, producing endless delays, and often resulting in a mish-mash of watered-down, over-specialized, cite-everything-but-read-nothing, and out-of-date articles cited by 2-3 friends and the author’s mother and otherwise sadly unread. I can publish a book with a top university press in less time than it takes to get a short journal article out. Give me a straight journal editorial yay or nay and I know how to proceed. And I speak from experience as a former journal editor and member of dozens of editorial boards.

Jason MacDonald August 29, 2012 at 10:47 am

I had the privilege of working as an editorial assistant for a great journal editor. What he did was offer R&Rs for manuscripts when “feasible” and “straightforward” revisions would lead to a publishable paper. And he made this well-known through his decision letters to authors that were also sent to reviewers. This practice reduced the incentive that authors had to send in shoddy manuscripts (one concern above) to this journal. If you sent in a manuscript that needed work-even if it had the potential to make a big contribution-it was going to get rejected. In practice, it eliminated the problem of revisions dragging on for years (another concern above). It also allowed authors to benefit from good comments from reviewers. Even if their papers were not published in this journal, the papers would be better because of reviewers’ comments and would be more likely to be published elsewhere.

So, I guess I’d prefer practices along these lines to the no revisions alternative.

One thing I’d note is that this journal received a ton of submissions and could afford to be “choosy” in accepting papers.

Sebastian August 29, 2012 at 10:53 am

I think this is much less necessary in political science than in economics – our journal lag times are a fraction of theirs.
I have made some very good experiences with R&R comments (along the lines of what Jonathan and Tracy describe above), so I wouldn’t want to miss them. One thing that I have seen & liked at Socio Economic Review (which is apparently doing quite well, especially for a new-ish journal) is that editors decide over R&Rs rather then send them back to the original reviewers (I’m not sure this is a general policy, but I know of at least two people with the same experience). That saves a significant amount of time in the process. They were then also pretty quick to get the article out online first.

Paul Cairney August 30, 2012 at 2:18 pm

Not convinced by the justification. If sloppiness is the problem, editor should reject before the review stage. If time lag is the problem, introduce online advance access. If the third reviewer is the problem, the editor should give a clear steer on what the author should do. Paper citations as an indicator of quality may solve one problem but not the problem as stated (since it will take a few years to build up citations), and it is vulnerable to the same cite-everything-read-nothing problem. I am also not sure that the editors have considered both aspects of the rule of Unintended consequences – at least we know what they are under the current system. Finally, how much receptivity to this idea results from us no longer wanting to be vulnerable to often dispiriting (but often useful) anonymous criticism?

Comments on this entry are closed.

Previous post:

Next post: