Arrow’s theorem update

A few months ago I wrote about Arrow’s Theorem (the weak form of Arrow’s Theorem is that any result can be published no more than five times. The strong form is that every result will be published five times), using, as an example, four overlapping papers on the same topic published in different journals by economist Bruno Frey and his collaborators.

The papers were various reanalyses and interpretations of the well-known dataset of survival on the Titanic; see this comment by Antony Unwin for details.

In general I’m not opposed the multiple publication of overlapping material—as I noted in my blog (and my follow-up) on the topic, I think a bit of overlap is ok and can stimulate development and clarification of an idea.

Given all this, I was surprised to learn from this article by Olaf Storbeck that Frey might be disciplined for self-plagiarism:

On Wednesday, 6 July the University of Zurich has started a formal investigation against Frey, based on the “suspicion of unethical scientific conduct”. . . . Bruno Frey (University of Zurich), Benno Torgler (Queensland University of Technology) and Torgler’s Ph.D. student David Savage simultaneously published a series of papers dealing with the sinking of the Titanic, but neither cross-reference their own work nor cite a number of older papers by other researcher addressing exactly the same topic. . . . For several months the authors have been criticised because they simultaneously published nearly identical papers in four different journals without mentioning their other work on the same topic to the editors. This is a clear violation of the submission guidelines of the journals and economists consider this being unethical. The “Journal of Economic Perspectives” rebuked the authors publicly and will publish the conversation with Frey in its forthcoming issue. The editor of the JEBO apparently black-listed the authors with regard to further publications. . . . Frey, Torgler and Savage by no means have been the first economists who econometrically address the survival probabilities of the people sailing in the Titanic. At least five older publications with a very similar research outline and similar results exist [but were not cited by Frey et al.] . . .

In statistics the Titanic example is ubiquitous in textbooks, but when I’ve seen it, it’s usually been presented more as a fun example than a serious bit of social science. So I don’t really hold it against Frey et al. that they didn’t cite (or perhaps even read) various previously published analyses of the Titanic data.

One key question to me in the Frey case is the quality of the work. It’s not so easy to publish a paper in a top journal—let alone, 4 top journals. On the other hand, Antony Unwin wasn’t so impressed by the analysis (see link above), and I respect Unwin’s opinions and judgment.

P.S. Update here: Storbeck’s translation of an interview with Frey from a German-language newspaper.

4 Responses to Arrow’s theorem update

  1. Olaf Stobeck July 12, 2011 at 10:51 am #

    Andrew, thanks a lot for referring to my posts on the issue.

    I’m not per se against using a dataset for different albeit related research questions. However, IMHO, the research question ought to be different, and there should be cross-references to the other papers by the same authors. Both was not the case wrt Frey/Torgler/Savage, from my point of view. The only slight difference was that two of their papers only dealt with the Titanic while two also drew a comparison to the Lousitania. The other papers by the same authors are not mentioned at all, and I guess at least JEP would not have been published the paper had they known that the same material was already published. Additionally, the authors are in clear violation with the submission guidelines of all four journals involved. (A document comparing of the content of the four papers and the submission guidelines of the journals can be found here: bit.ly/frey-et-al )

    With regard to the missing references to older literature. At least the Hall (1986) paper wasn’t just a funny way to teach regressions. http://espace.library.uq.edu.au/eserv/UQ:152940/HallSSM2261986.pdf
    While Frey said in his 20 Minutes interview it would not have been compelling to cite that paper even if they would have known it, Torgler wrote to me: “Thanks for referring us to other studies. We were not aware of them. We haven’t seen the Hall paper published in Social Science & Medicine beforehand, otherwise we would have cited it.”

    JEP editor David Autor also stresses that they should have known about the Hall paper and should have quoted it (http://olafstorbeck.com/2011/07/04/journal-of-economic-perspectives-rebukes-bruno-frey-plus-replys-by-torgler-and-frey/)

    Nevertheless, the big impact the Frey/Torgler/Savage paper IMHO was due to the (apparant) novelty and creativity of their research focus. They might not have know the older papers (a sign for amazingly sloppy literature research,I as a parcel-learnt journo found them within half a day…). If they had known the papers and had cited them, this would have made clear that the basic idea of the research question was 25 years old. This might have cooled down the euphoria of one editor or another.

    What I really find appalling is the weird mixture of sloppy literature research, missing cross-references and frantic eagerness to getting the stuff published as often as possible.

    Cheers
    Olaf

  2. Olaf Stobeck July 12, 2011 at 10:53 am #

    One more question: Do you have an RSS feed just for your posts? I’d like to follow them but I’m not really interested in the other stuff….

  3. Andrew Gelman July 12, 2011 at 11:11 am #

    Olaf: It’s funny; people keep asking that question to John Sides . . .

    Seriously, though: I would usually post this at my main blog, but I’m currently in the midst of 30 days of statistics there and so am posting my non-statistical items here during that period.

  4. Megan Pledger July 15, 2011 at 7:00 pm #

    Going off on a semi-tangent…
    (I didn’t read the papers just Olaf Stobeck’s summary)

    There is an interesting article
    “Women and children first”: Popular mythology and disaster at sea, 1840-1860″
    Journal of American Culture
    January 1, 1997
    B R Burg

    That talks about the myth of ” women and children first” and how and why it played out on the Titanic. I can actually read it at the moment because I am not at work (and I am hoping it’s the right one) but from memory…
    People on the Titanic knew their was a ship nearby (their were telegraph warnings about ice bergs between them) and passengers saw it’s lights while waiting to board life boats so that even when sinking became a possibility they thought they’d be rescued. (It turned out the wireless operator on the other ship turned in before the help messages were sent.)

    Women and children went off first becasue their was some risk of sinking but noone thought their was a risk to surviving (at that point). By the time they’d realised it was too late there was little they could do to survive.

    When their is great risk to survival the usual naval term is “All hands for themselves”. And this was pretty much the general rule.