I was thinking some more about those two recent studies (see here and here) published in Psychological Science, “the highest ranked empirical journal in psychology.”
Just to relive these for a moment, the papers are:
“The Ancestral Logic of Politics: Upper-Body Strength Regulates Men’s Assertion of Self-Interest Over Economic Redistribution”: 3 surveys, two of which were of college students, no actual direct measures of upper-body strength, regulation, assertion, or self-interest.
“The Fluctuating Female Vote: Politics, Religion, and the Ovulatory Cycle”: claims of implausibly huge within-person effects, no within-person measurements at all.
I’m pretty sure that neither of these papers would’ve had a chance of being accepted in the APSR or AJPS, the two leading journals in our field. I have no idea how far down you’d have to go to get such a paper published in a political science journal.
Just to be clear: I have no problem with such papers being published in a repository such as Plos-One (as was done by this later-debunked paper by some political scientists). Plos-One is a great place to put some shaky empirical work paired with speculative conclusions. You get it out there, make your data public, and other people can take a look. It’s in Plos-One, so savvy journalists know it’s speculation. Win-win.
If this stuff is being published in the top journal in your field, though, we’re in trouble.
I certainly don’t think political science is perfect, but the field seems to be pretty good at reserving space in the top journals for serious work. I’d say that maybe we need a journal for speculations, but (a) maybe such journals already exists, and (b) there’s always Plos-One, which seems to have the potential for playing a useful role as a sort of super-Arxiv across all disciplines.
The thing that bothers me about these various mini-papers that get so much attention is that they’re hyped beyond all recognition. Check out this official press release from the Association for Psychological Science (which, recall, split off several years ago from the American Psychological Association because they felt the ASA was not focused enough on scientific research):
Of course we can’t criticize the APS from running a press release, given that the paper was accepted in their journal. But the press release itself is a mix of caveman speculation and a misleading description of the research. It’s misleading to speak of “wealthy men” when two of the surveys were of college students). It’s also misleading to make statements about how “individuals reason” based on a few survey questions about issue attitudes; that one is particularly embarrassing coming from psychologists (indeed, the keywords in the press release include “Cognitive Processes” and “Judgment,” even though there’s nothing in the data about cognitive processes or judgment). There’s also the crude and politically naive association of an attitude on redistribution with “self-interest.” Suppose young man comes from a family with low socioeconomic status with conservative economic views (actually, I don’t know how many low-SES students or people with conservative economic views they will get in their sample of University of California students, but that’s another story), then he might well view it to be not in his self-interest to support economic redistribution.
These authors are psychologists, there’s no reason for them to know much about political science, any more than I should be expected to know much about macroeconomics (despite its relevance to some of my research), but this still seems pretty bad to me. At least I’d hope they’d show some humility about the subtleties of politics, in the way that John Jost and Jon Haidt do from their quite different perspectives in their studies of political attitudes.
There’s also a political angle, in that these sorts of studies often seem to be used to support a wife-in-the-kitchen agenda. Blogger Echidne gives a hilariously-horrible example from Fox News:
Erick Erickson, one of Fox’s newest contributors, was troubled by female breadwinners and claimed that people who defend them are “anti-science.” Erickson told viewers:
When you look at biology, look at the natural world, the roles of a male and female in society, and the other animals, the male typically is the dominant role. The female, it’s not antithesis, or it’s not competing, it’s a complimentary role. We as people in a smart society have lost the ability to have complimentary relationships in nuclear families, and it’s tearing us apart.
Remember that the topic is women who earn money for their families. So Erickson seems to be arguing that no female animal goes out to get food ever, that it’s the male lions which feed the pride and so on, and that the female wolves never go out to hunt.
That is all total rubbish. In fact, I can’t think of any mammal where the female stays in the nest or lair with the young and the male goes out and brings all the food home. If that happens, at least among mammals, it is extremely rare. My suspicion is that single mothers are much more common among mammals than that alternative fable. Indeed, chimpanzees seem to have the single mother system.
What these four men are upset about is the fear that the traditional gender roles are breaking down. They like those gender roles because they like to be dominant.
But in most ways the traditional gender roles aren’t even that traditional, because very few people in the olden days could live like the Victorian images of a bourgeois nuclear family. Farm-wives worked, wives of artisans worked and so on.
Contrary to stereotype, in this case it’s the man who is babbling and the feminist who has the common sense.
Ok, this happens in both directions. Not so much in evolutionary psychology, I think, but there is a left-wing equivalent, of sorts, in various cross-national comparisons that get published in journals of public health. The basic idea is that good things tend to go together. There are some countries that are richer, more civilized, have nicer governments that spend more money on social welfare, have happier and more educated citizens, etc.; and other countries that are poor, dangerous, etc etc. As a result it’s not hard to show a correlation between, for example, social security spending and longevity, or whatever.
As with the evolutionary psychology studies, I’m not saying that these claims made in these cross-national correlation studies are wrong, merely that the evidence has problems and the claims can be overstated. And they get published in top journals because they are important. Actually, I’m not so bothered by these studies because the topics are important (more so, in my opinion, than claims about ovulation and voting). Similarly, I’m not so bothered by observational studies of the macroeconomy such as Reinhardt and Rogoff’s (implementation details aside); when a topic is important, you go with what you have, you can’t wait for identification.