Do social scientists and others with little mastery of mathematics find research findings more persuasive when you just add a little math? Yes, suggests this article by Kimmo Eriksson in a recent issue of Judgment and Decision Making. Eriksson gave 200 participants abstracts of two published papers. Half of these abstracts were randomly enriched with a sentence and equation from an entirely unrelated paper in mathematics (“A mathematical model (TPP=T0−fT0df2−fTPdf) is developed to describe sequential effects.”) The respondents were then asked to judge the quality of the research.
The bottom line of the findings is that those with degrees in math and the sciences were not more impressed by the abstract with the nonsense sentence but those with degrees in the humanities and social sciences and (disturbingly) the medical sciences are.
One can interpret this finding as stressing the need for more math training in the social sciences. Or one could emphasize that mathematically oriented articles have an undue advantage in the peer review process. These conclusions are not mutually exclusive. More math training could lead to less deference to pointless math. Unfortunately, the experiment does not allow us to differentiate between the humanities and various social sciences so we can’t quite be sure who is being fooled here (the mathematically minded economists or the historians?). I would like to see this replicated with a more homogeneous group of scholars evaluating scholarship in their area of expertise.
ps. My description of the participants as “scholars” is misleading. As pointed out in the comments, the participants were recruited via Amazon Turk and mostly have master’s degrees. An interesting study but at best a pilot study for drawing deeper conclusions about academia (as per the last sentences of my post).