What are the (relatively) settled matters in the social sciences?

by Andrew Gelman on July 23, 2013 · 13 comments

in Science

Nicholas Christakis asks (scroll to the P.P.P.S. at the end of the post):

What are the (relatively) settled matters in the social sciences? Can we social scientists can ever say that “we have pretty much figured this out” (as in the way biologists have figured out certain topics)?

My reply: I dunno. Krugman would say it’s settled that it’s a good idea to expand government hiring during a depression, but others disagree! For an example from psychology, stereotype threat is claimed by some to be very well established, while others have difficulty finding it at all. In various ares of social research, there’s debate about the replicability of all sorts of claimed effects.

Ideally, I think, once something is settled, this can be the staging point for more research. For example, Kahneman and Tversky demonstrated “anchoring and adjustment” and other so-called heuristics and biases. Then later researchers followed up with studies trying to crack open anchoring-and-adjustment, to understand experimentally how it happens and how to alter it.

Overall, I’d say that, if anything, social scientists perhaps don’t spend enough time re-confirming the definitive statements. There’s a real push toward novelty, to the extent that maybe we don’t have enough “gold standards” of well-established social patterns.


Dan Nexon July 23, 2013 at 11:47 am

Yes. We need to publish more replication work. The problem at traditional journals is that the combination of disciplinary incentives, driving attention, and space make this difficult. Specific journals might be good. Or creating a significant space for replication in online supplements. I’ve come to realize, of late, that the institutional barriers to the last are thornier than I would have expected.

albatross July 23, 2013 at 1:51 pm

It might also be useful to ask what datasets or statistics are universally or near-universally accepted by social scientists working in a given area.

For example, people use the results of NCLB-required tests to evaluate schools, but I gather there’s some question about how much fraud there is. But not being an education researcher, I don’t have a good sense of the magnitude of the likely problems–like are we talking about tests that are broadly pretty correct but have occasional blemishes where one school cheats, or are we talking about such massive cheating that the information from the tests is meaningless?

I routinely look at official statistics (CDC mortality statistics, BJS crime statistics) and reputable pollsters’ numbers (Pew Center survey results) to get a clearer view of the world than I can get from newspapers. But this is a place where I’m kind-of at the mercy of the data. I honestly don’t know enough to know whether, say, the difference between black and white homicide rates reported by the BJS is partly an artifact of selective prosecution or different quality of legal representation leading to better plea deals. It would be really nice to know what working criminologists think–not “it’s flawed” (of course it is–so’s everything) but something more like an informed expert’s prior distribution on the errors.

James Bailey July 23, 2013 at 2:04 pm

Not much is considered definitively right, but lots is considered definitively wrong, for example the real bills doctrine in economics.

Tracy Lightcap July 23, 2013 at 3:38 pm

Bingo! That’s exactly it and it is the same in all the sciences. What the work does is separate the wheat from the chaff. Then you argue about the wheat. Endlessly.

I recently saw a TED about physics where the speaker pointed out that what we don’t know is about 95% of the actual subject matter and that the 5% left is constantly contested. That people who don’t do science don’t realize this is a really big problem. They, in general, are not impressed by science, but by engineering. Engineering is the useful end of the enterprise, but the (largely inductive) certainties it involves are misleading to the general public.

Not Tajfel July 23, 2013 at 3:48 pm

party id is pretty darn stable!

Scott July 23, 2013 at 5:01 pm

I’d second partyID (that part of the original American Voter stuff still holds up nicely). Also within political science I’d add Duverger’s law as pretty settled and suggest the idea that politicians are vote-maximizers as settled as well, although I suppose some people debate it. Not exactly ground-breaking these days, but they’re insights and I think they’re representative of growth from foundations in the field.

Other potentials: trade is good (mutually beneficial). talk is cheap. more educated people are (on average) more politically extreme.

Phillip Lipscy July 23, 2013 at 8:42 pm

One issue in the social sciences is the impact of scientific findings on behavior. Once a social scientific issue becomes “settled” and the conclusions well-known, that will influence how people behave. For a social scientific theory to be truly settled, it probably needs to be established empirically using past data, and also be characterized by some kind of self-reinforcing mechanism in regards to future behavior.

So, for example, Duverger’s law is probably self-reinforcing, because once politicians and voters know about it, they have incentives to avoid wasting their candidates/votes, making behavior more closely conform to the theory.

Some other findings are self-defeating. Most “anomalies” in finance fit into this category, like premiums associated with various assets and strategies. Once they are widely known, they ought to be arbitraged away.

I imagine something like the democratic peace falls in the middle. On the one hand, if it becomes common knowledge that democracies do not fight wars with each other, that might lead to greater mutual reassurance and peaceful behavior. On the other hand, it could lead to excessive demands and escalation in crisis situations because you don’t think the other side will resort to war.

Walt July 23, 2013 at 11:17 pm

I thought Mark Kleiman provided a useful list of useful and stable findings – if not exactly the same thing as settled science – in this 2010 post: http://www.samefacts.com/2010/08/health-and-medicine/medicare-health-and-medicine/social-science-and-deference-to-existing-institutions/

“I’m also less unimpressed than Manzi is with how much non-obvious stuff about humans living together the social sciences have already taught us. That supply and demand will, without regulation, come into equilibrium at some price was a dazzling and radical social-scientific claim when Adam Smith and his friends suggested it. So too for Ricardo’s analysis of comparative advantage, which, while it doesn’t fully support the free-trade religion that has grown up around it, at least creates a reasonable presumption that trade is welfare-increasing.

The superiority of reward to punishment in changing behavior; the importance of cognitive-dissonance and mean-regression effects in (mis)shaping individual and social judgments; the intractable problem of public-goods contributions; the importance of social capital; the problems created by asymmetric information and the signaling processes it supports; the crucial importance of focal points; the distinction between positive-feedback and negative-feedback processes; the distinction between zero-sum and variable-sum games; the pervasiveness of imperfect rationality in the treatment of risk and of time-value, and the consequent possibility that people will, indeed, damage themselves voluntarily: none of these was obvious when proposed, and all of them are now, I claim, sufficiently well-established to allow us to make policy choices based on them, with some confidence about likely results. (So, for that matter, is the Keynesian analysis of insufficient demand and what to do about it.)”

Again, not exactly the same thing, as these are broader and in some cases still quite debatable. But relevant.

albatross July 24, 2013 at 1:30 pm

How much of what social sciences can teach you comes down to conclusions, and how much comes down to useful tools to use when thinking about social phenomena?

My sense (coming from an undergrad economics background that I don’t use professionally, but that shapes my understanding of the world) is that social, economic, and political systems are too complicated to give a lot of always-true rules, and so someone looking for a simple statement of fact like “free trade always benefits both sides” or “incumbents win and lose elections based on the economy” is going to be disappointed. But the tools developed to understand those social and economic and political systems, those can be applied and can yield some insights in many different situations.

Walt July 24, 2013 at 1:58 pm

Agree. Thinking probabilistically, much of the success in social science has been about theories that very often prove useful, more than any real certainty.

JPL July 24, 2013 at 6:45 am

I’m not even a member of any of the disciplines represented by the Monkey Cage, but I consult it often when I want to understand how the current political scene is being interpreted by those whose calling it is to understand it. And I don’t know what Christakis is complaining about, because just taking the top few papers off my pile of “papers to read”, I find, e.g., “The neuroscience of in-group bias” by P. Molenberghs (Neuroscience and Biobehavioral Reviews) (sorry, I don’t have time to provide the links); “Fear as a disposition and an emotional state: a genetic and environmental approach to out-group political preferences” by P. Hatemi, et al. (AJPS); “Prejudice rivals partisanship and ideology when explaining the 2008 presidential vote across the states” by B. Highton (PS); “Old times there are not forgotten: race and partisan realignment in the contemporary south” by N. Valentino and D. Sears (AJPS); “Does biology justify ideology? The politics of genetic attribution” by E. Suhay and T. Jayaratne (Public Opinion Quarterly); “Disgorging the fruits of historical wrongdoing” by R. Goodin (APSR); “Beyond public and private: toward a political theory of the corporation” by D. Cieply (APSR); Why are there so many engineers among Islamic radicals?” by D. Gambetta and S. Hertog (Arch Europ social). Seems pretty relevant to me. To me as a concerned citizen, Christakis’s problems are milquetoast. Here’s a problem I would like to see the political scientists solve: How can we the people limit the power of plutocrats in political systems? Admittedly that’s a practical question, so here’s a theoretical counterpart: Why do all the political systems we’ve tried so far (in practice) tend toward plutocracy? What are the mechanisms? (I’m placing a good part of the blame for the racism, paranoia and general insanity of the current tea- party Republicans on the plutocrats.) Can we please have a review article on this? As an outsider, I do think, however, that the social scientists could perhaps do a better job of making the general public aware of the results of all this really valuable research, insights I’ve found to be very enlightening, but which I never seem to see evidence of in the public debates. I didn’t like Christakis’s article. I didn’t think it was to the point. Disciplines are not at their core administrative departments, but research communities, and research communities are defined by the critical standards by which work in those communities is evaluated as significant or not. The traditional names indicate unified research communities with long histories of theoretical evolution and validation of results.

JG July 25, 2013 at 12:57 am


Your questions about the democracy and plutocracy are important. Some conjectures have been offered by political theorists from Aristotle to Manin, but there is really not enough data for political scientists to offer any real concrete conclusions.

Personally, as a political scientist and former political professional, when money becomes “speech” any country is headed down that road. More appropriatly (or academically), perhaps, the question of whether we have ever witnessed democracy might also be an important question. Take the US for example, who drafted and debated the Constitution? Your average Joe or elites? How democratic can any country be if the rules are written by elites?

This is all food for thought more than answers to your questions. There is probably not enough data out there to convincingly test the hypothesis either.

Paul Gronke July 25, 2013 at 9:44 am

Andrew, I just don’t accept the notion that we should identify a set of “settled” Christakis items analogous to biology.

We aren’t studying the physical movements of object through space, genetic mutation, chemical reactions, etc.

We are studying human beings and society. At the very minimum, humans respond and adapt to each other and to the rules of the game, including of course changing the rules!

The same gravitational experiment conducted by Galileo could be conducted today. There is no concern that the heavier ball somehow changed the law of gravity.

Other readers have already listed some settled questions. But I contend that the whole question fundamentally mischaracterizes the social sciences, holding them to a standard that isn’t accepted by many economists and political scientists, and I suspect almost no anthropologists, historians, sociologists, or psychologists.

Comments on this entry are closed.

Previous post:

Next post: