If science tells you you can’t predict something, is it no longer science?

by James Fearon on September 2, 2013 · 13 comments

in Science

The NYT ran an op-ed last week by philosophers of Science Alex Rosenberg and Tyler Curtain that said that (1) economics is not a capital-S Science because it has no “record of improvement in predictive success” and (2) therefore the next Fed chair should be someone who understands that economics is not a capital-S Science and instead has “wisdom” and “a feeling for the economy.”  (Interestingly, experience was not mentioned, which you’d think would be more important the more the job is a matter of craft, as R and C say, than of book learning, which is mainly what Ben Bernanke had.  R and C applaud Bernanke for his craft.)

Harvard Econ theorist Eric Maskin wrote a letter in reply and then NYT did a “Room for Debate” exchange between Maskin and other readers.  Maskin made the point that explanation can be valuable and scientific independent of whether it leads to sharp predictions in some particular domain.  But most of the readers the Times printed were not buying this at all — for them, sufficient evidence that Economics is not a capital-S Science is that it can’t predict stock market movements and crashes reliably.  Rosenberg and Curtain also seem to have this in mind as the key example.

But what you could reasonably call a scientific thought experiment is enough to show that you can’t have a Scientific Theory that reliably predicts stock market movements and crashes:  If such a theory existed that predicted a giant market collapse on date T, the collapse wouldn’t happen on date T.  Ditto for forecasting market movements from publicly available data.  In the long run, or “in equilibrium” in the sense of the developed formal models of this kind of thing, market movements and crashes should be random or unpredictable based on public information.  Yes, that’s an “idealization,” and an idealization that R and C obliquely criticize at the end of their op-ed.  But it’s an idealization that provides an important and valid insight into the kind of system an asset market is.

The reply by reader David Berman raises this issue indirectly, but he concludes that it means that Economics can’t be a “proper” capital-S Science (“if only we could [predict market crashes] without the prediction’s entirely changing the behavior of the markets! That’s the other critical difference between economics and meteorology, or physics, or any of the disciplines we properly call scientific”).  I read what R and C are saying the same way, although it’s less spelled out there.  But can that be right?  I would have thought the argument should be that scientific inquiry has clarified the reasons for why a theory that predicts specific market movements or crashes is difficult, or impossible in the long run, in contrast to theories of simple physical systems.  In other words it’s not so much a case of inappropriate hubris of Economics in trying to be like Physics, but of a scientific effort producing a better and deeper understanding of why this sort of system (an asset market, here) is different from a physical system that doesn’t have agents that condition what they do on expectations about what others will do.

Of course, not all markets — or economic, political, or social interactions — have this specific dynamic that asset markets have, so the difficulties in predicting the stock market from public information do not imply that you can’t develop theory that makes reasonably good predictions in other areas (eg., basic supply and demand analysis for prices and quantities).  R and C were making a reasonable point about choosing Fed chairs.  Sure, I’d like someone who is wise and not rigidly attached to some particular mathematical idealization.  But the broader line of argument seems wrong.

{ 13 comments }

Charles Seguin September 2, 2013 at 4:13 pm

Thanks for this fascinating post. Sociologists have been studying the impact that theories can have on the social behavior they describe or explain for a while now.

Sometimes economic theories can also become self-fulfilling prophesies, or “performative” and thus can predict much better than they might otherwise, as when Black-Scholes became a better predictor of option prices simply because traders had themselves adopted to the model to price options.

McKenzie and Millo have a great paper on this dynamic for Black-Scholes in the American Sociological Review (2001).
http://bscw.cs.ncl.ac.uk/pub/bscw.cgi/d61382/MacKenzie,%20D.%20&%20Millo,%20Y.%20Constructing%20a%20Market,%20Performing%20Theory:%20The%20Historical%20Sociology%20of%20a%20Financial%20Derivatives%20Exchange.pdf

Maybe this is somewhat beside the point of your post, but it seems relevant.

Edwin Perello September 2, 2013 at 5:09 pm

I contend that it’s not so much that economics is a science or not but that the field of economics itself is not scientific and that is where the problem lies.

Even if you can convince other economists, and scientists, that economics is scientific, it’s going to be near impossible to convince the public of that.

Perhaps it’s not necessary, but the fact still remains.

When economist after economist allows their priors to get in the way of good, healthy, objective and scientifically reasonable economic information and disseminate false information to the public not in the interest to actually inform the public but to manipulate the public, your level of trust is going down the crapper. And it happens every single day of the year that there’s an economist or someone claiming to speak for economists on television or the newspapers.

And these people are going to keep coming onto tv and the news and will continue spelling out false yet advantageous information to make themselves or someone else richer or to prove correct their non-reflective, severely biased beliefs that are not based on strong data or is contradicted by it.

Perhaps economics is too complex to deem it a physics which take singular truths and try to explain them. Perhaps there are no singular truths in economics just as there are no singular truths in psychology. Economics is a science but don’t argue that it’s a hard science and don’t make the mistake of comparing it to particle physics like this article did.

When an economist tries to predict something, it’s based partially on a priori knowledge and data. When a particle physicist is looking for the Higgs-Boson, they’re looking for that extremely elusive particle that the we know has to exist because the data, all by itself instead of a priori knowledge, says it has to exist.

If economists can predict the existence of something singularly through mathematics, then it comes closer to a hard science. But that’s not really the point, is it? The question is whether economics is scientific at all and is basing that question on its ability to predict things.

Psychology is a science as well. It can’t predict much of anything. That is not its intent. Neither is the intent of physics to predict anything. Its intent is to figure things out, not predict things. It has predictive properties but you can’t call the search for Higgs-Boson a prediction. It was a certainty based on the theory, we just needed to actually make it come out of its hiding place.

Economics, however, is more like psychology in that it’s attempting to explain a dynamic world which has no hard truths like physics does. Whether our physics theories explain those absolute truths (under X circumstances, Y will happen) correctly or not, economics cannot do the same thing.

So the question isn’t whether economics is a science or not, the question is whether it’s a soft science or a hard science. That question is just silly. It’s a science but one that is incredibly corrupted by the fact it has no ability to manage its body of evidence.

What incentive does a physicist have to misconstrue evidence that cannot be replicated? He’d be laughed to the stocks and pelted with (hopefully) high-speed particles by other physicists. How soon can we find evidence of the Higgs-Moron particle from a lying scientist?

There is, however, plenty of incentive for an economist to misconstrue evidence because it cannot be so easily replicated. Someone can check your math and argue that you’re wrong but there will always be some other economist from the same school of economics as the original, devious economist who will back them up. Economists can appreciate “teaching the controversy” because it sows doubt and people can just believe whatever it is they want to believe based on their priors. This allows wealthy interests to manipulate what would otherwise be the scientific consensus and end up winning a big wad of cash.

Conor Hughes September 2, 2013 at 9:23 pm

Broadly I agree with your points, but I think saying “Even if you can convince other economists, and scientists, that economics is scientific, it’s going to be near impossible to convince the public of that.” is a bit off. If there’s any true success economists have been able to make, it’s been convincing the public that they’re a real scientific discipline.

Until ~250 years ago, the idea that there even was such a thing as an economy was a matter of great debate. The same can’t be said of the other “big S” sciences. And while I like to think this has changed since 2008, bankers and economists were incredibly successful at convincing most everyone “Hey, this is far too complicated for us to explain to you laypeople. We have rocket scientists working on this! But don’t worry, we know what we’re doing.” Most other disciplines with similar origins (e.g. psychology) can’t claim this sort of respect from the general public.

Eli September 3, 2013 at 9:34 am

I don’t mean to sound elitist, but I seriously doubt members of the general public can define the word ‘science.’

Ronan Fitzgerald September 3, 2013 at 10:46 am

Sure they could just google it

Kenneth Almquist September 2, 2013 at 8:07 pm

Physics says that you cannot predict certain things, such as when a particular atom will undergo spontaneous radioactive decay. Since physics is generally regarded as a science, the answer to the question posed in the title is no.

As for the question of whether economics is useful to the Federal Reserve, that depends on whether economics can make predictions that are relevant to the Federal Reserve.

David Piper September 4, 2013 at 4:26 pm

Physicists acknowledge that some physical properties are not predictable, but Rosenberg’s point is that big “S” sciences are enjoy increasing predictive capacities over time (see Rosenberg’s contribution to the Oxford Handbook of the Philosophy of Economics). On this score economics is not a big “S” science because there was nothing like agreement among economists about what to do in the wake of financial crisis. There is more math and data than ever – which makes economics feel like a science – but in 2008-9 economists were as divided as they were in the 1920s. I’m afraid Fearon’s point about supply, demand and prices is also weak because it mirrors the exact problems of financial markets: were it possible to accurately predict the prices of goods, they would become the object of speculative markets in which everyone acts according to the same predictions, which would see all “players” exit the market. The ceteris paribus assumptions on which prices are based are exactly what prevent meaningful prediction.

Paul Babbitt September 2, 2013 at 8:49 pm

Didn’t Popper make the point about the the impossibility of prediction in the social sciences over half a century ago?

mbka September 3, 2013 at 2:17 am

I think it is a really misguided definiti0n to pin everything on hard predictability. Firstly, there is such a thing as probabilistic predictability, and economics can certainly predict some matters with a certain degree of confidence, even in macroeconomics. Printing money can and will lead to inflation etc. Microeconomics again is full of laws that are used on a daily basis. If demand and supply curves were totally useless as predictors for price, no business could exist.

Now if one were to ask for specific, numerically exact predictions of prices in times and places, this is the kind of prediction where the complexity of a complete economy precludes a definite answer. But consider this: it is no different in biology, or even in physics. In biology you’d be very hard pressed to predict the exact consequence of a certain rise in CO2 on carbon uptake in the rainforest, or the consequences of the removal of a particular kind of insect on an ecosystem. And I don’t think ecology is stripped of its scientific status because of that inability to predict. But even physics often has trouble to predict real life situations, as opposed to strictly controlled experiments. Physicist Nancy Cartwright wrote a whole book about “How the laws of physics lie”. One of her illustrations of the limitations of physics is this: no atmospheric physicist would be able to predict the exact trajectory of a one hundred dollar bill dropped from some clock tower. And it’s not for lack of predictive power of gravity or atmospheric physics or mechanical flexibility of the paper bill. It’s just a system that is too complex to be accurately predicted more than a short period of time ahead (and one could add that meanwhile a bunch of sociologists would find in trivially easy to predict with high certainty what would happen once that hundred dollars land on a busy sidewalk).

In conclusion, people have a naive view even of what kind of things physics can predict. Prediction is possible in systems that are simple by nature (laboratory conditions in physics or other sciences, the Higgs Boson is a prime example). Long range prediction of complete systems behavior in complex, adaptive systems, is either impossible, even in physical systems, or at best probabilistic.

Wonks Anonymous September 3, 2013 at 11:55 am

Rosenberg did a diavlog with economist David Levine some years back. Levine argued that economics did actually have a track record of improved predictive accuracy, although he also didn’t seem to regard “forecasting” as an important test for the scientificness of economics.
http://bloggingheads.tv/videos/2285

Tracy Lightcap September 3, 2013 at 4:06 pm

The problem here – and, as far as I can see, it’s mainly a problem with some philosophers of science – is that the vision of science in the op-ed is impoverished.

Freeman Dyson came up with a useful distinction some years ago between what he called “Athenian” and “Manchesterian” science. Athenian = time invariant states of affairs, use of conclusive explanations based on experiments, deductive theories generating law-like relationships and predictions. Manchesterian = historical states of affairs, descriptive explanations based on narrative sequences, inductive theories generating probabilistic explanations. (Dyson put this better, btw.) Obviously, many think of Athenian science as the only legitimate model.

But, of course, it isn’t. Many sciences – medicine, geology, meteorology, the social sciences, even astrophysics – are more Manchesterian then not. This doesn’t make them any less sciences; the distinction is based on the states of affairs the science is trying to understand. We can predict that in the Klingon Empire hydrogen and oxygen in the presence of a flame will produce a reaction resulting in water. Explaining how the Imperial High Council works requires another, quite different set of explanations and approaches. Practicing scientists don’t appear to have much trouble realizing this – recall Einstein’s remarks about social science – but the general public and some philosophers of science seem puzzled about it.

Jeff Lazarus September 4, 2013 at 11:58 am

I agree wholeheartedly with Tracy – the op-ed authors’ view of science IS impoverished. Here’s a different perspective on how: their view of prediction is woefully incomplete.

They seem to think that they only predictions that matter are point predictions: unless you can accurately say that X will happen on Y date, you can’t call yourself a scientist. Well, that’s a tough standard. It’s too tough for meteorology (who can predict when and where a hurricane will land?). It’s too tough for medicine (who can predict which patients will respond to a given treatment?) It’s too tough for astronomy (predictions about when a given star going to go nova have standard errors of tens of thousands of years). But somehow the social sciences are held to this impossible standard.

What many observers seem unable to comprehend is that whenever you’re dealing with very complex systems (like the human body, weather patterns, or the macroeconomy) probabilistic predictions are as good as it gets. And economics, like political science and other social sciences, has been amassing knowledge via the accumulation of probabilistic predictions for decades.

Steve Sailer September 4, 2013 at 8:57 pm

People aren’t very interested in predictions about things that are highly predictable.

Comments on this entry are closed.

Previous post:

Next post: