A colleague told me that he met someone recently and, in the course of pleasantries, she asked what he did. He told her that he taught political science at GW. Her reaction to the topic of political science was something like, “Oh, what’s that guy, you, now, at 127.com or something like that?” “You mean 538.com,” he said. “Yeah, that’s it!” she said.
So what should political scientists think about 538? Here are my thoughts. First, let’s just acknowledge who’s on the masthead (inter alia). Andy, who is at least a part-time political scientist. Tom Schaller, who is a political scientist. And Nate Silver, the son of a political scientist. That much is good.
What about its content? Silver made his name doing election forecasting. On one level, I’m happy to have election forecasters out there. It’s better to have their statistical models than the ruminative burps of some cable news pundit. On another level, I don’t really anticipate any huge advances in election forecasting. Political scientists have been doing this for years—see, e.g., this 1983 book by Steven Rosenstone or this 1992 book by Michael Lewis-Beck and Tom Rice. All of the various models are pretty well-developed and they typically all point to the same winner, as far as presidential elections are concerned. I can’t tell that 538 adds a lot in this regard. Their 2008 Electoral College forecast was actually less accurate than Sam Wang’s, electoral-vote.com’s, or mine for that matter. (I rounded up some predictions here. All of the above were more accurate than Karl Rove, however!) 538’s model didn’t do as well as some others in the British election either. I’m not picking on 538 at all. The simple fact is that there will always be things that forecasting models do not and cannot account for. So whichever model “wins” will always be due partly to luck. I tend to look at a variety of such models, rather than assuming that any one has the goods.
This gets to a broader point: election forecasting is not, obviously, political science. And too much focus on election forecasting per se impoverishes our understanding of politics, because it simply draws people’s attention again and again to the horse race. More generally, we don’t need to predict the future to say something useful. Explaining the past is equally important.
In other ways, I think 538’s content is very useful and, to be fair, it involves much more than election forecasting. They have posts on elections abroad, public policy, the economy, and other current events. I also appreciate Silver’s bird-dogging pollsters (e.g., here). Mark Blumenthal of pollster.com had already begun doing this before 538 came along, and I think both of them are doing a tremendous public service. With the glut of polls on the market, and the insatiable appetite for them, someone has to be minding the store.
Ultimately, I think 538 helps political science by showing people, and especially journalists, that you can use quantitative evidence to understand politics. That’s one reason I’m glad for its new relationship with the New York Times. I don’t always agree with all of Nate Silver’s analyses—see, e.g., here or here—but they are quite an improvement over the views of people who think that a conversation with three Iowans absolves them of paying attention to systematic evidence or academic research.
The way I look at 538 is that it’s pushing analysis of politics closer to Moneyball, and further away from a world in which pundits simply make stuff up. The more that happens, the more doors will be opened to what political science can offer.
{ 9 comments }
I think 538.com counts as political science. Surely, part of the study of democratic states is the study of the dynamics whereby their governments are formed.
Models of that have informational value. They also have strategic value for political parties, and all those seeking to influence the political process.
John: You aren’t suggesting that political science=quantitative analysis of politics, are you? At least, not exclusively, right?
My view of 538 is that while they focus quite a bit on quantitative issues the greater service and insight (and this goes to Moneyball as well) is that they show the advantages to be gained by grounding any analysis or claim in a robust, sophisticated methodology. The greatest service is the “lifting of the hood” for claims and models and showing readers how to dissect arguments and reveal the flaws in the analysis that generate the claims which receive the greatest play in the press.
How to think methodologically, if you will–whether its quantitative, qualitative, game-theoretic, etc., to me that is what 538 does and conveys best.
“More generally, we don’t need to predict the future to say something useful. Explaining the past is equally important.”
Except that science’s best test of our understanding is prediction – explanation without prediction is not considered sound science in many fields (ex. string theory in physics).
Bill: Oh no, I certainly didn’t mean that. I should probably have said that 538 helps open the door for certain (mainly quantitative) kinds of political science in particular. And I think you state it well: it’s very useful to have an underlying methodology (although I think 538 could “lift the hood” on its own models a little more).
AySz88: That’s a fair point. But I think that for a lot of questions in social science, prediction simply isn’t an appropriate (or, at least, is an overly ambitious) standard.
Given the basically foundational status of evolutionary theory nowadays it hardly seems controversial to suggest that explanation without prediction is perfectly legitimate scientific practice, regardless of what some physicists or overzealous Popper fanatics would have you believe.
For obvious reasons, I do indeed think 538.com is political science. Yes, prediction is what made 538 famous, but we do all sorts of other things there too.
To put it another way: As you say, Nate’s predictions of the election were no better than many others, But then, why is 538 so popular? Partly because Nate (and the rest of us) don’t just mechanically offer predictions. We place the predictions in context and do a lot of analysis too.
@Mike: You seemed to be implying evolutionary theory doesn’t make predictions. That is simply not true.
To provide just one area in which this happens, it makes predictions about what we’re going to find in the genes of various related species constantly. For example, it predicted we would find the GULO pseudogene in humans and other primates (a prediction later verified).
I’ve written a bit about this in other venues and communicated a bit privately, but I am a skeptic.
First, it is not “political science” not because it lacks science, but because it lacks politics, at times embarrassingly so. Nate S. is a great success, and more power to him, but he has gone into areas both in politics (see his UK election forecasts) and other areas where his background knowledge is very shallow and it shows. The problem is that he is such a brand name now that journalists, never particularly good about understanding statistics, presume a kind of precision to his claims just because there are numbers attached.
To respond to Andrew, there is context and there is context. I seldom see any theoretical, historical, or institutional context at 538. It is most often high quality but quickie current events driven statistical analyses with commentary. Perhaps I am being overly harsh but I just reviewed the last few weeks of postings, and they are dominated by the short, quickie type. Yours and Tom’s are longer and more informed, but let’s face it, the brand name here is Silver.
I would hope “political science: requires a grounding in deep contextual and historical understandings, and/or behavioral and institutional theories of politics, not just statistics.
When I go on TV I may be playing a political scientist, and what I do may be good for the profession (although there is disagreement on this point) but I don’t kid myself that I am “doing” political science.
Fivethirtyeight.com is fine for the discipline if it pushes journalists, pundits, and others to be more precise and careful in their analyses, and if it pushes some of our rising stars (and I include Andrew in this list) into the public eye.
Fivethirtyeight.com is not helpful, though, to the degree it promotes the notion that a quick and dirty statistical approach is the be all and end all in politics and in policy.
By the way, to be fair, I am particularly crotchety today not because of 538 but because of this Esquire post:
http://www.esquire.com/features/data/nate-silver-car-culture-stats-0609
which I have to say I think is a very, very superficial analysis of the reasons why car usage may (or may not) be down (look particularly at the claim that relative housing price changes are explained by preferences for cars). Yet this article has been picked up by Richard Florida and others to promote as a strong argument in favor of a particular kind of development / policy path.
In Silver’s defense, he is not so conclusive in the article, but that does not matter, now the argument is out in the public sphere.
Comments on this entry are closed.