2012 isn’t 2008, but an Estimate Two Months Out isn’t an Election Either

Simon Jackman posted the following graphic on his blog, which earned a rave review from Lynn Vavrek on Twitter and a lengthy discussion by Chris Cillizza at the Washington Post.

I admit the graphic is a great – sums up a lot of information incredibly concisely to present a quite coherent story – but I wonder if we aren’t comparing apples to oranges a bit here. Don’t we really want to know what the “estimates” that Jackman has for 2012 looked like in these states at this point in time in 2008? For example, I don’t think an estimate of the type Jackman is calculating would have predicted in August 2008 that Obama wins Indiana either. Maybe Obama does a little better than his estimates in some states and a little worse in others? Or maybe he tends to catch fire after the convention? (arguably, an N of 1 in this instance is not that useful). Or maybe, as Wleizen and Erikson have argued, polls that are closer to the election are better predictors of the outcomes than those that are taken earlier. Either way, it would be interesting to see what the estimates in August 2008 would look like compared to the estimates in August 2012.

[Of course, September and October of 2008 were far from ordinary months, but who’s to say what is going to happen in September and October of 2012…?]

11 Responses to 2012 isn’t 2008, but an Estimate Two Months Out isn’t an Election Either

  1. Simon Jackman August 30, 2012 at 6:11 pm #

    Agreed.

    I really only put the graph out there in a “face-value” way. No explicit claim at forecasting (at least not yet). And some of the CIs on the state-level estimates are pretty wide.

    One thing I think would improve a lot of election punditry and poll gazing would be to put “where we are” up against the last election’s results. That I did. Putting “where we are” up against where we were at the same point in the previous cycle would be even better.

    You guys are a tough crowd… but that scrutiny makes our work better in the long run.

    • Joshua Tucker August 30, 2012 at 6:17 pm #

      Simon: would it be that hard to run the analogous August 2008 numbers? I wonder if Nate Silver has that kind of data easily accessible. Might be a good reminder about inherent uncertainty in this up to the minute state-by-state forecasting. I think the story is that Nate got 49/50 states right (if I recall, Indiana was the only one he got wrong), but that was with the model run through November – I bet the August predictions do not do that well.

      And I did really like the graph, especially the way you increased the spacing around 50% and changed the colors of the lines (although I guess now I’m getting into Andy’s territory here!).

      • Hamiltonian August 31, 2012 at 1:05 am #

        This is kind of missing the point. The lines aren’t (almost) all drifting down compared to 2008 because of noisier data owing to measuring earlier in the cycle this time around.

    • J August 31, 2012 at 9:19 am #

      Why not show the CI’s then? That would be more informative.

  2. Andrew Gelman August 30, 2012 at 7:37 pm #

    A parallel coordinate plot with only 2 time points??? That’s just terrible. A scatterplot would be much much better here (and one could also do a distorted scale as Josh requests).

    Simon can do graphs how he wants, but at the Monkey Cage we should have higher standards!!

    • Simon Jackman August 30, 2012 at 8:21 pm #

      Thanks, Andy; just super… Just in case you were being serious…

      99 times out of 100 I’ll go with a scatterplot for (x,y) data because regression or loess and/or 45 degree lines etc are natural things to overlay.

      But this struck me as nice way to avoid over-printing and lots of manual tinkering with the placement of the state labels. This isn’t something I know much about, but I found it easier to write a “smart” label spacer that works in one dimension than in two. And frankly, I think this presentation is more “media friendly” than a scatterplot, both in terms of ease of interpretation, layout, etc.

      Finally, the parallel co-ordinate plot has some similarities with “pendulums” that are often used in visualizing election results, used in places like the UK and Australia where uniform swing (aka “the pendulum”) is a decent approximation to election outcomes; and – as many of us have pointed out elsewhere – uniform swing has been pretty good summary of state-level outcomes in U.S. presidential elections, at least for the last couple of cycles — indeed, putting the 04 results ahead of the 08 results was something I thought about doing too.

      • Andrew Gelman August 30, 2012 at 9:00 pm #

        Simon:

        My point was serious by my overkill was for humor. In all seriousness, I prefer the scatterplot using 2-letter state abbreviations; I’ve done these all the time and they work well. However, I don’t dismiss the popularity of your graph. This indeed suggests that you’re sending your message clearly, maybe more clearly than would be done by a scatterplot in this case. I respect that people have found your graph to be useful.

        Uniform swing is very clear on a scatterplot too, though! It’s just a 45-degree line. One can also plot change (2012 minus 2008) vs 2008, again using two-letter state abbreviations. I think this would work. Lots of ways to go.

        • Stephan Hokanson August 31, 2012 at 8:59 am #

          For nitwits like me, one line is better than any number of dots.
          I think I’m in love with this chart.

        • Erik Voeten August 31, 2012 at 10:22 am #

          A scatter plot may be better to depict the uniform swing but the line plot makes the details much more visible. The downside, as Simon mentions, is that we may be overinterpreting the details due to wide confidence intervals. Still, I really like this plot.

        • Hamiltonian August 31, 2012 at 12:20 pm #

          I’d bet that many readers can more easily glean the overall downward drift in most states, and that Obama’s still looking “good enough” in enough states, from this graph than from a scatterplot. How many words would you have to write along with that scatterplot, to help people interpret the all-important 45 degree line? Simon’s chart doesn’t need any and it’s a really nice summary.

  3. Larry Bartels August 31, 2012 at 10:27 pm #

    I’m with Simon on the structure of the graph. However, since we have an Electoral College, it would be nice to vary the width of the lines to reflect the size of states, and/or run cumulative proportions of Electoral Votes up one side and down the other.