One man’s outlier is another man’s high-leverage case

Nate Silver, over at fivethirtyeight, objects to the conventional wisdom that Herman Cain, despite his place in the polls, has little chance of becoming the Republican presidential nominee. That wisdom, consistent in this case with our book, The Party Decides, is that that Cain may have some appeal with some elements of the party, but he is not broadly appealing among political leaders, and so the party leaders will find a way to nominate someone else.

Cain is an outlier, in that his poll performance is high, but his ranking on various non-poll measures, such as support from political insiders, is low. Silver presents a great graph showing the relationship, using a new measure. Since these various measures of candidate strength are highly correlated, Cain is unusual. If you wanted to predict his poll position from his non-poll strength, you’d miss. (Although notice that this poll of political insiders shows more favorable attitudes toward Cain than a lot of other conventional wisdom.)

Silver concludes from this that pundits should stop saying that Cain has no chance. We don’t know what will happen, but we could be in a “new normal,” in which support from others in the party is not as important as getting respondents in polls to say they will vote for you.

I think I look at Silver’s data differently than he does. Cain is an outlier in the relationship between poll standing and insider support. But that’s not the relationship that matters. What matters is the relationship between these factors and winning the nomination. Figuring that out is hard, though, precisely because they are so related to one another. Polls and elite support (and media coverage, and money raised) are what we call multicolinear. In such a case, it’s hard to say if raising more money is more important than getting insider support, because the people who get one also get the other, and then they win.

So you look for high-leverage cases. That is, cases that don’t fit the relationship among your predicting variables. There have been some. Certainly many early poll leaders were busts, although it is getting later. Still, what matters is not whether you can get a plurality in Iowa. What matters is whether the party will support you or oppose you if you do. Similarly, we can learn that money isn’t very helpful without elite support from folks like Pat Robertson and Steve Forbes, who had lots of money but little elite support. The analysis we do in the book suggests that polls are not more important than elite support, so pundits who see Cain as an outlier and go with the non-poll measures are probably doing the right thing. But all of this is hard to measure, and Silver’s new measures may be very helpful.

Cain is the next high-leverage case. If he gets the nomination, especially if he does so without getting any more support from party insiders, then that is evidence against the argument, which we advance, that party leaders generally get their way. But if he doesn’t get it, then that is still more evidence that a narrow focus on polls is less helpful than understanding the preferences of the people who devote their lives to party politics. We won’t know for sure until we see the dependent variable—who wins the nomination.

So I’m waiting for the outcome, but there’s nothing wrong with predicting which way you think it’s going to come out. Our best understanding of how these things work says Cain isn’t the next nominee. I don’t think pundits who say he has essentially no chance are really wrong. They are no doubt responding to the equally loud voices that keep calling Cain the front-runner. The evidence we have so far is that popular support without insider support isn’t good enough. It’s good journalism to communicate that.

4 Responses to One man’s outlier is another man’s high-leverage case

  1. Ken B. October 28, 2011 at 8:07 am #

    Three terms are used here, “little chance” “no chance” and “essentially no chance”

    Nate Silver only objected the the second one.

  2. Steve Smith October 28, 2011 at 10:08 am #

    Hans, can you elaborate a little more? I have not been quite clear about whether the argument is that elite support is a necessary, a sufficient, or a necessary and sufficient condition for nomination. I’m guessing that the prediction is that Cain fails because a necessary condition is not met. Is there more than one candidate who is likely to meet this condition?

  3. Damocles October 30, 2011 at 1:13 am #

    I don’t think simply ‘winning the nomination’ is the dependant variable here. If Cain does well, wins somes states, and makes a strong push for the nomination before being pipped by Mitt Romney, then that would argue against your thesis and in favor of Nate’s. You need a higher resolution metric of performance.

    It seems you’re arguing two different things however. you are saying a non establishment candidate is inviable. Nate is saying that we’ve never seen a candidate like Cain before, so we just don’t know.

  4. Simon October 30, 2011 at 6:04 am #

    This is a problem that could be solved very simply. Whenever a pundit gives a prediction, they have to give the probability of the outcome. After all, what does ‘essentially no chance’ mean. What are the numbers. 1 in 100? 1 in 1,000? 1 in 1,000,000?
    I think what Mr Silver is saying is, if experts had to do this, you very quickly find that they stopped calling so many events 1 in 1,000,000 chances, or at least we could demonstrate which ones are the most idiotic. Put another way, I think they are getting this wrong and given the number of responses to Mr Silver’s post that amount to “1 in infinity chance” I’m pretty sure I’m right about this. This is not good journalism, it isn’t good punditry.
    You say that we should look at the high leverage cases and look for a relationship, I would agree. What model are you using to do this? How are you extrapolating the mild outliers to the more extreme outlier that is Cain? From this post I have no idea what method you are using to determine Cain’s odds so I cannot tell if what you are doing is reasonable or not.
    If you aren’t using of one of the statistical models which allow one to consider the impact of highly correlated predictor variables, can you at least give a number for what you think Cain’s chances are? Given that you seem to have a decent grasp of the data how do you think that number would compare with the swathes of barely competent, mathematically illiterate pundits and journalists in politics who are so ready to run their mouths without looking at the numbers?