This is a guest post by my colleague Brandon Bartels.
As the issue of forecasting has been on the minds of political scientists and journalists as of late (examples 1 2 3 4), I thought I would weigh in on this issue as it pertains to predictions of the Supreme Court’s ruling on the Affordable Care Act (ACA). My goal here is to provide a means of evaluating various forecasts of this landmark ruling. I focus on 11 predictions—included at the end of this post—from political scientists, law professors, and journalists made in print (or broadcast). These 11 predictions, I believe, provide a good representation of the forecasts that were proffered more generally.
Just to review, in National Federation of Independent Business v. Sebelius, the federal government had argued that the individual mandate was constitutionally permissible under three clauses of the U.S. Constitution: (1) the commerce clause, (2) the necessary and proper (N&P) clause, and (3) the tax clause. By a vote of 5 (Roberts, Scalia, Kennedy, Thomas, Alito) to 4 (Ginsburg, Breyer, Sotomayor, Kagan), the Court ruled that the mandate was unconstitutional under the commerce clause and N&P clause. But by a vote of 5 (Roberts, Ginsburg, Breyer, Sotomayor, Kagan) to 4 (Scalia, Kennedy, Thomas, Alito), the Court ruled that the penalty assessed to individuals for not purchasing health insurance amounts to a tax, which is constitutionally permissible under the tax clause, and the ACA was upheld (with exception of the narrow interpretation on the Medicaid issue). The four dissenters (Scalia, Kennedy, Thomas, Alito) wanted to strike down the entire ACA as unconstitutional.
To evaluate various forecasts, we should assess not only predictions for the overall case disposition (strike or uphold), but the predictions for the vote splits on each legal provision and the reasoning individuals gave for their predictions. Below the fold are summaries of forecasts by three political scientists, four law professors, and four journalists categorized by these important components (as well as on the severability issue). On the overall case disposition that the Court would uphold the ACA, among political scientists, Bailey and Maltzman were correct, while Segal and Evans were incorrect. Three of the four legal scholars examined here correctly predicted that the ACA, and particularly the mandate, would be upheld. Note that Kerr (in 2010) and Adler registered their predictions a full 2 years before the Court’s oral arguments on the case. Early on, many legal scholars correctly believed that the Court would not overturn the ACA. Among the journalists analyzed here, three of the four were correct, with Toobin famously reversing course—thinking it would be struck down—in an interview on CNN after oral arguments.
None of these forecasts correctly predicted that the mandate would be ruled unconstitutional under the commerce and N&P clauses yet constitutionally valid under the tax clause. Schultz was close on these particulars when he predicted—albeit with little certainty—that the Court would “more likely [vote] 5-4 to strike down the individual mandate on commerce clause grounds (maybe upheld on taxing authority)” (emphasis added). Moreover, nobody predicted that Chief Justice Roberts would be the lone conservative justice to join the liberals to uphold the mandate, let alone on tax clause grounds instead of commerce or N&P clause grounds. While Bailey and Maltzman (who registered their forecast back in November 2011) correctly predicted the overall case disposition, their predictions on the vote splits and the reasoning were less than accurate (a point to which they concede to an extent). They incorrectly implied that the Court would find the mandate constitutionally valid under the commerce clause, arguing that Wickard v. Filburn (1942) and Gonzales v. Raich (2005) were “[the] most clearly relevant [precedents].” These precedents provided a legal justification for upholding the mandate under the commerce clause, and Bailey and Maltzman predicted that Roberts and Kennedy and maybe even Alito would vote to uphold the ACA on commerce clause grounds. All three of these justices, of course, voted that the mandate was unconstitutional under the commerce clause, and Kennedy and Alito voted to invalidate the entire ACA. The rationale for Bailey and Maltzman’s prediction is deference to some of the key legal precedents declaring Congress has broad regulatory powers under the commerce clause. Bailey and Maltzman conclude by saying, “Policy motivations won’t be irrelevant, but score this one for law.” After the decision, they wrote a follow-up post titled, “The Law Matters.” However, the type of legal constraint that Bailey and Maltzman predicted would occur—deference to key commerce clause legal precedents—actually did not occur, particularly among Roberts and Kennedy (and Alito, for whom Bailey and Maltzman had some doubts about voting to uphold).
Contrast this reasoning with that of both Segal and Evans, who predicted that the Court would strike down the mandate under the commerce clause along ideological lines (5-4, just as the vote actually occurred on the commerce clause issue). So while Segal and Evans both incorrectly predicted the ACA would be struck down, their predictions on the constitutionality of the mandate under the commerce clause were exactly right, while Bailey and Maltzman’s prediction on this specific issue was incorrect. Moreover, Evans’ prediction underscores the importance of oral arguments, a topic political scientists have analyzed. Some who thought the Court would uphold the ACA before oral arguments changed their tune after oral arguments (see Toobin), given the tough questioning of the Solicitor General by some of the conservative justices (including Kennedy) who some people thought would vote to uphold.
The broader point is that there can still be value in incorrect forecasts and flaws in correct forecasts, particularly in a case as complicated as this one. While Segal and Evans were wrong on the overall case outcome, their predictions on the reasoning contained considerable accuracy and therefore contain great value. The ultimate value of Segal’s forecast is his prediction that the case would be decided largely along ideological lines and that several of the justices would not be constrained by certain commerce clause precedents or judicial restraint and concerns over deference to the political branches. The only vote Segal missed, albeit the pivotal one, was Roberts’ vote on the tax clause. On that one vote on that one issue, Segal overemphasized the role of ideology, but for every other vote on the other issues, he was correct. Bailey and Maltzman overemphasized the influence of legal precedent and underestimated the role ideological preferences would play, particularly among Kennedy on all aspects of the ruling and Roberts on the commerce clause issue.
Many of the journalists and legal scholars, including Adler, Tribe, Sacks, and Kerr (2010), emphasized judicial restraint and predicted that, above and beyond issues related to commerce clause precedents, the Court would not strike down a law of this political magnitude, especially the president’s signature domestic achievement. That part of the reasoning underlying the upholding of the ACA arguably characterized Roberts’ vote (I return to this issue below). Furthermore, Sacks appears to be correct, to an extent, when he argues that the “battle this time is likely to be an intra-conservative conflict between the economic libertarianism underlying the mandate’s challenge and the traditional principles of judicial restraint that have defined right-wing jurisprudence for more than a half-century…. [I]nvalidating a sitting president’s signature legislative victory on commerce clause grounds is freighted with deeply unpleasant institutional memories both for the court and the conservative legal movement.”
While the commerce clause part was not quite right, Sacks anticipated the tension between Roberts and his four conservative colleagues. It is worth noting that Roberts had already demonstrated his deference to federal power, particularly in U.S. v. Comstock, which recognized broad congressional power under the N&P clause (see Kerr’s analysis of this topic). Kerr (in 2010) and Adler, while correct that the Court would uphold the ACA, seem to have overemphasized the extent to which several of the justices would be constrained by issues of judicial restraint and deference to Congress and the president on significant domestic laws. While Greenhouse correctly predicted the outcome, she overemphasized the impact of commerce clause precedent (even more than Bailey and Maltzman), having predicted that the Court would uphold on commerce clause grounds “by a wide margin.” Kerr’s revised prediction in 2011 also overemphasized the impact of commerce clause precedent to about the same extent as Bailey and Maltzman.
Despite some of these errors, most of these forecasts combined prediction with explanation about the rationale underlying the prediction, particularly tying the prediction to theory and the mechanisms underlying the behavior. Even those who incorrectly predicted the outcome still added value. As this exercise has hopefully shown, there is great value in assessing and comparing political forecasts—ones that were both correct and incorrect—after the fact.
Evaluating these forecasts also helps us think about what were the true motivations underlying the justices’ votes—particularly Roberts’. My own opinion is that Roberts got what he wanted: a decision that exhibits judicial restraint (he does appear to favor robust federal power, as evinced in the Comstock decision as well), interprets the commerce clause as not permitting the individual mandate (throwing a bone to the conservative legal movement), and holds that the penalty for not complying with the mandate amounts to a “tax,” which could lay the groundwork for later repeal of the ACA.
Another take on Roberts’ motivation comes from Steven Teles at SCOTUSreport. In the Court’s most high profile cases involving potential clashes with Congress and the president, Teles argues the following:
[A]s an institutional matter justices have some sense of how far they can legitimately push on the political branches, and they make that judgment and then figure out a legal rationale to back themselves up….. Roberts, no doubt influenced by his position as Chief Justice, made the call that he could pull at the seam of the law pretty hard but couldn’t unravel it completely….There is an element in Supreme Court decision-making that can be explained by statesmanship rather than jurisprudence…. On no really important aspect of jurisprudence did Roberts actually break from his conservative brethren, but he did make a different political judgment than they did—not on what the Court could get away with, but what was really appropriate for it to do on a matter of such great policy significance.
Teles’s argument suggests a sort of post-hoc rationalization process of decision making akin to the attitudinal model (Segal and Spaeth) but tweaked to fit broader separation of powers issues. That is, stake your position on what is the best judgment given the situation, then cloak that judgment around a legal rationale that makes enough sense to make it look as if one was guided in a bottom-up, law-driven decision process. From this point of view, maybe Roberts, who felt it was most appropriate to uphold the ACA, really did feel that the mandate was constitutional under the commerce and N&P clauses, but he felt he had to go with a less sweeping legal justification to uphold it—in this case, the tax clause—than either of those clauses in order to appease conservative audiences. One thing we do know for sure: just as the “switch in time” still evokes a great sense of scholarly intrigue about the true motivations of another Justice Roberts—Owen Roberts—some 75 years later, scholars and Court observers will be analyzing Chief Justice John Roberts’ vote on the ACA case for years to come.