Would Paul Ryan’s economic plan really send unemployment down to 2.8%?

by Andrew Gelman on April 6, 2011 · 20 comments

in Methodology,Political Economy

As I never tire of saying, I know nothing about macroeconomics, so I’m asking you, the readers, for help.

Paul Krugman posted this graph (update with more detail here) showing the unemployment forecast from the economic projection that the Republican budget proposal is based on:

heritage-ryan2.jpg

This indeed seems like a stretch, so I followed the link to the report prepared for the Heritage Foundation by William Beach, Karen Campbell, John Ligon, and Guinevere Nell. (Regular readers will know that I always like to refer to reports by their authors rather than their publication or commissioning agency.)

I searched on “unemployment” and found this:

A simulation of the House Budget Resolution using the U.S. Macroeconomic Model from IHS/Global Insight produced the following results for the period 2012 through 2021 . . . The unemployment rate shrank by an annual average of 2.1 percentage points over the forecast period.

This confused me at first—the unemployment rate certainly can’t shrink by an an annual average of 2.1 percentage points for the next 10 years, or it would go negative—but then I realized that what they meant was that they forecast that the unemployment rate, if Ryan’s plan is implemented, would be on average 2.1 percentage points lower than what would happen if his plan were not implemented.

Anyway, the thing I’m wondering is how the simulation worked internally: what was it in the simulation that got the forecasted unemployment rate down to an implausible (according to Krugman) 2.8%. In statistics—even in Bayesian statistics, nowadays!—we’re trained to think that if a model gives an implausible prediction, we should question the model (and also, of course question our preconceptions that the prediction is implausible). Here’s what Beach et al. write about their simulation:

Congressman Paul Ryan . . . specifically asked the CDA to perform conventional and dynamic budget analysis, or analysis that is based on largely ?static? budget models and on economic models with dynamic economic properties. . . .

Center analysts primarily employed the CDA Individual Income Tax Model for its analysis of the effects of tax law changes on a representative sample of taxpayers based on IRS Statistics of Income (SOI) taxpayer microdata.

OK, so far so good. Now let’s see what they say about employment:

The tax and program changes behind the Budget Resolution produce much stronger economic performance when compared to the rate and level of economic activity in the baseline.6 Lower taxes stimulate greater investment, which expands the size of business activity. This expansion fuels a demand for more labor, which enters a labor market that contains workers who themselves face lower taxes. Consequently, significantly higher employment ensues.

OK, and what’s the model that led to these forecasts? I read on, into the description of the dynamic simulation:

Labor Participation Rates. Taxes on labor affect labor-market incentives. Aggregate labor elasticity is a measure of the response of aggregate hours to changes in the after-tax wage rate. These are larger than estimated micro-labor elasticities because they involve not only the intensive margin (more or fewer hours), but also, and even more so, the extensive margin (expanding the labor force). The change in the labor supply variables were adjusted by the macro-labor elasticity of two, which is a middle estimate of the ranges. The adjustment to the add factors allowed the variable to continue to be affected both positively and negatively by other indirect effects. In the final stage of the simulations the add factors were endogenously recalculated in order to take account of the new estimates of the average tax rates mentioned above.

Damn! I didn’t catch that. They do write, however, that “Further details of the simulation are available upon request.” So maybe an interested reader can go through and learn more. It would be good to know where inside the model is the parameter that causes the forecast unemployment rate to go down below 3%.

Thinking about uncertainty

I don’t know enough about macroeconomics (hey, I said it again!) to have a sense of the plausibility of a forecast of 2.8% unemployment. But I think I do have something to add to this discussion, as a statistician:

Any forecast has uncertainty. And, except in rare cases, we’d expect a point forecast to be near the center of that uncertainty. Could 2.8% plausibly be near the center of the uncertainty about an unemployment forecast, ten years from now? I don’t think so. I understand that any mechanistic forecast is based on assumptions, so I’ll set aside the possibility of unexpected events that could hurt the economy (hurricanes, earthquakes, oil crises, dramatic changes in policy enacted in future years by Democrats or Republicans). Instead just suppose everything happens as plans and there are no bumps in the road. Still, there will be some uncertainty now about the unemployment rate in 2021.

Suppose that, on the high end, the unemployment rate in 2021 might be 7%. That’s not a super-high number—some economists argue that the natural rate of unemployment is around 6% in the United States (see, for example, page 4 of these slides which I found in a quick Google search). Considering that the economy might be in a dip in 2021—who knows?—7% seems like a reasonable, even a conservative, upper bound.

OK, if 7% is the upper bound and 2.8% is the point estimate, then the lower bound is . . . hmm, if (7+x)/2 = 2.8, then x = -1.4! A 2.8% point estimate is the center of a forecast interval that goes from -1.4% to 7%. That can’t be right.

OK, sure, the forecast distribution might be skewed. Still, the basic point remains: if the point forecast is 2.8%, you’ll have to have a lot of your uncertainty below 2.8%—and that is not plausible. 2.8% is at the very low end of what anyone thinks might happen, even in a strong economy. In fact, if the unemployment rate ever gets anywhere near 2.8%, I think we can expect a big push to slow the economy down and get inflation under control. Given that unemployment might well be 6% or more in 2021, I don’t see how 2.8% can ever come out as anyone’s point forecast.

Internal (probabilistic) vs. external (statistical) forecasts

In statistics we talk about two methods of forecasting. An internal forecast is based on a logical model that starts with assumptions and progresses forward to conclusions. To put it in the language of applied statistics: you take x, and you take assumptions about theta, and you take a model g(x,theta) and use it to forecast y. You don’t need any data y at all to make this forecast! You might use past y’s to fit the model and estimate the thetas and test g, but you don’t have to.

In contrast, an external forecast uses past values of x and y to forecast future y. Pure statistics, no substantive knowledge. That’s too bad, put the plus side is that it’s grounded in data.

A famous example is the space shuttle crash in 1986. Internal models predicted a very low probability of failure (of course! otherwise they wouldn’t have sent that teacher along on the mission). Simple external models said that in about 100 previous launches, 2 had failed, yielding a simple estimate of 2%.

We have argued, in the context of election forecasting, that the best approach is to combine internal and external approaches.

Based on the plausibility analysis above, the Beach et al. forecast seems to me to be purely internal. It’s great that they’re using real economic knowledge, but as a statistician I can see what happens whey your forecast is not grounded in the data. Short-term, I suggest they calibrate their forecasts by applying them to old data to forecast the past (this is the usual approach). Long-term, I suggest they study the problems with their forecasts and use these flaws to improve their model.

When a model makes bad predictions, that’s an opportunity to do better.

P.S. Yes, Democrats also have been known to promote optimistic forecasts!

{ 20 comments }

Jonathan April 6, 2011 at 12:49 pm

Hey Andrew, I’m not an economist, but I do have some familiarity with the hysteresis literature. There is significant evidence (most conservatives don’t believe this,Friedman in particular posited a natural rate of unemployment that would always be around 3-4 percent. However, there is significant evidence to show that prolonged periods of unemployment can change the ‘natural rate’, ie…there is no natural rate!

Here in the Atlantic they did a good summary of the literature:

http://www.theatlantic.com/magazine/archive/2010/03/how-a-new-jobless-era-will-transform-america/7919/

Benquo April 6, 2011 at 2:44 pm

Couldn’t you just pack a lot of that downside uncertainty into, say, the interval between 2.5 and 2.8?

bradluen April 6, 2011 at 3:03 pm

Bill Beach, who runs the Heritage Center for Data Analysis, hasn’t flat-out admitted the number stinks, but he has said it’s “pretty low”, and the offending numbers have been discreetly removed from the current version of the report.

Apparently the model starts with the CBO baseline unemployment number, then subtracts the estimated effect of the policy. For 2021, they start with a 5.2% CBO forecast, then subtract a 2.4% policy effect, giving 2.8%.

Beach said the questionable part of the forecast was the CBO’s 5.2%, not Heritage’s 2.4%. This begs the question of whether or not the effect calculation was conditional on the CBO estimate. It would seem silly to estimate the change in unemployment without accounting for the number that you’re changing from. But it also seems silly to think that any plausible policy would reduce unemployment by 2.4% when the baseline is 5.2%.

Thomas April 6, 2011 at 3:25 pm

Doesn’t the same criticism apply to the original estimate as well? And most other estimates, including the forward-looking estimates from CBO? We routinely see estimates of unemployment in future years at 5% (as the natural rate. (See the CBO’s most recent Budget and Economic Outlook, for example.) And yet if we take the recent high unemployment rate (10.1) as an obvious choice for the upper bound, then we’re down below zero for the lower bound and a broad part of the lower range is in the range you describe as implausible.

Mfsheldon April 6, 2011 at 4:33 pm

Andrew, Andrew…

I thought you would have caught their math error, but you did not.

They simply miscalculated the unemployment rates.

Notice that in 2012 the net jobs impact is 831,000 jobs. With a labor force of 154 million now (and probably 157 in 2012) that would only reduce unemployment by about 0.5%.

Thus, the net impact on unemployment CANNOT be 2.1%. It is a mathematical impossibility.

They simply had an error in their summary table.

Thus the 2012 baseline of 8.4% would be reduced to 7.9% based on their jobs figures.

That is why they removed it from the report. They flubbed the rate calculation.

It is embarrassing, and they should be more upfront in admitting the error.

I am just surprised you did not notice it. IT WAS OBVIOUS.

Mfsheldon April 6, 2011 at 4:58 pm

I have calculated what the real impact on unemployment would be, assuming an annual growth in the labor force of 0.9%, which is the average of the last 10 years.

Year Base Plan Net
2012 8.4% 7.9% -0.5%
2013 7.6% 6.9% -0.7%
2014 6.8% 6.1% -0.7%
2015 5.9% 5.2% -0.7%
2016 5.3% 4.7% -0.6%
2017 5.3% 4.6% -0.7%
2018 5.2% 4.4% -0.8%
2019 5.2% 4.2% -1.0%
2020 5.2% 4.1% -1.1%
2021 5.2% 4.0% -1.2%

This assumes a labor force of 170 million in 2021.

So the implication is that the final rate is not 2.8%, but 4.0%.

This is still quite aggressive, but certainly within the theoretical boundaries.

This basically matches the peak employment reached during the late 1990s economic boom.

…so it is not unprecedented.

Zach April 6, 2011 at 7:39 pm

“P.S. Yes, Democrats also have been known to promote optimistic forecasts!”

It doesn’t make the prediction any less overly rosy, but it’s worth thinking through what would happen had Obama’s team released a forecast that turned out to be exactly right in retrospect. What he did release fell squarely within the range of estimates by other groups at the time. Had the forecast instead been 12-13% unemployment without stimulus and 9-10% unemployment with stimulus, folks would’ve said he was ensuring success by moving the goalposts.

Tyson April 6, 2011 at 8:19 pm

I didn’t do the calculations by I suspect Mfsheldon is right. That would explain why they say that the proposal would reduce 2012 unemployment by 2.1 points (obviously ridiculous).

But the point about using some reality to inform the model is spot-on. Look at how well they predicted job creation from the Bush tax cuts. If they are using the same coefficients from those days, then we know their model has a serious mismatch with the real world.

Tyson April 6, 2011 at 8:33 pm

There is a pretty good critique of Heritage Foundation’s predictions from 2001 and 2003 regarding employment, economic growth, housing, etc. Their record isn’t too good, and it doesn’t seem like they updated their model.
http://www.americanprogress.org/issues/2011/04/ryan_heritage.html

Andrew Gelman April 6, 2011 at 8:50 pm

Mfsheldon:

It’s their job to get their calculations right. It’s not my job to find their errors. My real point is statistical, that there’s no way to construct any kind of reasonable forecast for the unemployment rate in 2021 in which 2.8% is a sensible point estimate.

Krugman found 2.8% to be implausible–but it’s even more implausible if you think of it as a point summary of a probabilistic forecast.

Thomas April 6, 2011 at 10:53 pm

Tyson, they’re using a third party model. You are welcome to attack its reliability, but its reliability is not Heritage’s or the authors’. (The model was cited favorably a few months ago when it suggested the Obama stimulus plan had been effective. I suppose you’ll want to throw that result out as well.)

Nameless April 6, 2011 at 11:09 pm

“It’s their job to get their calculations right”

Actually it is not. Their job is to produce numbers that would support conservative policies. They might have a valid model, but it will have many parameters, and those parameters will be tuned to make a model that leans conservative.

Unemployment rate is a highly derivative parameter. It is derivative of employment level, which, in turn, is derivative of labor demand, which is derivative of investment and spending.

The fundamental result of their model is, apparently, that changes in taxation (primarily the capital gains & corporate tax cuts) immediately lead to a massive, huge, incredible jump in (a) _pre-tax_ corporate profits (24% higher in 2012, compared to baseline!) and a comparable jump (21%) in “Residential Investment–Structures” (in other words, people buying new houses). It’s very hard to see where these come from, because there’s no corresponding jump in personal income, disposable income, or employment, interest rates are flat, and effective personal tax rate is actually up slightly. From there on it’s a chain of causation, with higher corporate profits leading to higher investment and higher employment across the board, higher incomes, etc. Every single one of this causal links is suspect or can be tuned in various ways to make results more conservative or more liberal. (For example, if you ask Krugman, he’ll probably say that, in the present economy, the effect of corporate profits on investment is zero, because companies already sit on vast quantities of money and see no need for investment.)

Andrew Gelman April 6, 2011 at 11:19 pm

Nameless:

I don’t care how you tweak the assumptions; I don’t see how you can get a plausible forecast distribution for which 2.8% is a reasonable point summary. But, I agree, I’d like to see what went into the model to see what went wrong.

More to the point, the 2.8% point forecast should’ve been a red flag to them that something was going wrong.

I understand that these people have a political agenda, but they must have some professional pride, no? Failing that, they have careers to consider. Put out a few discredited forecasts and you lose credibility yourself. Even a politically-connected think tank ultimately wants some credibility with the news media; otherwise why put out press releases at all?

Tyson April 6, 2011 at 11:41 pm

Thomas,

OK, they’re using a third party model. I didn’t know that, but I’m not sure it’s relevant. There are lots of third party models out there, right? Do you think if they bought a third party model and it predicted that a decrease in taxes would lead to big budget deficits, they would publish those results and issue press releases about it? I don’t think so. I’m sure they picked a model that fit with their ideological beliefs (I don’t mean that in a sinister way – they probably think that is most likely to be an accurate model). But when their predictions from the model bombed so badly over the past 10 years, shouldn’t they be a little more careful about making predictions from that model in the future, and not proclaim that unemployment will fall to historic lows, and housing will return to historic highs, and so on? If it’s a third party model, shouldn’t they tell that third party to tweak the model so they don’t keep looking ridiculous?

And yes, of course, if a bad model made the stimulus look good, toss that model out. It wasn’t just one model that said the stimulus helped, that was the majority opinion.

Nameless April 7, 2011 at 1:19 am

Truth is, I suspect that the 2.8% was put in manually.

A model like that would probably have the natural level of unemployment, aka NAIRU, as one of the input parameters. It should be set at something like 5% by default and occasionally adjusted upwards in presence of structural unemployment or as a consequence of periods of high inflation. Model can’t predict that the unemployment rate will go below NAIRU without predicting rising inflation. Since that is exactly what we see in Heritage predictions, it follows that they set NAIRU to 2.8%.

Why would they do that? That’s harder to answer. Lower unemployment results in higher real GDP and higher tax receipts. (Which is exactly what we see in their predictions: the entire difference in GDP in 2015 and beyond between baseline and Ryan’s budget is because the unemployment rate was allowed to go below 5%.) But that is not a big effect. Maybe they believe that Ryan’s budget will increase the labor participation rate, but their model was not equipped to accept that, or it was producing something opposite. Maybe they simply wanted to report that “total employment grew … above the CBO alternative budget baseline”. In fact, if you take their original data and use it to calculate total labor force (= payrolls / (100%-unemployment) ), the result is that total labor force SHRINKS by some 2.2 million people as soon as Ryan’s budget is adopted. If labor force shrinks and unemployment is kept at 5.2%, there’s no way you can report that total unemployment will grow.

From what I read about Ryan’s old budget back in 2010 (and assuming that it’s the same as Ryan’s new budget), it actually raises taxes on middle class, so shrinking work force would be a logical outcome.

Tyson April 7, 2011 at 1:56 am

Nameless – everything you say makes sense. But Beach says they changed NAIRU to make it 2 points higher, and nothing else changes. That’s a pretty interesting model where you can change something seemingly important like unemployment and nothing else changes.

Nameless April 7, 2011 at 3:05 am

Well, they didn’t say that nothing else changes. They said that “the overall results elsewhere in the model do not change _significantly_.” Which is not a complete lie. Their forecast for “federal debt held by the public” (a 40% reduction by 2021, compared to baseline), for example, is essentially unchanged by higher NAIRU. And, since they only seem to raise it to 4.2% (so it’s still much lower than the baseline assumption of 5.2%), they may even be able to continue claiming net gain in total employment by 2021.

Let’s see if they put out a new spreadsheet with everything recalculated.

Nameless April 7, 2011 at 4:19 am

Btw, I think this theory of mine could make a good blog post. I don’t have a blog of my own, but I could put together a post for this one.

Dave X April 7, 2011 at 11:06 am

Btw, http://www.blogger.com/ is giving them away for free.

Nameless April 7, 2011 at 1:57 pm

Comments on this entry are closed.

Previous post:

Next post: