Did the stimulus work? A review of the nine best studies on the subject
Posted by Dylan Matthews at 02:32 PM ET, 08/24/2011 TheWashingtonPost * Text Size * Print * E-mail * Reprints * Share: * * * More > * Facebook * Twitter * LinkedIn * Reddit * StumbleUpon * Digg * Delicious * top: -5px;">Google +1

(MLADEN ANTONOV - AFP/GETTY IMAGES)
If you ask the Obama administration, economists are virtually united in thinking the 2009 stimulus package worked. “I’m absolutely convinced, and the vast majority of economists are convinced, that the steps we took in the Recovery Act saved millions of people their jobs or created a whole bunch of jobs,” Obama declared at a press conference last month. Or, to quote NEC chair Gene Sperling from an interview a few weeks ago, “There is no question that the evidence is showing that the type of things the president did to help state and local governments really mattered, were really helpful in pulling us from the brink of depression to a recovery.”
But the stimulus’ critics allege that this evidence isn’t reliable. The studies the administration is relying on depend on models that “substitute assumptions for identification,” Harvard economist Robert Barro writes today in the Wall Street Journal. “To figure out the economic effects of transfers one needs ‘experiments,’” Barro writes, “in which the government changes transfer in an unusual way—while other factors stay the same—but these events are rare.”
The truth is, both studies of the type Barro prefers, and studies using models, which he criticizes, have been conducted to determine the effect of the stimulus on employment and output. Of the nine studies I’ve found, six find that the stimulus had a significant, positive effect on employment and growth, and three find that the effect was either quite small or impossible to detect. Five studies use econometric ”experiments,” which attempt to, as Barro encourages, sort out the effect of the stimulus from other factors using empirical data. Four use modeling instead.
Each approach runs into its own set of problems. The econometric studies have to deal with what social scientists call “endogeneity”: that is, the variable whose effect we’re trying to determine (the stimulus) could itself be affected by what we’re trying to study its effect on (the state of the economy). In this specific case, this means that econometric studies sometimes have to correct for the fact that harder-hit areas tend to get more stimulus spending. This says nothing about the stimulus’ effectiveness, but it can confuse attempts to evaluate that effectiveness statistically.
All of these studies have their own methods of overcoming the endogeneity problem, some of which are more effective than others. Whichever corrections one uses, however, one cannot run a perfect experiment with messy, real-world data, which necessarily limits what these studies can say. Of the five econometric studies detailed here, three conclude the stimulus had a significant positive effect, and two conclude it did not have much of an effect at all.
The modeling studies use an equation or series of equations meant to model the economy to compare the results of a certain policy change (like the stimulus bill) against the results of a baseline in which the change was not enacted. This avoids the messiness of econometric evaluation, as it allows the creation of a ready, stimulus-less counterfactual with which one can compare the results of the stimulus bill. But it also doesn’t take into account the actual changes in employment and output that occurred after the stimulus was passed. Further, there is considerable disagreement within the economics profession about macroeconomic modeling, and for any of these studies, one could find economists who dispute the value of the model used. Of the four modeling studies, three conclude the stimulus