The disappointing 2017 NAEP scores will likely be used, by some, as evidence of the failure of some recent education policy reform such as Race to the Top, the Common Core, or “Choose Your Pet-Peeve Policy.” However, I argue that the low scores may have been caused, in large part, by events that happened almost a decade ago. [1]
While 2017 scores are all well below those from 2013 (the last year of consistent test score growth), they are largely a continuation of the already-declining scores observed in 2015. Since the NAEP was originally administered 25 years ago, 2015 was the first time that math test scores had fallen in both 4th and 8th grade, and the first time that NAEP scores declined in three of the four key groups tested. [2] Some had hoped that the 2015 results were a momentary blip, but the 2017 scores reveal that this is not the case. Thus, to understand the relatively low 2017 scores, we need to answer the question, What could have led to this change in national trends in 2015? Some compelling evidence points to a decline in public school spending caused by the Great Recession.
Lining up the Years
Figure 1: NAEP scores and cumulative per-pupil spending
Because the recession started in December 2008, one might have expected NAEP scores to have declined in the 2009 or 2011 assessments, if my hypothesis is correct. However, this ignores some key facts. First, the American Recovery and Reinvestment Act of 2009 injected funds to insulate district budgets from recessionary cuts for a year, so national per-pupil spending didn’t decline sharply until 2010. Second, learning is a cumulative process, and 2015 outcomes reflect the schooling inputs for previous years. As such, a reasonable measure of the level of resources a student is exposed to at the time of testing is the average per-pupil spending during the previous five years. Using this measure, because spending was increasing before the recession, the 2015 testing cohort was the first cohort to have been exposed to appreciably lower levels of average spending for the preceding five years. To see how this cumulative measure of per-pupil spending lines up with NAEP math scores in 4thand 8th grade, Figure 1 shows these two side-by-side. [3] The cumulative spending measure and NAEP scores clearly move together. While one should be skeptical of national time trends as evidence of causation, the similarity in trend is suggestive and indicates that a deeper analysis is in order.
More Credible Evidence
Two recent studies have now examined the effect of the recession on test scores using finer-grained data. First, Kenneth Shores and Matthew Steinberg linked population-level achievement data from the Stanford Education Data Archive (SEDA) to district-level per-pupil spending data. They document that public school districts with the largest declines in instructional expenditures during the recession had worse achievement outcomes and that this grew with additional years of exposure to the Great Recession. This result highlights the cumulative nature of recessionary exposure, and also suggests that the national decline in test scores associated with lower per-pupil spending is echoed at the district level. Because recessions may affect student outcomes through channels other than school budgets (such as parental employment or neighborhood crime), the Shores and Steinberg result likely reflects all ill-effects of the recession rather than those through reduced per-pupil spending per se. This motivates the next study.
To isolate the effect of the recessionary spending cuts from that of the general ill-effects of the recession, Cora Wigger, Heyu Xiong and I rely on the fact that states that relied heavily on state taxes to fund public schools experienced the deepest education revenue cuts during the recession, on average. But importantly, these states were no more or less likely to experience higher unemployment or other ill-effects of the recession. We compare the outcomes of states that were heavily dependent on state revenues to fund public education before during and after the recession. We find that such states were on a similar trajectory of per-pupil spending before the recession, but experienced appreciable declines in per-pupil spending after the recession (see top panel of Figure 2). Recall that the spending declines associated with dependence on state funds are not associated with other ill-effects of the recession. We then examined if NAEP test scores also declined in these states relative to other states. We find that states that experienced larger recessionary budgets cuts (but not higher levels of unemployment or poverty) experienced a decline in NAEP scores that is most pronounced in the 2015 NAEP (see lower panel of Figure 2)—further evidence that recessionary school spending cuts did indeed lower student achievement.
Figure 2: Evolution of school spending and test scores for states with heavy reliance on state revenue sources to fund public education
Note: Because per-pupil spending is measured in logs, the coefficient represents percentage changes (e.g. a decline of 0.2 is roughly a twenty-percent reduction). The dashed connected lines depict the difference in outcomes between states that relied heavily on state revenue sources in 2008 and those that did not in each calendar year. To aid interpretation, this difference is centered on the 2007 value. The gray area depicts the 90 percent confidence interval for each estimate. The solid lines represent the linear fit during the pre-recession period and post-recession periods.
Source: Jackson, Wigger, and Xiong (2017).
To Conclude
Taken as a whole, the evidence is compelling that the dip in NAEP scores in 2015, which has persisted into 2017, was due in some measure, to the Great Recession and its impact on school spending. The results suggest that school spending does matter, and that even transitory school-spending cuts may be experienced for several years. If my hypothesis is correct, then NAEP scores will likely not return to 2013 levels until per-pupil spending has returned to pre-recession levels for about three years. Even though current spending has almost recovered from the Great Recession, based on current trends, cumulative spending may not recover until 2019 or even 2021.
—C. Kirabo Jackson
Kirabo Jackson is professor of human development and social policy at Northwestern University.
Notes
1. After the drop in NAEP scores in 2015, researchers speculated that the cause was Common Core. While the Common Core states did underperform other states on average, even the Common Core states experienced a decline in scores in 2015, suggesting that Common Core was not the cause of the broad general decline in scores. (https://www.brookings.edu/research/why-did-naep-scores-drop/)
2. Math scores were lower in both 4th and 8th grade than 2013, and 4th grade reading scores were not different (in a statistical sense) in 2015 than in 2013, and were lower in 8th grade reading in 2015 than in 2013. (https://www.nationsreportcard.gov/reading_math_2015/#?grade=4)
3. Using this relationship, increasing per-pupil spending by 10 percent is associated with about 0.12 standard deviations higher test scores (this relationship is statistically significant at the 1 percent level).
This post is part of a series from Education Next analyzing the 2017 results from the National Assessment of Educational Progress. Click here to check out the series, and stay tuned throughout the week of April 10 for more analysis.
