This blog entry is part of a debate over “Boosting Educational Attainment and Adult Earnings,” by C. Kirabo Jackson, Rucker C. Johnson and Claudia Persico, a study which was published in the Fall 2015 issue of Education Next. That article was based on “The Effects of School Spending on Educational and Economic Outcomes: Evidence from School Finance Reforms,” The Quarterly Journal of Economics (forthcoming). Eric Hanushek first critiqued the study in “Money Matters After All?” published July 17, 2015 on the Ed Next blog. The authors of the study responded in a blog entry, “Money Does Matter After All,” also published on July 17, 2015 on the Ed Next blog. What appears below is a response to “Money Does Matter After All.”
My critique of the paper by Jackson, Johnson, and Persico is very simple and might be lost in the dazzling misdirection of their response. When I learned computer programming, I was taught to use simplified approximations of results to make sure that my more complicated, and harder to check, programs produced answers that were in the right ballpark. This step apparently is no longer taught.
Jackson, Johnson, and Persico claim that a 22.7 percent spending increase is large enough to eliminate the average outcome difference in school attainment between the poor and non-poor. This is their estimate of the causal effect of added spending. Between 1970 and 2010 we saw real spending increases per pupil of roughly 150 percent, or over six times what they claim is necessary to close the average attainment gap between poor children (those in families below two times the poverty level) and nonpoor children (those in families above two times the poverty level). Applying their estimates of the causal effect of added spending to the actual increases in spending suggests that the average poor-nonpoor gap in school attainment should have been more than closed – which is not even close to what we observe. A large and frustratingly-resilient average attainment gap continues to exist between children in poor and nonpoor families.
Separating wheat from chaff is difficult, so a few added remarks are useful.
•Nobody suggests that one should use the time series pattern of spending increases to estimate the causal effect of spending. The approximation in my critique was introduced only to see if their answer could be in the right ballpark. It does not look to be.
•There is also no suggestion in my critique that there would not continue to be a distribution of outcomes within poor and nonpoor groups. If the average poor and nonpoor students got something close to a 150 percent increase in spending, the Jackson, Johnson, and Persico results imply that the gap in average attainment by poverty status should have been more than closed and indeed reversed. This estimate of the effect on average differences says nothing about whether there is a remaining distribution of attainment, which one would guess that there would be.
•My calculations are not an extrapolation beyond the bounds of their data, because they pertain to the data Jackson, Johnson, and Persico use in obtaining their causal estimates.
•Nobody disagrees that over time the U.S. student population has changed. There are more children in single-parent families, more children in families where English is not the first language, and more children in families below the poverty level – all factors that might lead to a more difficult to educate student population. But there are also more educated parents and smaller families over time – factors that might lead to an easier to educate population. No analysis has accurately netted out these influences. But, back to their causal estimates: one would have to believe that the net changes in families fully absorb the six-fold spending increase above their mean-equalizing level in order to reconcile actual spending increases with the clear lack of any dramatic outcome improvements. If the “quality” of our children has really deteriorated so much, we as a nation are in much more trouble than most people believe.
•The investigation of “mechanisms” by Jackson, Johnson, and Persico does not show that reducing pupil-teacher ratios or increasing teacher salaries is causally related to better outcomes. They show that court-induced funding tends to go more toward these crudely-measured inputs than non-court-induced spending – but they do not show that these factors actually led to the outcome improvements that they identify. Indeed these findings on mechanisms also fail a simple sniff test: Between 1970 and 2008, pupil-teacher ratios for the nation fell by over thirty percent. Indeed this is the biggest driver of the increase in spending over that period, and spending unrelated to court orders followed this policy very closely – but outcomes did not.
•When proclaiming the importance of their causal estimates of the impact of school spending, there are no qualifications: “Our findings provide compelling evidence that money does matter, and that additional school resources can meaningfully improve long-run outcomes for students.” But, when interpreting their results for policy, they are much more circumspect: “Spending increases should be coupled with systems that help ensure spending is allocated toward the most productive uses” – a conclusion that is unrelated to their causal analysis but that reflects exactly the conclusion to which most people have come.
The central issue remains: How do we ensure that added funds to schools are used in ways that improve student outcomes? Jackson, Johnson, and Persico and I completely agree that just providing money is not sufficient for getting good results. The time series evidence shows that the existing incentives in schools have not produced consistently better results to go along with dramatically increased funding, even if some funds have been used effectively. We apparently disagree on the other half – the necessity of spending, but I fail to see how their analysis or other available evidence shows that more money is a necessary condition for improvement. It may or may not be necessary, but arriving at any conclusion on this depends on specific policies. We need to see the results of a set of policies that consistently improves achievement in order to assess whether (and how much) more funding is really necessary.
– Eric Hanushek
Eric Hanushek is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University.
