If you mostly care about test scores, private school choice is not for you. Despite the vast majority of randomized control trials (RCTs) of private school choice showing significant, positive test score effects for at least some subgroups of students, some of those gains have been modest and other effects have been null for at least some subgroups. And now we have two RCTs, in Louisiana and DC, showing significant test score declines for at least some subgroups and in some subjects. The Louisiana decline is large and across-the-board, but the significant, negative effect in the new DC study appears to be “driven entirely by students in elementary grades not previously in a needs-improvement school.”
People will quibble over why these new DC results showed at least a partial decline. They will note that the prior RCT of DC vouchers showed significant test score gains after three years (although the p value rose to .06 in year four even as the positive estimate remained). They will note that vouchers in DC are worth almost 1/3 as much as the per pupil funding received by DC’s traditional public schools and almost half as much as DC’s charter schools. Imagine how they might do if they received comparable resources (and yes resources can matter if there are proper incentives to use resources productively). They will note that almost half of the control group attended charter schools, so to a large degree this study is a comparison of how students do in vouchers relative to charters.
But these largely miss the point — the benefits of private school choice are clearly evident in long term outcomes, not near-term test scores. In the same DC program that just produced disappointing test score effects, using a voucher raised high school graduation rates by 21 percentage points. Similarly, private school choice programs in Milwaukee and New York City were less impressive in their test score effects than in later educational attainment, where private school students in both cities were significantly more likely to enroll in college.
But if what you really care about is raising test scores, you’d be pushing no-excuses charter schools. Rigorous evaluations, like the one in Boston, show huge test score gains for students randomly assigned to no-excuses charter schools. You don’t even have to have school choice to produce these gains. The same team of researchers showed that schools converted into no-excuses charters as part of a turnaround effort produced similarly big gains for students who were already there and did nothing to choose it. The lesson that a fair number of foundations and policymakers draw is that we don’t need this messy and controversial choice stuff. They believe that they have discovered the correct school model — it’s a no excuses charter — and all we need to do is get as many disadvantaged kids into these kinds of schools as we can, with or without them choosing it.
Unfortunately, no excuses charters don’t seem to produce long-term benefits that are commensurate with their huge test score gains. The Boston no excuses charter study, for example, shows no increase in high school graduation rates and no increase in post-secondary enrollment despite large increases in test scores. It’s true that students from those schools who did enroll in post-secondary schooling were more likely to go to a 4 than 2 year college, but it is unclear if this is a desirable outcome given that it may be a mismatch for their needs and this more nuanced effect is not commensurate with the giant test score gains.
This same disconnect between test scores and later life outcomes exists in several rigorously conducted studies of charter schools, including those of the Harlem Promise Academy, KIPP, High Tech High, SEED boarding charter schools, and no excuses charters in Texas. While of course we would generally like to see both test score gains and improved later life outcomes, the thing we really care about is the later life outcomes. And the near-term test scores appear not to be very good proxies for later life outcomes.
So, what should we think about these new test results from DC vouchers, showing some declines for students after one year in the program? We already know from rigorous research that the program improves later life outcomes, so I don’t think we should be particularly troubled by these test results. It may be that control group students are in schools that will fare as well or better on test score measures. But we should remember that 42% of that control group are in the types of charter schools that other research has shown can produce giant test score gains without yielding much in later life outcomes. And we know that treatment group students are in a program that has previously demonstrated large advantages in later life outcomes.
I understand that many reporters, foundations, and policymakers act like they mostly care about test scores and these new results from DC have them all aflutter. But if people could only step back for a second and consider what we are really trying to accomplish in education, the evidence is clearly supportive of private school choice in DC and elsewhere.
— Jay P. Greene
Jay P. Greene is endowed chair and head of the Department of Education Reform at the University of Arkansas.
