Of all education policies debates, few draw stronger opinions than school vouchers. The issue has almost become a litmus test for Democrats and Republicans. Given the strong passions and interests, I have long argued that this was a topic where research was unlikely to really influence the debate.
Maybe I was wrong. Last week, we released four Technical Reports and a Policy Brief on the Louisiana voucher program, led by the prolific University of Arkansas voucher researcher Patrick Wolf. Days later, there were open calls to eliminate the program. The main driving force behind this pressure is the special session of the Louisiana Legislature to address the state’s major fiscal crisis, but the evidence is being cited. Is that justified? Does the evidence mean the program should be eliminated?
Below, I summarize the evidence and try to help put this in perspective for policymakers as they weigh their options:
• Effects on Achievement Among Voucher Users . Students who use the voucher to enroll in private schools end up with much lower math achievement than they would have otherwise, losing as much as 13 percentile points on the state standardized test after two years. Reading outcomes are also lower for voucher users. While the size of the reading effects remain essentially constant across the two years, it is only statistically significant in the first year. In short, it looks like these students had a bad first year, then regained some, but certainly not all, of the losses in the second year.
• Competitive Effects on Achievement Among Public School Students .The program may have modestly increased academic performance in public schools, consistent with the theory behind school vouchers that they create competition between public and private schools. That said, most of the estimates are statistically insignificant, except for public schools located quite close to private schools participating in the program.
• Effects on Non-Achievement Measures . There is no evidence that the voucher program has positive or negative effects on students’ non-cognitive skills, such as “grit” and political tolerance. (On this point, the authors conclude that the data are not up to the task of identifying effects. So, I am going to follow suit and not say anything more on this.
• Effects on Racial Integration . The program reduced the level of racial segregation in the state. The vast majority of the recipients are black students who left schools with student populations that were disproportionally black relative to the broader community and moved to private schools that had more white students.
The way we interpret these multiple findings depends a bit on which outcomes you care about most, but it’s hard to see this as anything but bad news for Louisiana vouchers. While the positive competitive effects partially offset the negative effects for users, I think most would agree that we shouldn’t sacrifice the achievement of one group by sending them to low-performing private schools only as an incentive to public schools to get better. Also, while I personally think segregation is an important issue, it is not one of the objectives of the program.
So, how should we look at this evidence? Interestingly, even those skeptical of vouchers did not call for their repeal in the panel discussion we held when we released the reports. But here are the key points that I think policymakers should understand as they consider the future of the program:
First, whether we look at the first or the second year, these are the worst effects I have ever seen–for vouchers or anything else. Remember, the first-year effect was -24 percentile points and this “improves” to only -13 percentile points in the second year.
Second, the results do show improvement in the second year, but what this means for the future is unclear. If the improvement continues, then perhaps, eventually, the results will turn neutral or positive. This is what happened in New Orleans with the charter-focused post-Katrina reforms as the RSD began closing low-performing schools. The same could happen here since similar accountability is built in to the voucher program. That said, the initial results with the charter reforms were not so negative the way they are here with vouchers, so it’s hard to imagine the voucher story ends as well.
But this is still speculation. Wolf and his colleagues plan to study the program going forward, including adding more important outcomes such as high school graduation and college-going. Getting these longer-term results is especially important given what I wrote earlier about how the negative effects are probably driven by the misalignment between the state test and the private school curriculum. Test scores alone may not be the best way to measure the effectiveness of any education program and that problem may be worse with vouchers.
The researcher in me would like to see the program continue at least a bit longer so we can see whether things improve. Public schools are often criticized, unfairly, when they try something and it doesn’t generate immediate benefits. As Rick Hess has written, they end up spinning their wheels in an endless cycle of new programs. They also need to consider the possible effects on students currently in the program. Also, completely cancelling the program would force students to bounce back and forth between schools, which could be bad for everyone. On the other hand, we want policymakers to use evidence to inform their decisions, so we can’t tell them to wait forever, especially with the initial poor results.
How long should we wait to see whether the program is working? That is a question that only lawmakers can answer.
– Doug Harris
This first appeared in Education Week’s Urban Education: Lessons from New Orleans on March 2, 2016. Reprinted with permission from the author.
