Do public-school students who move to a private school with a government-funded voucher benefit from making this switch? A growing body of research is shedding light on this question. Of particular interest are findings coming out of three states and the District of Columbia, all of which have implemented ambitious voucher programs over the past dozen or so years. That evidence does not seem to justify the fervor for vouchers displayed by many education reformers and now by U.S. secretary of education Betsy DeVos.
First, what do we know about the students who choose to participate in voucher programs? It almost goes without saying that families who choose to use tuition vouchers are less satisfied with their traditional public schools than those who stay. And since the vast majority of private schools are religiously affiliated, it comes as little surprise that voucher users tend to be more religious.
Participation in voucher programs is also driven by eligibility requirements. For example, programs that target low-income families directly or indirectly (by virtue of being based in urban areas) will enroll low-income students. Even when limited to low-income populations, though, vouchers tend to serve a socioeconomically advantaged portion of that group, those who are best positioned to leverage choice. Why? Mostly because this is how markets work. Economic research shows that more-educated adults are more likely to get what they want in the marketplace writ large.
This dynamic intensifies in the schooling market. In a practical sense, families who lack personal transportation or live far away from private schools do not have access to alternatives. Also, most private schools, in the absence of vouchers, are designed for the wealthy and middle class. It is wealthier families who can afford to pay tuition, and school eligibility requirements often exclude students who have academic and disciplinary challenges. That is largely because parents know what researchers have confirmed: students benefit from attending school with more-advantaged classmates. School reputations therefore depend to a substantial degree on exclusivity. This is also why we must view parent satisfaction cautiously. Research shows that white or affluent parents often avoid schools that have high concentrations of minority and low-income students. This might make them more “satisfied,” but it is hardly a reason to celebrate.
Some evidence about exclusivity comes from my fellow participants in this forum, who have shown that private schools considering whether or not to accept voucher students often worry about having to lower their academic standards. Given the correlation between family socioeconomic advantage and the student characteristics that schools look for, this concern on the part of private schools will restrict access for voucher-bearing students. Religious schools, a partial exception, are more willing to participate in means-tested voucher programs, but they, too, often have academic and behavioral admissions requirements. In some cases, schools institute these policies in a well-intentioned effort to build strong scholastic communities, but their criteria effectively serve to exclude many students. Even though voucher programs often officially prohibit selection practices, these rules are rarely enforced, and research shows that schools have many ways to shape enrollment that fall outside the rules.
Vouchers that are targeted to disadvantaged students could theoretically help address the affordability barrier, but, in general, when governments target attractive financial benefits to one part of the population, politically powerful groups will exert pressure to loosen eligibility requirements to gain their own access. We have seen this in Indiana, Louisiana, and other places where small-scale, means-tested voucher programs gradually expanded to include families closer to middle class. The trend is toward voucher programs becoming less well targeted, with funding shifting to socioeconomically advantaged students who already have some degree of choice.
Effects of Vouchers
And what do we know about the academic outcomes of students participating in voucher programs? Most of the rigorous research, now dating back more than a decade, focuses on programs in large urban areas, such as Milwaukee and New York City. Averaging across 16 U.S.-based programs, Patrick Wolf and colleagues find that these small-scale voucher programs have statistically insignificant effects on standardized test scores across academic subjects.
This interpretation is different from what Wolf concludes from the same results in his contribution to this forum. The reason for the difference is noteworthy. In his Figure 1, Wolf shows that studies of voucher programs that examine students’ outcomes longer after they switch to a private school produce more positive results. However, this conclusion assumes that the programs had no effect on the achievement of students who switched to a private school only temporarily and, even if valid, could only be generalized to the small subset of students who used a voucher consistently when given the opportunity to do so, ignoring the smaller and perhaps negative effects on the majority of voucher users. The analysis also includes a now dated study of the Washington, D.C., program, the effects of which have since turned negative (more on this below). Moreover, the other handful of studies that lasted for four years might only have done so because they were relatively successful, so that the less successful ones are omitted. In contrast, my interpretation—that there is no statistically significant effect—places equal weight on all students observed using a voucher to enroll in a private school, including all studies and students regardless of how long they were followed, therefore capturing all the positive and negative effects at work. Note, too, that even Wolf’s more optimistic interpretation asserts a positive achievement effect only for reading, not for math.
These early programs’ effects on non-achievement outcomes, such as graduating from high school, tend to be somewhat more positive. Even if we place more weight on these outcomes, however, we need to keep in mind that effects found in small-scale programs often do not generalize to larger scales. School-choice initiatives, including charter schools, seem to work better in cities than statewide because it is easier to exercise choice where there is better mass transit and higher population density, and the performance of traditional public schools is generally worse in urban areas, making it less challenging for choice programs to improve on baseline student outcomes.
The limited scale of the programs examined in most prior studies is important, because the United States is now in the midst of a full-scale nationwide expansion of these policies. Twenty-five states and the District of Columbia have some type of voucher program. Just four statewide voucher programs have been formally evaluated, and only one has shown any signs of success. Ohio’s statewide program has shown clear negative effects on test scores. Two others, in Indiana and Louisiana, started off with some of the worst test-score results any education program has ever demonstrated, though these subsequently improved somewhat so that the net effects are now essentially zero. In the fourth program (Florida), the authors conclude that test-score effects cannot be determined with any confidence because of the program design.
Why are the results for test scores not more positive? One possible reason is that the state-mandated tests were not well aligned to the curriculum taught in private schools. However, the most recent experimental evaluation of the D.C. voucher program showed negative test-score effects after one year, even though the study did not rely on a state-mandated test—and despite the fact that an earlier study of the program showed no effects. The more likely explanation is that private schools in the city are competing with a public- and charter-school system that has demonstrated substantial academic improvements in recent years. More generally, where vouchers are competing with charter schools—which have produced increasingly positive results over time—the voucher results are likely to continue to be less positive. It is harder to look good against stronger competition.
Another possible reason for the uninspiring results is that the private schools that participate in statewide voucher programs are simply not very effective. This could be interpreted as a failure of the voucher concept or, as voucher supporters have asserted, it could be that the regulatory burden of the programs, while very small compared with those of public or charter schools, kept the best private schools from joining. Research by Wolf and colleagues does not seem to support the latter interpretation, however. Based on ratings from the organization GreatSchools, the schools participating in the Louisiana voucher program were not of lesser quality than those that did not participate, though the voucher-accepting schools did charge lower tuition.
The effects of the Florida Tax Credit (FTC) scholarship program on college outcomes have been widely cited as a success story, but several caveats apply here. First, unlike the other studies mentioned above, the design of the FTC program precludes a rigorous research design. In their Florida study, Matthew Chingos and Daniel Kuehn do their best by matching students on observable characteristics that are somewhat removed from the outcome of interest. But this strategy is unlikely to yield an apples-to-apples comparison. In studies using test scores as the outcome, matching is much more effective, because the treatment and comparison groups of students can be matched on their scores—the variable
of interest—before students receive vouchers. This is not possible when studying college-going. Also, because the assignment to the FTC is not random, the more positive effects they see for students participating for more years may, as they acknowledge, reflect selection bias; that is, any student who stays in the same school for more years is likely to have better outcomes.
Earlier research in D.C. provides evidence of positive effects on another important long-term outcome, high school graduation, but these findings are now difficult to interpret for another reason. The downward trend in test-score results in D.C. calls into question whether prior outcomes still reflect the current reality, given increased competition from charter schools.
Still Waiting
What has all this taught us about how states ought to design and oversee voucher programs—and, indeed, whether they should do so at all? How about slow down? The latest results should give everyone pause.
Try this exercise: Let’s drop the word “voucher” and simply say, “Statewide _________ programs show a mix of tenuously positive and negative results.” Now, fill in the blank with your favorite non-performing program. It is hard to
imagine that any objective observer would respond by saying, “Great, let’s expand this to states across the country.” Yet this seems to be what DeVos and half the states in the country have concluded about vouchers.
Robert Pondiscio, senior fellow at the Thomas B. Fordham Institute, has argued that empirical evidence is largely irrelevant to determining whether vouchers and other forms of choice programs should be adopted, because choice aligns schooling with a core American value: freedom. Certainly, it is desirable that education policy support our most fundamental principles, and freedom is at the top of the list. However, this argument assumes that choice policies, by definition, increase freedom. Whether that assumption holds true depends on what form of freedom we mean and how policies are designed. A voucher program that allows schools to set their own rules and does not provide transportation will increase freedom only for those students who meet the schools’ standards and who can find their own way to and from school.
Even that interpretation assumes we mean freedom in the libertarian sense—that is, freedom from restrictions on individual choice. School attendance zones, which assign children to a particular school by their neighborhood of residence, do curtail this kind of freedom. But the freedom that comes from promoting educational opportunity is also important. Unlike the libertarian form of freedom, opportunity is mostly an empirical issue, as it requires not only that families be unfettered by government policies in selecting schools for their children, but also that they are able to choose from among accessible, high-quality options. We can debate what criteria define quality—strong test scores versus parent satisfaction, for example—but the assertion that opportunity is an empirical issue is hard to dispute.
It is also important to consider how voucher programs contribute to or detract from other salient cultural values such as equity, community, and democracy. The fact that vouchers are likely to open access only for some creates an immediate concern for equity. Apprehension that vouchers will undermine neighborhood schools—and the neighborhoods themselves—is also well founded at a time when geographically based communities are already under great stress.
The voucher debate, therefore, is a question not just of values but also of effectiveness, and research should play a significant role. So how should we interpret the available evidence? At most, only one of the more than two dozen states that have tried statewide vouchers and tuition tax credits has yet to demonstrate convincing, measurable success with them, Given this reality, it is hard to make a case for substantially replacing our system of public schooling on a national scale. The American workforce continues to be the most productive and creative in the world. This does not mean we cannot do better, but it does indicate that we should proceed with caution and care.
Finally, we cannot interpret the voucher evidence without thinking about the alternative policy options. Vouchers represent just one form of choice. Given the multitude of ways in which we would expect a free market in schooling to fail, perhaps other forms of choice that strike a different balance between government and market forces would be more effective. The evidence on charter schools, for example, is increasingly positive—even at scale. Perhaps what some call the “portfolio model,” and what I have called “managed competition,” will do more to increase freedom, equity,
efficiency, and community. A system of managed competition gives families genuine choice in schooling, but it also ensures 1) true accessibility to these options; 2) transparency, including data reporting and open board meetings; 3) coordination of school operations with a government body that has some degree of authority; and 4) government enforcement of the rules and protection of students’ civil rights. It also seems likely that different localities need different systems, and many might be best served by maintaining traditional public schools.
We have been debating vouchers for decades, even centuries, without much evidence to inform those debates. Today, policy advocates are way out in front of the evidence, especially with the current proliferation of statewide voucher programs. The new federal expansion of tax-favored 529 savings plans to include tuition for private schools, a move that constitutes reverse targeting to the affluent, has even less justification. It would be wise to put a hold on further broad-based experiments until we see whether the dozens of relatively new programs yield more positive results than the older ones. When it comes to convincing evidence, we are still waiting.
This is part of a forum on private school choice. For alternate takes, see “Programs Benefit Disadvantaged Students,” by Patrick J. Wolf, or “Lessons Learned from Indiana,” Mark Berends, R. Joseph Waddington, and Megan Austin.
This article appeared in the Spring 2018 issue of Education Next. Suggested citation format:
Wolf, P.J., Harris, D.N., Berends, M., Waddington, R.J., and Austin, M. (2018). Taking Stock of Private-School Choice. Education Next, 18(2), 46-59.
