In a recent Brookings report, Mark Dynarski cites studies of Indiana and Louisiana private school voucher programs that show negative effects, raising provocative questions regarding the impact and viability of such programs.
There is a better response to Dynarski’s questions than those typically offered by ideologues on both sides of the voucher debate. Rather than arguing for or against voucher programs, policymakers should start with a simpler question: how can we most efficiently offer as many high-quality schooling options to low-income Americans as possible?
A productive response to that question would not contemplate the merits of “voucher programs” per se but would instead view vouchers as one vehicle among others for growing the number of high-quality individual school choices available to low-income families. Such a response would necessarily insist on transparent measurement of academic progress in participating schools, with consequences for the school based on academic performance and proportionate to the scale of public investment in the school. And such a response would acknowledge that, as in any industry, it takes time for organizations to respond to accountability pressures. Over time, as in public schools, the good will separate from the bad, and policy makers and parents will respond to information at hand.
This is a simple formula: fund individual schools of all types, measure their performance on an equivalent basis, provide information to the consumer, and intervene in instances of significant failure. The Louisiana studies Dynarski cites, in spite of their initial, negative effects, offer a nascent view of both the challenges and opportunities of such an approach.
School Choice in Louisiana
The goal of Louisiana’s private school choice policy is to expand over time the number of high quality, free or low-cost schooling options available to low-income families.
The state thus regulates its private school choice programs through a “sliding scale,” providing schools multiple options of varying regulatory intensity through which to make seats available to publicly funded students.
In the least regulated circumstance, called the Louisiana Tuition Donation Rebate Program, private donors receive state rebates for up to 95 percent of their contributions to approved non-profit scholarship funds, which in turn award scholarships of up to $4,000 for elementary and middle school students and $4,500 for high school students. Families apply directly to the participating school and are responsible for the tuition and fees that exceed the amount of the scholarship. Schools select eligible students they wish to admit, based on the school’s established admissions criteria, and participating students are required to take state tests in English language arts and math. The state reports results, but the test results do not impact a school’s participation in the program.
In a moderately regulated environment, eligible families receive state vouchers that are equivalent to participating schools’ published tuitions, up to a limit equivalent to state and local public school per capita funding (approximately $8,500 on average). Families rank their choices on a state application, and schools must accept all students assigned to the school; there are no admissions criteria permitted. Schools must administer state tests in four subjects to participating students, but for schools participating at a lower level – those with fewer than 40 publicly funded taking state tests and fewer than 10 publicly funded students on average in all grade levels (state tests exist only in grades 3-11)—the state reports results publicly but does not use them to rate the school or enforce sanctions as a result of those ratings.
In the most regulated environment, larger participants—those schools with 40 or more students funded through vouchers in testing grades, or with an average of 10 or more students per grade across all grade levels—receive a rating through a formula identical to the school performance score system used by the state to gauge public school performance, inclusive of test score performance, graduation rates, and other outcome metrics. Participating private schools with unacceptable ratings are barred from accepting new students receiving vouchers for the following year.
This is a system designed to invite nonpublic schools into the market through multiple avenues, to measure their performance, and to intervene quickly when there is evidence of failure. By accommodating schools’ varying tolerance levels for regulation, it opens the door to many actors. By requiring testing in the same grades as in public schools, and by enacting significant consequences for results in larger scale participants, it is designed to identify and grow effective schools, with an acknowledgment that, as in public schools, some schools will have difficulty adjusting to new academic expectations and a small number may prove incapable of ever adequately responding to expectations.
Studies of Private School Choice in Louisiana
Dynarski cites two studies of Louisiana schools participating in the state’s voucher program. The first, conducted by Abdulkadiroglu, Pathak, and Walters, showed significant negative effects for students participating in private school voucher programs in both math and English in 2012-2013, the first school year of the statewide voucher program’s existence. [i] In order to meet their own publishing deadlines, the researchers opted not to include available results from later years in their study. The second study, conducted by Mills and Wolf, showed similar negative effects in year one but also showed that by year two of the program, differences in math had moderated and there were no significant differences in literacy performance. [ii] The Arkansas and Tulane researchers will soon conclude research on year three of the program and will continue the evaluation over time.
Click to enlarge
Conventional metrics collected by the Louisiana Department of Education show that performance among the students in Louisiana’s voucher program has considerably improved since the first year. The gap in basic proficiency on state tests between participating private schools and public schools statewide, for example, has closed from 27 percentage points in 2013 to 18 points in 2015. Were Louisiana’s private school voucher program considered a school system for purposes of analysis, it would have ranked number 9 out of 71 districts across the state in 2015 for annual improvement in the district performance score system—inclusive of test score performance, graduation rates, and other outcome metrics—used by the state to gauge overall district performance.
This does not, of course, guarantee that the researchers’ findings in year three and beyond will show continued upward movement. But it does seem to comport with a recent meta-analysis of global and U.S. voucher initiatives, by Shakeel, Anderson, and Wolf, showing a prevalent and consistent trend toward early dips in performance followed by year-three and year-four positive effects, in both literacy and mathematics. [iii]
Click to enlarge
Louisiana schools participating in the statewide voucher program in 2012-2013 had never before participated in an accountability system using standard measures. As a result, school leaders had yet to address complex but fundamental academic issues, such as rudimentary alignment between curriculum and assessment. Similarly, absent a quality rating system, neither parents nor state administrators had comparative measures to use in assessing schools’ strengths and weaknesses.
In the years since, the state has used quantifiable results to identify persistently weak schools, suspending their ability to accept new students. Stronger schools likewise now understand better the state’s regulatory environment, making it more likely they will participate and grow. Parents review test scores and school ratings for participating schools alongside parallel information for public schools; in the city of New Orleans, parents seeking admission in fact use the same form to rank public school preferences and publicly-funded private school preferences. And schools themselves have developed increasingly sophisticated academic responses to accountability, ranging from closer examination of curriculum and assessment alignment, to more aggressive personnel strategies, to establishing relationships with outside support organizations that provide significant academic support.
Whether results in Louisiana will correspond with the trends found by Shakeel, et al, is yet to be determined. But it is at least plausible that possessing both initial years’ results and familiarity with the instruments used to produce those results allows school leaders, parents, and regulators to make affirmative, strategic decisions they were less likely or less able to make in the program’s nascent years.
Against Programs, for Schools
None of this is to argue for the supremacy of private school voucher programs. In Louisiana, we have seen many life-saving schools accept students using vouchers, just as we have seen a small number of fiscally or academically irresponsible schools accept such students. The challenge lies not in defining whether public schools or private schools are better but in how we avail families of measurably good schools of all types while countering the risks that come with inevitable bad actors in the public and private sectors alike.
For a model of how private school vouchers can achieve measurable quality at scale, advocates and policy makers might look to the charter school sector strategy in states like Louisiana, Massachusetts, New Jersey, New York, and Tennessee, where non-governmental actors understood the centrality of measurement to making the case for scale. In these states, non-profit charter school boards fought for equal funding and autonomy from regulation, while embracing accountability and acknowledging the need for low-achieving schools to close down shop. This strategy has engendered increasingly bi-partisan support for charter schools, along with the philanthropic and governmental investment that comes with it. In turn, these states and schools have made critical investments in talent and infrastructure that have allowed for sustainability, growth, and improvement over time.
Such a strategy—one anchored in measurable results—takes time. Policymakers will benefit from research showing effects over multiple years and decades. Such a strategy also calls for researchers to ask more nuanced questions than simply whether or not voucher programs are better than public school programs. Inevitably there is great variation within both the public school sector and the private school sector, not just between the sectors writ large. Researchers should seek to understand variation among schools rather than among sectors, identifying factors that lead some schools to excel irrespective of sector.
Advocates of such a strategy should encourage liberals to accept that their argument against the school choice rights of low-income Americans runs contrary to the principles of equity they otherwise embrace. The conscience of a liberal should struggle with supporting a system in which the children of the poor are consigned to attend the school that is assigned to them by public officials, regardless of its quality, whereas more affluent parents can shop for the school they want for their children by purchasing a home in the vicinity of the public school they prefer or paying private school tuition. A better liberal position is to support non-biased admissions policies and transparent measurement for all schools that accept publicly funded students, regardless of the sector in which they operate.
Likewise, conservatives must understand that the current case for voucher expansion too often rests on the shaky foundation of ideology rather than the sturdy foundation of measurable results. The very legitimate conservative case for school choice would be bolstered by a commitment to measurable, comprehensible results and a low tolerance for failure.
A country that is serious about its education system and the civil rights of its people should be serious about making all quality school options available to low-income people. Private schools are an important part of this picture. A long-term approach that defines academic quality and funds nonpublic schools that verifiably meet the definition is both feasible and warranted.
—John C. White
This post originally appeared as part of the Evidence Speaks series at Brookings
Notes: [i] http://www.nber.org/papers/w21839 [ii] http://www.uaedreform.org/downloads/2016/02/report-1-the-effects-of-the-louisiana-scholarship-program-on-student-achievement-after-two-years.pdf [iii] http://www.uaedreform.org/the-participant-effects-of-private-school-vouchers-across-the-globe-a-meta-analytic-and-systematic-review
