Matthew Chingos and I have just released a study that for the first time makes use of data from a randomized field trial to identify the impact of school vouchers on college enrollments. We found that that college enrollments for low-income, African American students who used a voucher to go to private elementary school increased by24 percent. In the absence of a voucher, only 36 percent of the African American students in the study went to college either full-time or part-time within three years of their expected high school graduation date, but that percentage jumped to 45 percent, a 24 percent increase, among those who used the voucher.
In the spring of 1997 over 20,000 New York City elementary school children applied for a half-tuition voucher offered by the School Choice Scholarships Foundation, and a lottery was held to pare the number of lottery winners to around thirteen hundred students, the number that SCSF resources could support. Because a lottery had been used to award vouchers, the evaluation of their impact took the form of a randomized experiment, the gold standard research design. Those who won the lottery were compared to those who did not. Any difference between them could be attributed to the voucher opportunity.
Working on this project in the summer of 2012 has brought back memories of the picnic atmosphere in Central Park that summer day when SCSF handed out voucher certificates to the excited families that had won the lottery. I was no less enthusiastic than those standing on the podium as this was for me an opportunity to evaluate for the first time a school voucher initiative by means of a randomized field trial, the gold standard for evaluating the effectiveness of a program, the same design as the pill-placebo design used in medical research to ascertain whether pills are effective.
On that summer day I did not realize just how difficult it would be to follow the lives and accomplishments of both those who won the lottery and those who did not. We wanted to see whether students performed better on standard tests if they won the voucher opportunity and went to private school. But we needed to test students outside the school day so that public and private school students were placed on equal footing. It was an expensive struggle to arrange for this testing for three years in a row. As it turned out we were unable to test about a third of the students.
Given time constraints and thousands of applicants, plenty of mistakes were made. MPR, the evaluation firm that administered the lottery and collected the data, over-ran its first-year budget, and only a last-minute plea for additional funding prevented the lottery from falling apart. Later, disagreements within the evaluation team spilled into the news media. Some thought the best evidence came from averaging all the test score results together, while others thought the scores of students at each grade level should be looked at separately.
In my view it was pretty clear that the results showed test score gains for African Americans but not for the students from Hispanic and other ethnic backgrounds. But critics said nothing could be learned from a study with such a high rate of attrition. Quite apart from that issue there remained a nagging question. Even if low-income African American students scored better on tests in elementary school, do those tests have any significance for more meaningful outcomes in the future, including high school graduation and college enrollment?
Yet one little-noticed part of the evaluation was brilliantly accomplished: Before the lottery was run and the vouchers were awarded, students and parents had been asked to attend verification sessions where eligibility for the voucher was established. For most students, a social security number was obtained.
Gradually, both SCSF and the evaluation faded from public discourse. Scholarships were extended through eighth grade but the program was otherwise discontinued. Those who participated in the debate over the evaluation moved on to other projects.
It wasn’t until quite recently that enough time had passed to examine the college enrollment decisions of students in the voucher study. We obtained college enrollment information from the National Student Clearinghouse (NSC), which has information on college enrollments from institutions of higher learning that serve 96 percent of the college students in the United States.
Amazingly, 99.1 percent of those included in the lottery could be identified precisely enough so it was possible to see if they appeared in NSC records as a college student. The attrition problem that had plagued the earlier research simply evaporated. All the controversy over test scores could be placed on the shelf, as much better information on something much more important was now available.
Our just released findings are striking. In addition to boosting overall enrollment, the voucher opportunity enhanced enrollment in private universities, which have a better record of seeing students through to graduation day. Better than 14 percent of the African American students attended a 4-year private college if they were offered a voucher, a 57 percent increase above the 9 percent that would have attended otherwise. The percent enrolled in a selective college (where the average student had a SAT score of 1100 or better) more than doubled from 3 percent to 7 percent.
As in the earlier study, we found no significant impact on Hispanic student performance. The best explanation for the lesser impact of the program on Hispanic students is that they were more likely to enroll in college even in the absence of a voucher. Forty-five percent of the Hispanic students in the control group enrolled in college within 3 years of expected high school graduation (as compared to just 36 percent of African American control group members). In other words, the Hispanic students were less at risk of not getting to college. Also, Hispanic parents reported less of a climate change when their child moved from public to private school. For African Americans, more than Hispanic students, going to private school greatly reduced the amount of fighting, cheating, property destruction and other negative influences.
Ours is not the first study to find positive benefits from vouchers for African Americans. What’s new is solid evidence that those benefits persist over the long haul.
Paul E. Peterson, professor of government, is the director of Harvard University’s Program on Education Policy and Governance and is a Senior Fellow at the Hoover Institution.
NB: An op-ed about the new study appears in this morning’s Wall Street Journal. And a videotaped interview about the study appears here.
