Yong Zhao, “Side Effects in Education: Winners and Losers in School Voucher Programs.” Phi Delta Kappan, 100(5), pp. 63-66.
Checked by Patrick J. Wolf
University of Kansas Professor Yong Zhao has jumped into the school choice debate with his recent essay in the Phi Delta Kappan. Welcome, Dr. Zhao.
Zhao makes several important claims about private school choice. He says that researchers who are proponents of school choice exaggerate the positive tilt of the findings. He claims that the debate over the achievement effects of school choice has largely ignored substantial variation in those effects across student subgroups. Finally, he concludes that we know almost nothing about the effects that school choice has on civic outcomes but should expect those effects to be negative.
School choice is new academic territory for Zhao, a Distinguished Professor who specializes in education technology and virtual learning. Permit me to offer him some friendly advice from someone who has studied the topic for over 20 years.
First, don’t call people names. Zhao labels school choice researchers as either “proponents” (a.k.a., “advocates”) or “independent researchers.” He doesn’t tell us how he arrived at those determinations, but two possibilities immediately come to mind. A scholar might be a school choice “proponent” if their intention is to promote choice and an “independent researcher” if their intention is to arrive at the truth about choice. To judge accurately who is which, Zhao would need to possess the capacity to look inside of the hearts of his fellow human beings and therefore observe their intentions. He would have to possess the God-like ability of comedian Woody Allen’s character in the movie Annie Hall, who said:
I was thrown out of NYU my freshman year for cheating on my metaphysics final, you know. I looked within the soul of the boy sitting next to me.
Human beings in general, and social scientists in particular, are terrible at judging other people’s intentions. We are better off classifying school choice researchers based on the rigor of their methods, not a feigned ability to know the deepest desires of their hearts.
Zhao might, instead, be classifying school choice scholars as “proponents” or “independent researchers” based on the nature of the findings that we report. It appears that, in Zhao’s view, a school choice “proponent” is any scholar who reports that a school choice program has positive effects on student achievement. In contrast, an “independent researcher” is any analyst who agrees with Zhao that school choice is bad. Zhao’s claim that school choice “proponents” report “more significant positive effects than independent researchers” (p. 64) thus reduces to a tautology. Proponents report more positive effects because reporting more positive effects makes them proponents. A better approach for Zhao and others would be to forgo the arbitrary and unscientific labels he foists on the researchers and focus, instead, on the quality and findings of the research itself.
Second, get your facts right. Zhao criticizes the body of research on the achievement effects of school choice for focusing “only on the average effect of school choice on the students who participate in it.” Really? Few education interventions have been evaluated with greater attention to their possibly varied effects on different subgroups than has private school choice. The first study published about a school voucher program in the U.S. examined differences in the achievement effects of the program across student subgroups. Almost every one of more than two dozen major private school choice evaluations in the subsequent 21 years has examined the extent to which voucher achievement effects vary by student characteristics. The focus of the three-city evaluation that informed William Howell and Paul Peterson’s seminal book The Education Gap: Vouchers and Urban Schools (see “Vouchers in New York, Dayton, and D.C.,” research, Summer 2001) was the finding that African American students experienced larger and more consistent achievement effects from school choice than students of other ethnicities. One can easily find additional peer-reviewed publications of the possible heterogeneous effects of vouchers in New York City here, here, here, and here; in Washington, DC here, and here; in Milwaukee, here and here; in Indiana here and another one under review; and in Louisiana here and here. Contrary to Zhao’s claim, examining possible variation in voucher achievement effects has been an obsession of school choice researchers, not a blind spot. Who does and does not benefit academically from school vouchers has been central to the debate.
There is a scientific method for determining if the effects of an intervention vary across subgroups that Zhao seems not to use. Zhao claims that voucher programs have heterogeneous achievement effects if the impacts are positive and statistically significant for some subgroups but not statistically different from 0 (i.e. null) for other subgroups. He makes the same claim regarding voucher effects that are null for some subgroups but statistically significantly negative for other subgroups. Zhao is making much of nominal differences in the pattern of voucher achievement effects that may be due to statistical noise endemic to the small subgroup samples in most of these studies. A program has a single, general effect on participants unless its effects on different subgroups are, themselves, significantly different from each other based on statistical tests. The only statistically valid heterogeneous pattern so far uncovered in a study of the achievement effects of school vouchers is the positive and statistically significant effect of the private school choice program in New York City on African American students, which itself was significantly different from the program’s null effect on non-African American students. School choice researchers have uncovered few scientifically valid differences in the impacts of school choice on achievement for different subgroups of students, though not for lack of trying.
Third, know the subject of which you speak. Zhao concludes his essay with the claim that, “Little, if any, empirical evidence has been collected concerning other equally important outcomes of schooling, such as preparing students for civic engagement and betterment of a shared society.” Actually, there is a deep and broad research literature on the mostly positive effects of school choice in general and private schooling in particular on civic values such as political tolerance, volunteering in one’s community, political knowledge, political engagement, social capital, and patriotism. I co-edited a book, Educating Citizens, on the topic in 2005. Rigorous studies demonstrating the positive private school effects on enhancing civic values can be found here, here, here, here, here, here, here, here, here, okay you get the picture. So many studies of this relationship exist—when Zhao claims there are “little, if any”—that comprehensive peer-reviewed summaries of the evidence have been published in these pages (see “Civics Exam,” research, Summer 2007) and elsewhere.
The evidence supporting the private school advantage in promoting civic values is so compelling that one would think it would be a settled matter by now. So long as doubters like Professor Zhao continue to ignore the wealth of published evidence to the contrary, empirical research on school choice and civic values will continue. Many commentators, however, insist on trusting their ideological preferences on the matter of school choice and civic values instead of their lying eyes. I would say that “proponents” of a public school advantage in promoting civic values present a different picture “than independent researchers”, but I don’t like to call people names.
Dr. Patrick J. Wolf is Professor and 21st Century Chair in School Choice in the Department of Education Reform at the University of Arkansas College of Education and Health Professions.
