In George Orwell’s classic novel, 1984, the author paints a picture of a dystopian society ruled by an all-powerful central government. One of Big Brother’s strategies for keeping people in line is to modify the existing language into Newspeak, where key words are “not merely changed into something different, but actually contradictory of what they used to be.” War actually is peace. Freedom is slavery. You get it.
Newspeak plays a central role in the latest Center for American Progress (CAP) broadside against the idea of low-income parents choosing private schools for their children (because, you know, freedom is slavery). The most extreme claim in the essay, among many, is that “the effect of vouchers on student achievement is larger than the following in-school factors: exposure to violent crime at school…” Yep, you read that correctly: selecting a private school for your child is as damaging to them as witnessing school violence.
How does CAP get from parental choice to Marjory Stoneman Douglas? The journey is truly Orwellian. Let’s start with the essay’s first paragraph:
How bad are school vouchers for students? Far worse than most people imagine. Indeed, according to the analysis conducted by the authors of this report, the use of school vouchers—which provide families with public dollars to spend on private schools—is equivalent to missing out on more than one-third of a year of classroom learning. In other words, this analysis found that the overall effect of the D.C. voucher program on students is the same as missing 68 days of school.
First, let’s unpack the phrase “analysis conducted by the authors of this report.” We Oldspeak social scientists tend to use the term “analysis” to mean the examination of statistical patterns in original quantitative data. If that is what you expect from the CAP essay, you will be sadly disappointed. The “analysis” by Ulrich Boser, Meg Benner, and Erin Roth merely cherry-picks the worst result from among many in an actual analysis of the DC Opportunity Scholarship Program and compares it to other negative results from other education studies. Scholars would not and should not call that “analysis.” In Oldspeak, we would describe it as “selective and speculative extrapolation.” But Newspeak is more efficient than Oldspeak. Analysis is whatever CAP says it is.
The use of the term “report” also is important. In social science, “report” is short for “a report of the findings from an analysis.” It is synonymous with the noun “study.” We’ve already established that the CAP essay is not based on an original analysis. The authors use the term “report” likely because of its close association with the more scholarly term “study.” In Oldspeak, we would describe the CAP essay as “commentary.”
Is there a legitimate basis for the claims made against school vouchers in the CAP commentary? Let’s move on to the second paragraph of the essay:
This analysis (again the Newspeak) builds on a large body of program evaluations in Louisiana, Indiana, Ohio, and Washington, D.C., all of which show that students attending participating private schools perform significantly worse than their peers in public schools—especially in math.
Here are the Newspeak translations:
• “Large body” means “five studies,” selected out of 20 rigorous experimental and quasi-experimental evaluations that exist on private school choice in the U.S. The authors claim the commentary relies on six studies but one of the supposed studies is a commentary by Mark Dynarski and Austin Nichols that discusses the other actual studies. So “six”, in Newspeak, means “five,” which somehow is a “large body.” Got it?
• “All of which” means “some of which,” as the multi-year Louisiana study cited (of which I am co-author) reports no significant achievement impacts of the program after three years and the Indiana study cited reports that the initial negative results of that program turn positive in reading by year four.
• “Worse than their peers in public school” is incorrect for all five studies. For the Louisiana and DC studies, the analyses compare students who won a voucher lottery to students who lost a voucher lottery. In both places, some of the students who lost lotteries enrolled in private schools anyway but remained in the randomized control group for purposes of calculating the effects of the program. Even more of the control group members attended high-performing public charter schools in their communities after losing the lottery. True, charter schools are “public” schools, but they are special kinds of public schools and should be described as such (at least in Oldspeak). In the Indiana study, the most rigorous program estimates come from an individual fixed-effects analysis, where the achievement gains of students while in the voucher program are compared to their achievement gains when not in the program. They are not compared to their peers but to themselves. The Ohio study matched EdChoice students with descriptively similar public school students at baseline and kept every student in their original group after that, regardless of who in either group actually attended private or public schools. These studies are rigorous precisely because they do not simply compare voucher students with “their peers in public school.”
• “Especially in math” means “almost exclusively in math.” The only lasting negative reading effect in this selective set of voucher studies comes from the Ohio study. The DC study, which is the focus of the commentary, only observes negative effects in math.
That is no fewer than six Newspeak incidences in just the first five sentences of the CAP commentary! The authors also do a bunch of other things that are questionable from a scientific perspective, such as generalizing to all school voucher programs from the results of a single study (of the most atypical city in our country, DC), that only just started (it has two more years of results to report). They also convert the standardized effect sizes from the DC study into percentile changes for the “typical” DC student at the 50th percentile. Few students in the DC study were performing even near the 50th percentile at baseline and the percentile scores varied dramatically by grade level, a crucial factor that the DC researchers accounted for in their conversions but the CAP commentators did not. The only legitimate takeaway from this Center for American Progress commentary on private school choice is that, in Newspeak, “American Progress” appears to mean something very different.
— Patrick Wolf
Dr. Patrick J. Wolf is Professor and 21st Century Chair in School Choice in the Department of Education Reform at the University of Arkansas College of Education and Health Professions.
