Diane Ravitch is angry. She is upset because parental school choice is thriving in Milwaukee. Over 25,000 students are enrolled in the city’s pioneering private school voucher program and nearly 19,000 more attend the city’s public charter schools. The fact that so many parents are choosing alternatives to traditional Milwaukee Public Schools (MPS) bothers Ravitch, as is apparent from her latest screed.
Ravitch spends much of her blog post attacking my motives and credibility as an evaluator of school choice programs. I am delighted when commentators such as Ravitch spend their time and energy attacking me as a person because that demonstrates that they don’t have the ability to critique the methodological rigor and quality of my actual research. For the most part, the best that Diane Ravitch can do is call me names. Fine. Doesn’t bother me. I keep winning the competitions to perform the most important private school choice evaluations around the country, and regularly publish my results in the very best scientific peer-reviewed policy journals (see , , and here), Ravitch’s ad hominem attacks notwithstanding.
But Ravitch does spend at least a few paragraphs discussing my team’s research findings regarding the Milwaukee Parental Choice Program, and that part of her blog post is riddled with factual and methodological errors. To be fair, Diane Ravitch is not a social scientist. She has never performed a statistical evaluation of anything, so perhaps it is not surprising that she doesn’t understand the social science that she nevertheless attacks. She is an education historian, however, and historians are supposed to care about facts — supposed to, at least.
Ravitch dismisses the findings from my DC and Milwaukee voucher evaluations that these programs increased the educational attainment of students in the form of higher rates of high school graduation, college enrollment, and persistence in college. She ignores the finding that perhaps because that is an inconvenient truth that she wishes were not so. Instead she claims that the similar Milwaukee finding of higher educational attainment from vouchers is questionable because “75% of the students who started in a voucher school left before graduation.” For support, she cites a review of our study performed by the National Education Policy Center (NEPC).
Now, professional historians cite original sources to make their claims, but, remember, we are talking about Diane Ravitch here. Is the NEPC claim credible? Let’s examine the original sources. From page 16 of our report, “the majority of students (approximately 56 percent) who were enrolled in 9th grade in MPCP were not enrolled there by the time they reached 12th grade.” Also, from page 163 of our article published in the prestigious scientific , “less than half (44 percent) of the original MPCP panelists examined were enrolled in a voucher school by the time they reached 12th grade.” I realize that Ravitch is no statistician but even she should know that 56 percent is not 75 percent and 44 percent is not 25 percent. It doesn’t excuse Ravitch that the factual error was first promulgated by NEPC. She should know better than to trust the accuracy of their “reviews” when primary source material clearly contradicts them.
Ravitch compounds her major factual error with a methodological one. She says, “So of the 25% who persisted, the graduation rate was higher than the Milwaukee public schools. But what about the 75% who dropped out and/or returned to MPS? No one knows.” Every element of that statement is wrong. Our primary results regarding the higher attainment of the Milwaukee voucher students are not drawn from the students who remained in private schools for all four years. Our conclusions are based on the graduation rate for all students in the choice program who were in 9th grade in the fall of 2006, regardless of whether or not they left the program prior to graduation. Scientific evaluators will recognize this approach as an “intention-to-treat” analysis which corrects for selective attrition from a program over time. We clearly explain and justify our approach in the actual report and our peer-reviewed publication, neither of which Ravitch appears to have actually read.
Ravitch claims that “No one knows” what happened to the students who left the choice program during high school. This is another falsehood. We were able to track all of the students in our study into college (or not) via the National Clearinghouse of College Enrollments, regardless of whether they switched schools or school sectors during high school. Regarding high school graduation, for the voucher students who switched to MPS later in high school, we know exactly what happened to them, because we had access to MPS enrollment and graduation data. If they failed to graduate from high school, that fact pulled down the average graduation rate for the voucher program. If they did graduate, that improved the average graduation rate for the voucher program. The effect of being a 9th grader in the MPCP in 2006 was to increase your likelihood of graduating high school, enrolling in college, and persisting in college, regardless of where you were schooled after 9th grade. Professional evaluators will recognize that ours is a rigorous and highly conservative estimate of the educational attainment benefits of the MPCP.
Finally, Ravitch states “Not even Wolf’s evaluations have shown any test score advantage for students who get vouchers, whether in DC or Milwaukee.” Is she right? The executive summary of the final report in our longitudinal achievement study of the Milwaukee voucher program states: “The primary finding that emerges from these analyses is that, for the 2010-11 school year, the students in the MPCP sample exhibit larger growth from the base year of 2006 in reading achievement than the matched MPS sample.” Regarding the achievement impacts of the DC program, Ravitch quotes my own words that there was no conclusive evidence that the DC voucher program increased student achievement. That achievement finding was in contrast to attainment, which clearly improved as a result of the program. The uncertainty surrounding the achievement effects of the DC voucher program is because we set the high standard of 95% confidence to judge a voucher benefit as “statistically significant”, and we could only be 94% confident that the final-year reading gains from the DC program were statistically significant.
Diane Ravitch’s claim that school voucher programs have failed is based on ignoring much of the scientific evidence of their success, misreporting the facts regarding the studies that she does discuss, and the 1 percent difference between 95% confidence and 94% confidence. It takes a lot of doing for a person to mislead so many about so much, but apparently Diane Ravitch is up to the job.
-Patrick Wolf
