Ravitch Blow-Up on School Choice

By Guest Blogger 04/01/2013

15 Comments | Print | NO PDF |

Diane Ravitch is angry.  She is upset because parental school choice is thriving in Milwaukee.  Over 25,000 students are enrolled in the city’s pioneering private school voucher program and nearly 19,000 more attend the city’s public charter schools.  The fact that so many parents are choosing alternatives to traditional Milwaukee Public Schools (MPS) bothers Ravitch, as is apparent from her latest screed.

Ravitch spends much of her blog post attacking my motives and credibility as an evaluator of school choice programs.  I am delighted when commentators such as Ravitch spend their time and energy attacking me as a person because that demonstrates that they don’t have the ability to critique the methodological rigor and quality of my actual research.  For the most part, the best that Diane Ravitch can do is call me names.  Fine.  Doesn’t bother me.  I keep winning the competitions to perform the most important private school choice evaluations around the country, and regularly publish my results in the very best scientific peer-reviewed policy journals (see , , and here), Ravitch’s ad hominem attacks notwithstanding.

But Ravitch does spend at least a few paragraphs discussing my team’s research findings regarding the Milwaukee Parental Choice Program, and that part of her blog post is riddled with factual and methodological errors.  To be fair, Diane Ravitch is not a social scientist.  She has never performed a statistical evaluation of anything, so perhaps it is not surprising that she doesn’t understand the social science that she nevertheless attacks.  She is an education historian, however, and historians are supposed to care about facts — supposed to, at least.

Ravitch dismisses the findings from my DC and Milwaukee voucher evaluations that these programs increased the educational attainment of students in the form of higher rates of high school graduation, college enrollment, and persistence in college.  She ignores the finding that perhaps because that is an inconvenient truth that she wishes were not so.  Instead she claims that the similar Milwaukee finding of higher educational attainment from vouchers is questionable because “75% of the students who started in a voucher school left before graduation.”  For support, she cites a review of our study performed by the National Education Policy Center (NEPC).

Now, professional historians cite original sources to make their claims, but, remember, we are talking about Diane Ravitch here.  Is the NEPC claim credible?  Let’s examine the original sources.  From page 16 of our report, “the majority of students (approximately 56 percent) who were enrolled in 9th grade in MPCP were not enrolled there by the time they reached 12th grade.”  Also, from page 163 of our article published in the prestigious scientific , “less than half (44 percent) of the original MPCP panelists examined were enrolled in a voucher school by the time they reached 12th grade.”  I realize that Ravitch is no statistician but even she should know that 56 percent is not 75 percent and 44 percent is not 25 percent.  It doesn’t excuse Ravitch that the factual error was first promulgated by NEPC.   She should know better than to trust the accuracy of their “reviews” when primary source material clearly contradicts them.

Ravitch compounds her major factual error with a methodological one.  She says, “So of the 25% who persisted, the graduation rate was higher than the Milwaukee public schools.  But what about the 75% who dropped out and/or returned to MPS?  No one knows.”  Every element of that statement is wrong.  Our primary results regarding the higher attainment of the Milwaukee voucher students are not drawn from the students who remained in private schools for all four years.  Our conclusions are based on the graduation rate for all students in the choice program who were in 9th grade in the fall of 2006, regardless of whether or not they left the program prior to graduation.  Scientific evaluators will recognize this approach as an “intention-to-treat” analysis which corrects for selective attrition from a program over time.  We clearly explain and justify our approach in the actual report and our peer-reviewed publication, neither of which Ravitch appears to have actually read.

Ravitch claims that “No one knows” what happened to the students who left the choice program during high school.  This is another falsehood.  We were able to track all of the students in our study into college (or not) via the National Clearinghouse of College Enrollments, regardless of whether they switched schools or school sectors during high school.  Regarding high school graduation, for the voucher students who switched to MPS later in high school, we know exactly what happened to them, because we had access to MPS enrollment and graduation data.  If they failed to graduate from high school, that fact pulled down the average graduation rate for the voucher program.  If they did graduate, that improved the average graduation rate for the voucher program.  The effect of being a 9th grader in the MPCP in 2006 was to increase your likelihood of graduating high school, enrolling in college, and persisting in college, regardless of where you were schooled after 9th grade.  Professional evaluators will recognize that ours is a rigorous and highly conservative estimate of the educational attainment benefits of the MPCP.

Finally, Ravitch states “Not even Wolf’s evaluations have shown any test score advantage for students who get vouchers, whether in DC or Milwaukee.”  Is she right?  The executive summary of the final report in our longitudinal achievement study of the Milwaukee voucher program states:  “The primary finding that emerges from these analyses is that, for the 2010-11 school year, the students in the MPCP sample exhibit larger growth from the base year of 2006 in reading achievement than the matched MPS sample.” Regarding the achievement impacts of the DC program, Ravitch quotes my own words that there was no conclusive evidence that the DC voucher program increased student achievement.  That achievement finding was in contrast to attainment, which clearly improved as a result of the program.  The uncertainty surrounding the achievement effects of the DC voucher program is because we set the high standard of 95% confidence to judge a voucher benefit as “statistically significant”, and we could only be 94% confident that the final-year reading gains from the DC program were statistically significant.

Diane Ravitch’s claim that school voucher programs have failed is based on ignoring much of the scientific evidence of their success, misreporting the facts regarding the studies that she does discuss, and the 1 percent difference between 95% confidence and 94% confidence.  It takes a lot of doing for a person to mislead so many about so much, but apparently Diane Ravitch is up to the job.

-Patrick Wolf

Comment on this article
  • Dr. Julian Vasquez Heilig says:

    Dr. Patrick, Please hurry and de-identify the data you used in your papers and provide it to independent researchers. I have the ability to critique the methodological rigor and quality of your actual research. I am very very much looking forward to it.


    Julian Vasquez Heilig, Ph.D. ▪ Associate Professor, Educational Policy and Planning ▪ Associate Professor, African and African Diaspora Studies (by courtesy) ▪ Faculty Associate, Center for Mexican American Studies ▪ Faculty Affiliate, Center for African and African American Studies ▪ Coordinator, M.Ed. Program in Educational Policy and Planning ▪ The University of Texas at Austin College of Education Department of Educational Administration

    Cloaking Inequity named one of Top 100 education blogs in the world. http://teach.com/teach100

  • […] of Arkansas’ Patrick Wolf, in Education Next, offering another example of the once-respectable education historian’s intellectual […]

  • P. L. Thomas says:

    Methinks he doth protest too much.

    Wolf is among the cluster of choice-advocate “researchers” who take a better-than-thou pose to mask their essential advocacy.

    Follow the money—in more than one way: http://coehp.uark.edu/4109.htm

    Choice advocacy masked as complex statistical research makes for a powerful tool among politicians, the media, and the public.

    Just as researchers for the tobacco industry always made claims and produced data pleasing to Big Tobacco, Wolf et al. will continue to make claims and produce data that appeases the pro-school-choice mechanisms behind their work and even their positions.

  • JB says:

    Julian —

    Your ability to engage in accurate statistical analysis is rather questionable. Recently, you wrote a blog post comparing the “college readiness gap” for dropout recovery charter schools in Texas to the public schools from which the dropouts had come, as if this proved something about the charter schools’ effectiveness. A professional like Patrick Wolf would know better than to do that.

  • David N. Cox says:

    You can argue all you want about whether “choice” (vouchers, TTC, Opportunity Scholarships, or whatever new name to get tax dollars for private schools) is better for students NOW. What is not being discussed is what it will do to private schools long term.

    If tax dollars are approved to go to private schools everywhere, the first thing that will happen is that private schools who refuse to take the voucher, because they don’t want the government mandates that absolutely will come, will go out of business, just as private colleges did that refused to take Pell grants (the college voucher). They can’t compete financially with those who do. They will either have to take them or go out of business. (Notice that all private schools are now liberal as well.)

    When the government hook is good and set (all privates either taking the money or folded), THEN all the mandates and restrictions that are messing up the public schools will come, including protections for unions, etc.

    And then the only difference between private schools and public schools will be the ownership of the building. That’s what happened in every country who have implemented voucher plans, Sweden and the Netherlands particularly. And then there really will be NO CHOICE, because every choice will be the same!

    Can’t we see that vouchers are a Trojan Horse for private schools?

    The real answer is in eliminating government interference in local schools and dividing large school districts back to being truly local school districts. See http://www.smallerschools.org for a full plan for deregulation of schools.

  • David N. Cox says:

    Oh and by the way, vouchers will do the same thing for K-12 schools that Pell grants did for colleges. They INCREASED the cost, because of that little thing called supply and demand.

  • Bill Delaney says:

    Dr. Wolf,
    In this post you do your fair share of criticizing Dr. Ravitch as a person. Please forgive us for not being intellectual enough to understand the sophisticated nature of your own ad hominem attacks.

  • Hyddyr Caradoc says:

    Shame on you! You need to recheck your facts, apologize, and be VERY public about it!

  • Kevin Welner says:

    Patrick, your references to NEPC’s review are (at best) misleading. You and your colleagues published the 75% figure, which was correctly relied upon by Prof. Cobb in his review. Your republishing of a new version with the 56% figure does not change that. You are wrong to then dishonestly attack NEPC and Dr. Ravitch, implying incompetence or worse — and to mislead your readers in the process.
    I write more about this here: http://dianeravitch.net/2013/04/02/nepc-patrick-wolf-should-apologize/
    And your readers can see the original and revised pages of your report here: http://cloakinginequity.com/2013/04/02/4169/
    As I write on Dr. Ravitch’s site, I understand that you may have simply forgotten that you changed the 75 to a 56. But in any case a retraction and apology is in order.

  • phillip cantor (@phillipcantor) says:

    Ravitch’s response:


  • JB says:

    The response from Ravitch and her cronies is rather pathetic. They are seizing on one possible misprint (very quickly corrected) in one early version of a draft report, ignoring the fact that Ravitch is writing a year later and could easily have read the published version by now.

    But even so, Ravitch and her cronies completely ignore:

    1. Her willful omission of the important fact that vouchers in two cities have increased graduation rates.

    2. Her ignorance of what an “intent-to-treat” estimate is — that is, her ignorance of the fact that the Milwaukee study counted all the kids who started out in the voucher program towards the voucher program’s graduation rate, even if they had left the voucher program in the meantime.

    (What does this mean? It means that you have a higher chance of graduating even from a public school if you even spent time as a voucher student. It is ignorant to hold this fact over Wolf’s head as if it was bad for vouchers, rather than the opposite.)

    3. Her willfully incorrect claim that “no one knows” what happened to kids who left the voucher program and switched back to public school.

    4. Her willfully incorrect claim that Wolf’s own research has not shown any test score gains for voucher students.

    The only thing that’s surprising here is that a serious scholar would be forced to waste time responding to such a dishonest propagandist.

  • Brian Ford says:

    The essential question of changing schooling — whether it be charters or vouchers or public school choice — cannot be easily answered by research studies. What would have been the aggregate results for all students if the voucher program had not been implemented? This is the essential question – whether the system of schools is better overall, The answer requires evidence we do not have and will never have — counter-factual scenarios of what would have happened otherwise.

    In the Milwaukee case, just as in C. Hoxby’s study of NYC Charters, the effect of the program on the public schools — especially the peer group effect — is profound. That is, there is both a siphoning effect and an enclave effect (or what G.Canada refers to as the Quarantine Model). Given this, the results for either charters or private schools receiving vouchers need to be substantially better than for public schools. That, at least, would be my estimation.

  • John Norquist says:

    Brian Ford, you point to a key issue; does the implementation of private and/or parochial school options improve outcomes for students and the whole community over time. In Milwaukee, where I served as Mayor from 1988 to 2004, I believe the choice options have had a net benefit to students and to the community at large. Before school choice options were implemented Milwaukee had a school choice system like that in Detroit, St. Louis and many other cities that have concentrations of low income populations surrounded by more affluent suburbs with mostly Caucasian populations. Under the Detroit model families, with financial means and school age children tend to move to public school districts with affluent populations and rich tax bases avoiding the urban poor by not living anywhere near them. This fosters a system of extreme segregation, but it does preserve exclusive access to tax generated resources for the government schools. Detroit and St. Louis public schools have lost most of their student enrollment over the last 20 years not to private schools, but to the suburbs. The private and parochial schools in Detroit have nearly disappeared. By contrast, Milwaukee’s overall population has stabilized and the public schools have lost far lower percentages than Detroit or St. Louis. Instead of disappearing, Milwaukee’s private and parochial systems have held steady. In Milwaukee both private and public schools have attempted to please parents by offering attractive programming. For example, Montessori schools, which many parents are attracted to, in MPS have grown from 2 to 6 with a similar increase among privates. Even the Catholics have a school, St. Rose Academy, that uses Montessori methods even though Maria Montessori was a non believer. Programming and curricula with little or no attraction to parents has tended to disappear. I.e. the African American Immersion program at Malcolm X Academy was discontinued after even the activists who got MPS to create it wouldn’t send their children to it.
    The point is that Milwaukee, as a community, has experienced a net benefit from vouchers and charters. In my opinion the situation faced by Milwaukee parents, including those in the middle class, has, on balance, improved.

  • You Scared Bro says:

    JB – Have you guys released the data Dr. Heilig-Vasquez is requesting yet?

  • JB says:

    I have nothing to do with the Milwaukee data — it’s not up to me. And if the Milwaukee study is like pretty much every other educational study, it’s not up to the researchers to decide to release data to some third-party — there would be strict confidentiality agreements in place.

    Moreover, Vasquez Heilig (you got his name backwards) wouldn’t be a good choice: given the things he has said in public about intent-to-treat analysis and about comparing dropout recovery high schools, it’s unfortunately obvious that he is incompetent to evaluate programs in any sort of professional manner. He has the simulacra of qualifications, but he either doesn’t know what he’s talking about or is too much of a hack to care.

  • Comment on this Article

    Name ()


    Sponsored Results

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform