Checked: David Dodenhoff, “Fixing the Milwaukee Public Schools: The Limits of Parent-Driven Reform.” Wisconsin Policy Research Institute Report, Vol. 20, No. 8(October 2007).
Checked by Martin West
Heads understandably turned last October when the Milwaukee Journal Sentinel announced the release of a new study concluding that “school choice isn’t a powerful tool for driving educational improvement in Milwaukee Public Schools.” Since the early 1990s, Milwaukee has been home to an increasingly varied array of school choice programs that now includes the nation’s oldest voucher program, numerous charter schools, and extensive inter- and intra-district public-school choice systems. Had credible evidence emerged that these programs were for naught?
Equally startling was the study’s source: the Wisconsin Policy Research Institute, a conservative think tank funded in large part by the Lynde and Harry Bradley Foundation, one of the nation’s leading backers of school choice.
“The report you are reading did not yield the results we had hoped to find,” wrote George Lightbourn, a senior fellow at the institute and former secretary of the Wisconsin Department of Administration, in a foreword accompanying the study. Lightbourn lamented the study’s central finding that, despite the impressive array of school choice options available to Milwaukee parents, “only 10 percent have been the active consumers that would exert market-based influence to [sic]the school system.” He fretted, too,about low levels of parental involvement in the district, especially among parents of older children. “For children ages fourteen to seventeen,” he wrote, “only 11 percent of MPS [Milwaukee Public Schools] parents are actively involved both in the school setting and at home.”
Lightbourn may well have spared himself this agonizing. The study contains no direct information about the actual behavior of Milwaukee parents whatsoever—and certainly nothing that would justify the very specific claims stated above. Its author,“ David Dodenhoff, Ph.D.,” makes no claim to have interviewed a single Milwaukee parent, nor to have surveyed any of them by mail or on the Internet.
Instead, Dodenhoff bases his study on information gathered from a national sample of parents, not from anyone in Milwaukee. His approach assumes that, when it comes to school choice, the behavior of Milwaukee parents is identical to that of parents of similar demographic background nationwide, despite the fact that Milwaukee’s school choice environment is unique.
The method is akin to estimating the share of Hawaiians who surf by counting the number of surfers nationwide, no matter their proximity to a beach, the height of the waves, or the warmth of the water.
A flawed approach
Milwaukee has the most extensive system of school choice in any American city. As of 2005, more than one-third of the city’s parents chose either to enroll their child in a charter school, use a voucher to go to a private school, or seek out a place in a suburban public school. All other students in Milwaukee may choose among the city’s traditional public schools, a policy put in place years ago to foster school integration. Each winter, the school district asks parents to list up to three schools they want their child to attend the following fall. The vast majority of those who complete an applicationreceive their first choice.
Dodenhoff set out in his study to assess the potential for public school choice to improve student achievement in Milwaukee Public Schools. Despite the fact that such information is readily available, he did not find out how many public school parents are sending their children to suburban schools, or selecting a charter school, or filling out the form listing their three school choices within the traditional public sector.
Instead, Dodenhoff looked at what parents around the United States are doing, as reported in the National Household Education Survey conducted in 2003 by the U.S. Department of Education. That survey asked parents to report on whether they chose their child’s school, whether they chose from among more than two schools, and whether they took academic considerations into account when doing so. He uses this information to estimate the relationship between four parental characteristics (ethnic background, educational attainment, whether both parents are in the home, and mother’s employment status) and whether the parent is choosing the child’s school.
Dodenhoff claims that these characteristics are highly correlated with the likelihood that a parent will be choosing a school, but he presents no direct evidence on this point. He simply notes that another U.S. Department of Education Study based on a different data set showed that the variables he uses were “particularly influential determinants of parental involvement.” He does not report on their usefulness for predicting the likelihood of exercising school choice,in his data set or any other.
Dodenhoff then employs census data to estimate the distribution of Milwaukee parents on each of the four characteristics and uses their relationship with choice activity nationwide to estimate of how much choosing is going on in Milwaukee. The validity of his results thus depends on the four characteristics perfectly predicting a U.S. parent’s likelihood of choosing a school and the assumption that Milwaukee parents do exactly what parents are doing everywhere else.
Nonetheless, Dodenhoff straightforwardly reports that “just under 35 percent of MPS parents actively choose a school for their child, rather than simply opting for the default neighborhood school.” Worse, he reports less than 10 percent do so from among multiple schools based on academic considerations. He ultimately concludes that promoting school choice “may be a distraction from the hard work of fixing the district’s schools.” Of course, Dodenhoff’s failure to examine any data from the Milwaukee school district renders this conclusion entirely unsupported.
What if his estimate is correct?
But what if Dodenhoff were correct that only 35 percent of Milwaukee parents choose, and only 10 percent choose from among more than two schools, using academic criteria for their judgment? Would that prove that efforts to promote school choice are misguided?
Consider this thought experiment: What if only 10 percent of Americans purchased cars based on comparisons they made regarding the quality of the engineering? Would the quality of cars decline, or would the well informed consumers set the trends that others follow?
Most analysts agree that even when only a few consumers are making smart choices, those few still drive competition in the marketplace, in part because information spreads outward from the better informed to others. As education scholars Mark Schneider and Paul Teske have explained, “Competitive markets do not need all consumers to be informed—competitive pressures can result even if a relatively small subset of consumers engage in informed, self-interested search.” At least in theory, a few quality-conscious consumers can drive systemwide improvement and lead to a better matching of parents and schools, even in the absence of extensive choice activity.
Dodenhoff clearly believes that far more than 10 percent of parents need to base their school decisions on academic considerations if school choice is going to enhance school quality. In this regard, he could be right. But to know whether or not 10 percent of Milwaukee’s parents are making informed judgments, he needs to talk to a few hundred of them, randomly chosen. So far he has talked to none. That will not do.
Martin West is an assistant professor of education, political science, and public policy at Brown University and an executive editor of Education Next.
