This article is part of a new Education Next series commemorating the 50th anniversary of James S. Coleman’s groundbreaking report, “Equality of Educational Opportunity.” The full series will appear in the Spring 2016 issue of Education Next.
The study for which James S. Coleman is best known today makes no mention of private education. The 1966 “Equality of Educational Opportunity” (EEO) study—better known as the Coleman Report—focused exclusively on the distribution of resources and student achievement in America’s public schools.
But the report’s ink was barely dry before Coleman injected the issue of school choice into the discussion. “The public educational system is a monopoly,” he wrote in 1967, offering choice only to “those who [can] afford to buy education outside the public schools” and thereby amplifying the influence of family background on student achievement. Later, he amended that observation, noting that the opportunity to choose one’s residence permits school choice within the public sector as well. But in reality, only the middle class and the affluent can fully exercise that choice, he pointed out. “Public schools are no longer a ‘common’ institution,” Coleman wrote. “Residential mobility has brought about a high degree of racial segregation in education, as well as segregation by income … and it is the disadvantaged who are least able to select a school … that continues to function reasonably well.”
With such concerns in mind, Coleman jumped at the opportunity when the U.S. Department of Education in 1979 asked him to lead another national survey of American students, known as “High School and Beyond,” that would follow young people as they progressed from 10th to 12th grade and on into college. Unlike the EEO study, “High School and Beyond” was to include both public and private schools. The study team looked closely at Catholic schools, since Coleman deemed the sample of non-Catholic private schools too unrepresentative to warrant close analysis. They reported that students in Catholic high schools both learned more and had higher graduation rates than their public-school peers. Minority students in particular appeared to benefit from the Catholic school experience.
Both Coleman’s methodological approach and his conclusions about the superior effectiveness of Catholic schools sparked controversy at the time and remain contentious today. Yet Coleman’s work triggered an avalanche of research comparing the success of public, private, and (later) public charter schools in preparing students for college and adulthood. The best of this work has taken advantage of the lottery-based admissions processes used by many school-choice programs, enabling researchers to draw far stronger conclusions about how schools affect student outcomes than the methods Coleman employed, which relied on simple regression techniques to adjust for differences in students’ family background. By comparing students who won the opportunity to attend a school of choice to applicants who missed out, scholars have provided experimental evidence roughly akin to that generated by the randomized clinical trials used in medical research.
This research does not show that private or charter schools are always more effective than district schools in raising student performance on standardized tests—the indicator that is often put forth as a measure of a school’s success. In fact, there is little evidence that using a voucher to enroll in a private school improves student test scores, and any differences in the average performance of charter and traditional public schools by this metric are modest relative to the amount of variation in performance within the charter sector.
The modern literature on school choice does, however, confirm two promising patterns that Coleman was the first to document: First, the benefits of attending a private school are greatest for outcomes other than test scores—in particular, the likelihood that a student will graduate from high school and enroll in college. Second, attending a school of choice, whether private or charter, is especially beneficial for minority students living in urban areas. These findings support the case for continued expansion of school choice, especially in our major cities. They also raise important questions about the government’s reliance on standardized test results as a guide for regulating the options available to families.
Coleman’s untimely death in 1995 kept him from witnessing the developments that brought school choice out of the realm of academic theorizing and to the forefront of efforts to equalize opportunity for American students. His pioneering research, however, spurred the development of an evidence base that has enlarged our understanding of how school choice is changing American education.
A Catholic School Effect?
Even before Coleman embarked on “High School and Beyond,” the role of private schools had become a salient topic in federal policy debates. Enrollment in urban Catholic schools, originally designed for an immigrant population, was falling, as Catholic families moved from central cities to the suburbs (see Figure 1). This decline prompted leading members of Congress to propose a federal tax credit for private school tuition. Meanwhile, evangelical private schools in many southern states were attracting more students, as white families fled desegregated public systems. That trend invited a proposal by the Internal Revenue Service in 1979 to require that private schools meet strict racial-enrollment requirements in order to maintain their tax-exempt status. Data from “High School and Beyond” were expected to shed light on these and other hotly contested questions.
The study surveyed some 70,000 students in more than 1,000 public and private high schools in the spring of 1980. Coleman and a team of graduate students, including Sally Kilgore and Thomas Hoffer, issued their initial findings the next year. Many non-Catholic private schools had refused to participate in the study, so the researchers focused on Catholic schools, which at the time still represented more than 60 percent of private school enrollment.
Their findings stunned education researchers and the public. The Coleman team reported that sophomores and seniors at Catholic schools outperformed their public-school peers by roughly a full grade level after adjusting for differences in an extensive set of family background measures. They also found that achievement gaps along lines of parental education, race, and ethnicity were all smaller in Catholic schools than in public schools. In other words, they ironically concluded, “Catholic schools more nearly approximate the ‘common school’ ideal of American education than do public schools.”
The findings came as no surprise to Coleman, but they sent shock waves through the broader education-research community. As Paul E. Peterson, Harvard professor of government, recounts, “It was about as dramatic as the first proof of Einstein’s theory of relativity.” The tacit assumption in the field was that most families enrolled their children in Catholic schools out of religious conviction, often at the expense of their academic success. Coleman and his colleagues were subjected to blistering attacks in both academic journals and the popular press, many of them questioning whether the analysis had adequately controlled for the self-selection of students from more-advantaged families into the tuition-charging Catholic sector.
The second round of data from “High School and Beyond” enabled Coleman to provide a partial response to his critics. Rather than merely studying the level at which sophomores were achieving, he could now track the growth in their achievement between 10th and 12th grade—effectively using initial performance levels to better capture differences in family background between Catholic and public school students.
Results based on this second wave of data, published with Thomas Hoffer in 1987, seemed to confirm Coleman’s prior findings about Catholic schools’ success in boosting the achievement of minority students. (Any test-score gains for white students were modest at best.) More important, the results showed that students in Catholic schools were far less likely to drop out of school before graduating, and these positive effects were again more pronounced for black and Hispanic students. Coleman and Hoffer showed that Catholic schools had stronger disciplinary standards than public schools and that their students were more likely to take advanced courses. But above all, they attributed the success of Catholic schools to the high levels of “social capital” available in the tight-knit communities that many of these schools served.
How have Coleman’s conclusions about Catholic high schools held up over time? Dozens of studies have sought to confirm or refute them, using either the original “High School and Beyond” data or subsequent federal surveys tracking more-recent student cohorts. The most widely cited among them is a 2005 paper by a trio of scholars who employed advanced econometric techniques to analyze data from a national survey of the graduating class of 1992. They concluded that Catholic high schools “substantially increase the probability of graduating from high school and, more tentatively, attending college” but found “little evidence of an effect on test scores.” Like virtually all prior work, their results suggested that the effect on educational attainment is largest for urban minority students.
Experimenting with Vouchers
Of all the critiques of Coleman’s research on Catholic schools, one was particularly cogent. Studying how Catholic schools affect students who already attend is an imperfect way to learn about what would happen if the government were to expand access to private schools through vouchers or tuition tax credits. Students who are able to enroll in private schools only with government support could well differ from tuition-paying students in ways that make them less likely to reap the same benefits. And nonreligious private schools might not be able to draw on the community sources of social capital that Coleman deemed vital to Catholic high schools’ success. Only once policymakers began experimenting with school vouchers in the 1990s did it become possible to study the consequences of expanding access to private schools to families who could not afford that option on their own. As of 2014, 19 states operated one or more school-voucher programs, enabling some 140,000 students to attend private schools with government support.
In an authoritative 2015 review of the literature on school vouchers, economist Dennis Epple and colleagues concluded that “the evidence does not suggest that awarding students a voucher is a systematically reliable way to improve their educational outcomes.” But the researchers tempered this negative conclusion, saying, “There is also evidence that in some settings, or for some subgroups or for specific outcomes, vouchers can have substantial positive effects.”
The federal government’s official evaluation of the Washington, D.C., Opportunity Scholarship Program points to a common pattern that has emerged amid these mixed results. Using an experimental design, the study found no clear effects of using a voucher to enroll in a private school on students’ test scores four years later. Yet the evaluation also found that using a voucher improved students’ chances of graduating by as much as 21 percentage points.
Other studies corroborate these positive effects on educational attainment. For instance, a 2015 study of a privately funded voucher program in New York City found that being offered a voucher to attend a private school increased college enrollment rates among black and Hispanic students by 4.4 percentage points, a 10 percent gain relative to the control group, and also increased bachelor’s degree completion rates among black and Hispanic students by 2.4 percentage points, a 27 percent gain. As in the case of the Washington, D.C., program, however, the students did not experience test-score gains as a result of the program.
The latest research on the nation’s longest-running school-voucher initiative, the Milwaukee Parental Choice Program, although based on nonexperimental methods, shows a similar pattern of results. A 2013 study found that students using vouchers to attend private schools, 70 percent of whom were black, were 5 percent more likely to enroll in a four-year college after graduating than were a carefully matched sample of students in Milwaukee public schools. The same program in recent years has generally not shown positive effects on students’ test scores.
What explains the positive impacts of private schools on the amount of schooling students complete, even in the absence of test-score gains? Many private schools do lay claim to a broader range of educational goals than do their public-sector counterparts. Schools affiliated with Notre Dame’s Alliance for Catholic Education, for example, aim to prepare students for “college and heaven.” It has been difficult to study the effects of private school attendance on indicators of students’ character development, such as their behavior in school, owing to differences in disciplinary norms between sectors. However, a study by economists David Figlio and Jens Ludwig shows that students who attended a Catholic high school in the early 1990s were less likely to be arrested and to engage in risky behaviors, including teen sexual activity and cocaine use. Thomas Dee has found that students who attended Catholic high schools are more likely to vote as adults. And in numerous experimental studies, voucher parents express far more satisfaction with their child’s education than do their public-school counterparts—particularly in areas such as discipline and safety.
Empirical research of the kind Coleman initiated will never be able to offer a comprehensive assessment of how private school attendance, with or without the assistance of a voucher, affects students’ development across all relevant dimensions. Indeed, the fact that families differ in the weight they place on different educational goals is a key rationale for policies that expand parental choice. What is clear, however, is that both Catholic schools and voucher programs for low-income families show stronger effects on students’ educational attainment than on their achievement as measured by standardized tests. Research that focuses solely on the latter is likely to understate the benefits conferred by schools of choice.
Lotteries for Charter Schools
In recent years, most of the growth in school choice has come through the charter sector. Forty-three states have laws permitting charter schools, which now enroll roughly 5 percent of American students. Charter schools are privately managed and typically enjoy more autonomy than their district-run counterparts. Unlike most private schools, however, they are required to participate in state testing systems and can be closed by their authorizers if they fail to meet performance goals.
How does the performance of charter schools compare to that of the traditional public schools their students would otherwise attend? Again, the best evidence on charter school performance comes from studies exploiting the lottery-based admissions processes of schools that are oversubscribed. As in the case of private schools, the only short answer is: it depends.
Consider a series of recent studies of Massachusetts charter schools conducted by researchers at Harvard, the University of Michigan, and MIT. They demonstrate that attending an oversubscribed charter middle or high school has a clear positive effect on students’ math and reading achievement, but also find that this “on-average” result obscures dramatic variation. In Boston and the state’s other urban centers, each year of attendance at an oversubscribed charter middle school increases students’ achievement by roughly 15 percent of a standard deviation in reading and 32 percent of a standard deviation in math. The latter result is large enough to close more than two-thirds of the black-white achievement gap in the state while students are in middle school. In contrast, attending a charter middle school in a suburban or rural area lowers students’ achievement in both reading and math—despite the fact that these schools are popular enough to hold admissions lotteries (see Figure 2).
This pattern of test-score effects—showing positive results in urban areas with many low-income students, but neutral or even negative effects elsewhere—also appears in a national study of oversubscribed charter middle schools funded by the U.S. Department of Education. Why the drastic differences? Urban charter schools are more likely to take a “no-excuses” approach that features high expectations, strict student discipline, and longer school days and years—a formula that seems to offer a reliable path to higher test scores. In nonurban areas, where many students achieve at reasonably high levels even without a charter school option, parents may not be looking for this approach. Indeed, many nonurban charter schools have a distinctive curricular emphasis, such as a focus on the arts, that may explain their sustained popularity despite a lack of success in improving test scores.
Of course, the lottery-based approach to studying charter schools has a significant drawback: it can only be used where schools are oversubscribed, and focusing only on the most popular charter schools may well provide too rosy a view of the sector’s performance. But a large body of evidence based on nonexperimental methods paints a consistent picture of the effects of charter schools. A comprehensive 2013 study of charter schools in 27 states conducted by Stanford University’s Center for Research on Education Outcomes (CREDO) found that charter schools, on average, have no effect on students’ math achievement and only a small positive effect in reading. Yet this same study showed more-substantial positive effects in both subjects for low-income black and Hispanic students. And a separate CREDO study that focused on charter schools in 41 urban areas with a large charter presence found that, on average, charter schools in these cities have strong positive effects on achievement that are again largest for low-income black and Hispanic students (see Figure 3). In short, when it comes to improvements in student test scores, the benefits of charter school attendance are clearly greatest for low-income urban minority students.
Nascent research on the effects of urban charter schools on other outcomes also shows promising results. Students at the same Boston charter high schools that have boosted test scores are also more likely to take and pass Advanced Placement courses and to enroll in a four-year rather than a two-year college. In a study of the Promise Academy middle school in the Harlem Children’s Zone, economists Will Dobbie and Roland Fryer found that attending the oversubscribed school sharply reduced the chances of teenage pregnancy (for girls) and incarceration (for boys). Promise Academy students were also more likely to enroll in college immediately after high school and to choose a four-year school. These effects are all larger than what would have been predicted based on the same students’ test-score gains, leading the researchers to conclude that “high achieving charter schools alter more than cognitive ability.” Finally, the only study to have estimated the effect of charter school attendance on students’ job prospects, although based on nonexperimental methods, finds that attending a Florida charter school increased students’ earnings as adults despite having no impact on their standardized test scores. Once again, the broader lens offers the more favorable view of school choice.
Choice among Choice
As the evidence on school choice continues to grow, it is tempting to compare the results achieved by school voucher programs to those of charter schools—to ask whether one option or the other represents a more promising avenue for expanding educational opportunity. School reform advocates and policymakers need to decide where to invest their energies, and the charter sector’s growth does appear to have played a role in the recent decline in private school enrollment. Most obviously, some Catholic schools have elected to convert themselves into charters, sacrificing their religious identity (at least during school hours) to gain access to a stable funding stream.
In my view, the available research does not point to a clear winner. Indeed, the findings show striking similarities between the results of the two approaches. The chief beneficiaries of policies that expand parental choice appear to be urban minority students—precisely the group that Coleman argued has the least choice in a public school system in which school assignment depends on where a family lives. And the benefits of school choice for these students extend beyond what tests can measure.
Policymakers continue to wrestle with the question of how best to regulate systems of school choice. In recent years, charter school authorizers in some cities have taken on a more active role in managing the options available to families—closing some charter schools and allowing others to expand, using student test results as the primary yardstick of success. Meanwhile, some states have required private schools accepting voucher students to participate in state testing systems, blurring what had been a distinction between the two approaches. These efforts aim to produce more consistent quality among both charter and private schools and to equip parents with information to make sound decisions regarding their child’s schooling. Yet such measures, when used to limit the options available to families, assume that overall test score results at a particular school can accurately indicate the long-term benefits for an individual child of attending that school. Increasingly, researchers are casting doubt on that assumption.
Test scores are strong predictors of a student’s success in college and the labor market, and ensuring transparency about how students in schools of choice are faring academically is essential. Schools that consistently fail to equip many of their students with basic skills should not receive public funding. But policymakers should keep in mind that parents may be the best judges of whether a specific school is a good fit for their child—an environment that will keep the student engaged through graduation and increase the likelihood that he or she will attend college.
The question of how we can most effectively broaden school choice for America’s families lies at the heart of the movement to reform K–12 education. Only continued experimentation with multiple approaches will make it possible to inform this debate with evidence. Were he still alive, James S. Coleman would surely be leading the effort to do just that.
Martin R. West is associate professor of education at the Harvard Graduate School of Education and executive editor of Education Next.
This article appeared in the Spring 2016 issue of Education Next. Suggested citation format:
West, M.R. (2016). Schools of Choice: Expanding opportunity for urban minority students. Education Next, 16(2), 46-54.
