Compared to most education interventions, charter schools have unusually good research evidence on impacts on participating students. Researchers have been able to take advantage of natural experiments that are created by the admissions lotteries of oversubscribed charter schools. The lottery losers represent an ideal comparison group, because they are similar to the winners in all respects except the luck of the admissions draw. Studies relying on charter-school lotteries have not found uniformly positive effects, but they have identified particular charter-school operators (such as KIPP) and particular cities (such as Boston) where effects on charter students are unambiguously favorable.
For the policy debate, however, lottery-based studies of the impacts of charter schools on their students tell only half the story. One of the central arguments about charter schools (and other forms of school choice) is about indirect effects on students who remain in district-operated public schools. Indeed, the opponents of next month’s Massachusetts ballot initiative that would raise the current cap on the state’s charter schools have made clear that they are most worried about indirect effects that could result from the transfer of funds from district schools. Similarly, recent statements opposing the expansion of charter schools from the NAACP and the Movement for Black Lives have prominently cited concerns about charter schools’ indirect effects.
In contrast, proponents of charter schools have hoped that they will positively affect students in district-operated schools, by creating innovative approaches that district schools can borrow, and by producing healthy competitive pressure on district schools that would otherwise hold a local monopoly. In Massachusetts, for example, supporters of removing the charter-school cap might point out that the growth in charter schools in Boston has been accompanied by improving student outcomes in Boston’s district-operated schools, as measured by rising scores on the National Assessment of Educational Progress and graduation rates that have reached all-time highs.
In short, there’s an empirical disagreement about whether charter schools will have positive or negative indirect effects on students in district public schools. Unfortunately, the charter-school admission lotteries can’t help us identify indirect effects, which are methodologically much more difficult to measure than are the direct effects. Short of randomizing charter schools to different communities, no study of indirect effects will ever have the level of internal validity that is possible in a lottery-based study of direct effects. Even so, it is possible to conduct a careful non-experimental study of indirect effects, relying on longitudinal data on individual students and observing changes in student outcomes in district schools following the establishment or growth of charter schools nearby. As charter schools have grown in communities across the country over the last two decades, 11 different studies have done exactly that.
My colleague Kevin Booker and I reviewed these studies in writing the entry on the competitive effects of school choice for the latest edition of the Handbook of Research in Education Finance and Policy. [1] Dennis Epple, Richard Romano, and Ron Zimmer did a similar review in a recent paper, reaching conclusions similar to ours.
Collectively, the 11 studies examined effects in 11 different cities and states plus one nationwide sample. One study included separate assessments in several different communities, and a few communities were studied more than once. In the table below, I’ve sorted the findings of the 11 studies based on the direction of findings, separating the results by location for the study that included analyses for multiple locations.
As the table indicates, the literature provides some support for the “healthy competition” hypothesis and almost none for the hypothesis that students in district schools are harmed by the growth of charters. Six studies found some evidence of positive effects, four found no effects, and one found negative effects. Breaking the results out by locations, in six cases that encompass five cities and states, there is evidence that charter schools produce (small) positive effects on the achievement of students in nearby public schools. In nine other cases, encompassing eight cities and states and one nationwide sample, charter schools have been found to have no effect on students in nearby district schools, positive or negative. The literature has only a single case—involving a single school district—in which charter schools have been found to have negative effects on the achievement of students in nearby district schools.
| Consistently positive | Mixed positive and neutral |
Neutral | Mixed neutral and negative |
Consistently negative |
| Texas (Booker et al 2008) | Florida (Sass 2006) | California (Buddin and Zimmer 2004) | Unnamed district in the Southwest (Imberman 2011) | |
| North Carolina (Jinnai 2013) | New York City (Winters 2012) | North Carolina (Bifulco & Ladd 2005) | ||
| New York City (Cordes 2016) | Milwaukee (Nisar 2012) | Chicago (Zimmer et al 2009) | ||
| Milwaukee (Zimmer et al 2009) | ||||
| Philadelphia (Zimmer et al 2009) | ||||
| Denver (Zimmer et al 2009) | ||||
| San Diego (Zimmer et al 2009 | ||||
| Ohio (Zimmer et al 2009( | ||||
| Nationwide (Davis 2013) |
The 11 studies vary somewhat in methodological approaches, and the particular methodology may affect the results. As noted above, none of them are as rigorous as the randomized experimental studies of charter schools’ direct effects, and researchers may reasonably disagree about how much stock to put in their findings, or which specific non-experimental approach is best. Collectively, however, they represent the best available evidence for moving the debate beyond the indirect effects of charter schools on institutions, toward a discussion of the effects on students in those institutions. And collectively they suggest that the opponents of charter-school expansion may be overly pessimistic about the indirect effects on students. It appears that charter schools need not represent a zero-sum game. Instead, most of the research implies that charter schools do no harm to students in district schools, and may even promote improved outcomes for all students.
—Brian Gill
Brian Gill is a senior fellow at Mathematica Policy Research in Cambridge, Massachusetts. A shorter essay on this topic was previously published in Commonwealth.
Notes:
[1] There were 10 studies at the time we wrote the entry in the Handbook. The 11th study, which I’ve added here, appeared this year, after the Handbook was published.
References:
Bifulco, Robert and Helen F. Ladd. 2006. “The Impact of Charter Schools on Student Achievement: Evi- dence from North Carolina.” Journal of Education Finance and Policy 1, Winter: 50–90.
Booker, Kevin, Scott Gilpatric, Timothy Gronberg, and Dennis Jansen. 2008. “The Effect of Charter Schools on Traditional Public School Students in Texas: Are Children Who Stay Behind Left Behind?” Journal of Urban Economics 64: 123-145.
Buddin, Richard, and Ron Zimmer.2005. “Is Charter School Competition in California Improving the Performance of Traditional Public Schools?” Santa Monica, CA: RAND Corporation, WR–297–EDU, 2005. Online only: http://www.rand.org/publications/WR/WR297/.
Cordes, Sarah. 2016. “In Pursuit of the Common Good: The Spillover Effects of Charter Schools on Public School Students in New York City.” Working paper, Temple University.
Davis, Tomeka M. 2013. “Charter School Competition, Organization, and Achievement in Traditional Public Schools.” Education Policy Analysis Archives 88: 1-29.
Imberman, Scott A. 2011. “The Effect of Charter Schools on Achievement and Behavior of Public School Students.” Journal of Public Economics 95: 850-863.
Jinnai, Yusuke. 2013. “The Impact of Charter Schools’ Entry on Traditional Public Schools: New Evidence from North Carolina.” Rochester, NY: University of Rochester.
Nisar, Hiren. 2012. “Heterogeneous Competitive Effects of Charter Schools in Milwaukee.” NCSPE working paper.
Sass, Tim R. 2006. “Charter Schools and Student Achievement in Florida.” Education Finance and Policy 1: 91-122.
Winters, Marcus A 2012. “Measuring the Competitive Effect of Charter Schools on Public School Student Achievement in an Urban Environment: Evidence from New York City.” Economics of Education Review, 31(2), pp: 293-301.
Zimmer, Ron, Brian Gill, T. Kevin Booker, Stephane Lavertu, Tim R. Sass, and John Witte. 2009. “Charter Schools in Eight States: Effects on Achievement, Attainment, Integration, and Competition.” Santa Monica, CA: RAND.
