Among the regulatory freedoms available to most charter schools is the ability to restrict the entry of new students. Oversubscribed charter schools must offer seats randomly, and thus they can’t choose among applicants. But most charter schools can choose whether to “backfill” vacancies created by students who leave. Though there is no reliable estimate for the number of vacant seats in charter schools, there is ample reason to believe that decisions not to backfill substantially reduce charter school enrollments in several localities. For example, of the 53 New York City charter schools analyzed in a 2015 report by the Independent Budget Office, 20 replaced fewer than 70% of students who left.
Despite relying on per-pupil funding, many charter schools choose not to backfill because they are concerned that the disruptions caused by incorporating new students would reduce their effectiveness for the students already enrolled in the school. Many think that incorporating new students would be especially disruptive within charter schools that apply models that emphasize developing a consistent culture and norms over time. Fordham Institute President Michael Petrilli summarized this argument well in a 2015 commentary:
…there are strong instructional arguments for not backfilling. Great schools spend a lot of time building strong cultures—the almost-invisible expectations, norms, and habits that come to permeate the environment, such as the notion that it’s cool to be smart and it’s not OK to disrupt learning. Culture-building is a whole lot harder to do if a school is inducting a new group of students every year in every grade….This isn’t just a technical challenge; there’s a moral question too. Backfilling is surely good for the student who gets to claim an empty seat. But what if it’s bad for their new peers? What if the disruption to the many outweighs the benefits to the few?
Though the argument has intuitive appeal, whether backfilling enrollment harms incumbent charter school students has remained an open question. In a new paper for the Wheelock Educational Policy Center, my co-author Cheonghum Park and I fill this gap in our understanding of charter school impacts by measuring the causal effect of required backfill on the achievement of incumbent students within Boston’s charter middle schools.
Our study leverages a 2010 change in the law governing charter schools in Massachusetts. In addition to other reforms, the new law required charter schools to begin backfilling vacancies that occur within the bottom half of grades offered within the school. In practice, the change in the law didn’t impact enrollments for charter schools operating outside of Boston because they were already backfilling vacancies at very high rates. However, Boston charter schools backfilled very few vacancies prior to the change in the law, and we show that introduction of the requirement had a substantial impact on their enrollment patterns.
Boston is an especially interesting setting in which to consider the effects of backfill for two reasons. First, we know from several prior studies that Boston charter schools are highly effective. Second, many of the city’s charter schools apply the so-called “No Excuses” approach, making them precisely the type of schools that are likely to emphasize building a consistent culture of high expectations. Thus, if backfill harms charter schools by disrupting their norms, we might especially expect to see harms to incumbent charter school students in Boston.
Our analysis exploits differences in exposure to the backfill requirement occurring over time (before vs. after the change in the law), across school sectors (Boston charters vs. traditional public and non-Boston charters), and across impacted and not-impacted grades within each school. Essentially, we ask whether following the change in the law, Boston charter middle schools experienced a decline in test scores for incumbent students within grades that now required backfill relative to grades that did not require backfill, and whether this decline differed from the patterns observed in traditional public schools and non-Boston charter schools.
Figure 1 illustrates the intuition of our strategy and our key results. Each dot represents the estimate and each vertical line the 95% confidence interval for the change in the respective outcome following the adoption of the requirement in impacted and not-impacted grades for each school type. Panel A in the figure illustrates our estimate of the effect of the backfill requirement on the percentage of students within a school-grade cohort who entered the school through backfill that year. As anticipated, the blue line on the far right of Panel A shows that within Boston charter schools there is a clear jump in the percentage of backfilled students within impacted grades following the introduction of the backfill requirement, but there was no change in backfill within not-impacted grades. In contrast, within both non-Boston charter schools and traditional public schools (TPS), the change in backfill following the new law was similar in both impacted and not-impacted grades. This pattern of results implies that the new law indeed led to the intended increase in backfill within impacted grades in Boston charter middle schools.
Nonetheless, as illustrated on Panel B of the figure, Boston charter schools did not experience a similar change in incumbent students’ test scores within impacted and not-impacted grades. Indeed, though not statistically significant, the pattern is the opposite of what those concerned about deleterious effects of backfill would expect: After the requirement was introduced, test scores in Boston charter middle schools went up in grades that the schools were required to backfill relative to grades that they were not required to backfill.
Boston’s experience provides strong evidence that high-performing charter schools can continue to benefit their current students even as they incorporate new students into vacant seats. This result is at least somewhat surprising. Why wouldn’t requiring backfill impact the performance of incumbent charter school students? Indeed, prior evidence suggests that incorporating new students is harmful for incumbent students within the context of traditional public schools. Our data do not allow us to investigate the reasons underlying the pattern of results directly. However, one intriguing possibility is that perhaps rather than exacerbating the disruptive aspects of incorporating new students, their consistent culture and intense focus on student behavior make Boston’s charter schools especially adept at addressing the challenges created by new student entry.
Whether charter school proponents use our findings to support calls to require charter schools to backfill vacancies will, we expect, depend largely on how those proponents value charter school autonomy. Perhaps more importantly, our results should be of interest to the seemingly large number of charter school leaders nationwide that leave seats vacant despite a financial incentive, and some would argue moral imperative, to maximize their enrollments. Our results from Boston suggest that concern that backfilling enrollments would substantially harm charter school impacts are unwarranted.
Marcus A. Winters, Ph.D. is an associate professor and chair of the department of Educational Leadership & Policy Studies at the Boston University Wheelock College of Education and Human Development, and faculty director for the Wheelock Educational Policy Center.
