AP Photo / Tennessee Daily Life / Alan Spearman
Prior studies have shown that strong charter high schools can improve students’ college outcomes, but evidence on the long-term effects of charter middle schools has been less encouraging. Until now: we recently released the first research paper that shows charter middle schools can have a long-term impact on college outcomes. The paper describes findings from a new, nationwide randomized control trial that provides the first rigorous estimates of the long-term effects of charter middle schools affiliated with KIPP—the Knowledge is Power Program.
Research has consistently shown that large urban charter school networks such as KIPP can dramatically increase students’ test scores. But do these schools simply “teach to the test” in a manner that has few long-lasting benefits, or are they succeeding in preparing students for college and beyond? Recent studies in Boston and Chicago found that a set of charter high schools that boosted student test scores also boosted enrollment in four-year college programs. In the case of charter middle schools, however, a recent national study found no relationship between charter-school effects on middle school test scores and longer-run effects on college enrollment or college graduation. In that study, even charter schools that were successful in improving middle school test scores (including a sample of high performing urban charter schools) did not improve students’ postsecondary outcomes.
How does KIPP fit into this debate? KIPP is the nation’s largest network of public charter schools, serving more than 100,000 students—most of whom are low-income students of color—across a network of more than 240 schools. Evidence of KIPP’s impacts on student test scores is especially convincing, with multiple randomized controlled trials showing positive effects across a variety of different KIPP middle schools (see studies from 2010, 2013, and 2015). Examining the longer-run outcomes of KIPP students is of particular interest because these schools prominently emphasize preparing students for success in college: one study also found that KIPP high schools increased the degree to which students engaged in a set of college preparatory activities and behaviors, such as discussing how to pay for college and taking Advanced Placement classes. Given these promising signs, KIPP serves as an important test case for whether a high-performing charter school network can improve students’ long-term educational attainment.
Our new study tracked students who applied to 13 KIPP middle schools through an admissions lottery in 2008 or 2009. In the primary analysis, we compared college enrollment patterns of students who were offered admission to a KIPP middle school (the treatment group) with those of students who were not offered admission (the control group), using data from the National Student Clearinghouse. On average, students in the treatment group attended KIPP middle and high schools for an average of 3.5 years, and students in the control group attended KIPP schools for an average of about one year (this happened because some of the control students managed to enter KIPP at some point after the initial lottery). To account for these students, we examined both the effects of the initial offer of admission to a KIPP middle school and the effect of actually attending a KIPP school.
We found that KIPP middle schools do produce a statistically significant increase in students’ enrollment in four-year colleges. On average, students who received a lottery-based admissions offer to a KIPP middle school were 7 percentage points more likely to enroll in a four-year college than students who applied to KIPP but were not offered admission. Forty-eight percent of treatment students enrolled in a four-year college within two years of high school graduation, compared with 41 percent of control students (Figure 1).
Figure 1. Impact of KIPP middle schools on four-year college enrollment
Note: Study includes 1,177 students who applied to enter KIPP middle schools via admissions lotteries, and compares the outcomes of students offered admission to KIPP (treatment group) to those not offered admission (control group) at the time of the lottery.
Since only a subset of the students who were offered admission actually attended a KIPP school, we also estimated the impact of KIPP attendance in an exploratory analysis. Under a range of assumptions we found that attending a KIPP school led to a 10-13 percentage point improvement in the likelihood of enrolling in college. The effect of enrolling at KIPP is statistically significant, and in our view an impact of this size represents a meaningful change in college enrollment rates. In 2017, the national gap in college enrollment rates between white students and black or Hispanic students (for any college type) was approximately 14 percentage points among 20- and 21-year-olds. For a given black or Hispanic student, in other words, the impact of attending a KIPP middle school would be almost large enough to offset the nationwide racial disparity in college enrollment rates.
For the students in our study, it is too early to say whether KIPP middle schools increase college graduation rates, since we can only track most of these students for two years after high school graduation. At this point, we are limited to examining “initial persistence” in college, which we define as (a) enrolling in college immediately after high school and (b) persisting in degree programs continuously for the following two years. Early persistence rates in the treatment group (30 percent) appear higher than the control group (26 percent)—a difference of four percentage points—but thus far the effect is not statistically significant. If you adjust for which students actually attended KIPP, the positive effect estimate increases to 9 percentage points but remains statistically insignificant.
Future analyses will track student persistence in college through graduation, and also allow more time to observe any “catch up” patterns among students who enter college after a delay. Examining rates of college graduation will be important, since disadvantaged students who enroll in college often have low rates of college completion, and dropping out of college is likely to be especially harmful for students who have accumulated substantial debt while in school. Recognizing that their former students face these challenges, KIPP has developed a “KIPP through College” program to provide former students with supports after they have entered college. While the program was still in its early stages of development at the time of college-entry for the cohorts of KIPP students we studied, future analyses of the impacts of KIPP on college graduation will begin to provide evidence on whether these alumni-support initiatives at KIPP have been successful.
Although some questions about the long-term outcomes of KIPP students will take more time to answer, our core finding remains clear. KIPP middle schools improve students’ initial achievement outcomes, and these schools also go on to boost enrollment rates in four-year college programs. This pattern is noteworthy because we know from other rigorous studies of (non-KIPP) charter schools that initial achievement improvements are not guaranteed to lead to longer-run gains in college enrollment or college graduation. Which practices within the KIPP network are producing these long-run outcomes, and how does KIPP differ from other “high performing” charter middle schools that are not translating achievement gains into college attainment? The results of that line of research could yield important lessons for any school network—including traditional public schools and schools of choice—that is seeking to prepare students for college and beyond.
Ira Nichols-Barrer is a senior researcher at Mathematica. Philip Gleason is senior fellow and associate director of human services research at Mathematica. Thomas Coen is director of performance management and evaluation at Social Finance.
