Albert Shanker first proposed the idea of charter schools in 1988 as laboratories from which the traditional public schools could learn. So when a charter school chain like the Knowledge is Power Program (KIPP) posts extraordinary gains on student test scores for large numbers of low-income students, it’s important to know why the schools succeed so lessons can be shared with other public schools.
Several theories for KIPP’s success seem plausible. One focuses on the pedagogical approach employed. KIPP explicitly teaches middle-class habits and norms, assigns lots of homework, and boasts a longer school day, week, and year. Another explanation considers the related issue of per pupil expenditures. (Those longer school days, weeks, and years are expensive). Another possibility stresses the importance of favorable peer environments. Students in KIPP schools may be surrounded by classmates who are, on the whole, more supportive of academic achievement than peers in traditional public schools with similar poverty rates. Of course, a combination of these factors (and others) could come into play as well.
On Tuesday, Education Next published a lengthy article by a group of four highly-respected researchers from Mathematica Policy Research Inc. seeking to examine the third hypothesis — that peer influences are an important explanation of KIPP’s success. According to the Media Alert, the study knocked down this theory. “Student Achievement Gains at KIPP Schools Cannot Be Explained by Peer Effects,” the alert suggested. It continued, “Mathematica study finds students are similar to those in other local schools and most patterns of attrition are no different.”
On closer examination, however, the study was far less definitive than advertised because it addressed, at most, one of the three ways in which peer influences might make a difference in KIPP’s success.
1. Self-selection. KIPP students might enjoy a more positive peer environment because students are surrounded by classmates whose families are attracted to the challenging academic program provided by KIPP schools. Imagine two families attending a KIPP open house. When the children hear about the rigorous regimen — the extra hours of school time and considerable homework assignments – and the parents hear about the requirement that they sign a contract to help their children with homework every day, particularly motivated families might be excited to sign up, while less motivated families could be scared away.
2. Attrition. Likewise, KIPP students might leave at high rates and the weakest least-motivated students may be the most likely to leave. If so, remaining KIPP students would be surrounded over time by an even stronger set of peers than the initial cohort.
3. Replacement. Finally, where traditional public schools need to take in new students who happen to move into the neighborhood at any time, causing disruption to the norms and routines established by the teacher, KIPP students would enjoy a more positive peer environment if KIPP either limited its influx of new students or tended to take in mostly high achieving students.
What does the Mathematica study find?
On the first issue — self-selection — the study finds that KIPP students do not have higher levels of prior achievement, and are broadly similar to students in nearby public schools demographically. But the researchers lacked “direct measures of parent characteristics, prior motivation, or student behavior.” Therefore, the authors concede, “KIPP students might benefit from attending school with peers who are especially motivated to accept KIPP’s academic and behavioral demands. If this were true, the presence of motivated peers at KIPP could help bolster the effectiveness of the KIPP model…” Mathematica’s evidence is silent on this topic, which the authors concede is “a potentially important limitation.”
On the second issue – attrition – the study found high percentages of KIPP students leave (34%), but that this level is no higher than in neighboring schools. Likewise, they find the KIPP students who leave have “markedly lower” test scores, but that is also true of students who leave comparison middle schools. Apparently, mobile students are more likely to have low test scores.
But on the third issue – replacement – the study finds significant differences between KIPP schools and comparison middle schools on a couple of dimensions. KIPP schools, which begin in 5th grade, admit a considerable number of new students in grade 6 (the time when most public middle schools begin) but “the number of new enrollees declines substantially” in 7th and 8th grade. Comparison schools “are more likely than KIPP schools to replace the students who leave through attrition” in grades 7 and 8, they find. Other research, by Gary Miron, Jessica Urschel and Nicholas Saxton found that a typical KIPP grade cohort shrinks by about 30% between grades 6 to 8.
More significantly, the Mathematica researchers find that late entrants to KIPP schools have above average test scores, the opposite of late entrants to comparison middle schools, where the scores are considerably lower. The authors give no explanation for this anomaly, which seems odd since on the attrition side, movers tended to have lower test scores. Clearly, this peculiar phenomenon in KIPP schools helps create a more positive peer environment: there are fewer new peers to disrupt the school culture, and the new entrants tend to be higher achieving.
The Mathematica researchers draw on other studies to try to estimate the potential peer effects, but clearly the best way to resolve the issues of self selection, attrition and replacement would be for KIPP to run a substantial number of “conversion” charter schools – schools in which KIPP educates students who happen to live in a particular neighborhood, rather than a self-selected group of students. If students moved into the neighborhood, KIPP would have to take them in, like regular public schools do.
The one time KIPP came close to doing this, educating a regular student population at Cole College Prep in Denver, Colorado, KIPP abandoned the effort after two years. Washington Post reporter Jay Mathews, a strong supporter of KIPP, called the effort a “failure.”
Circling back to the original question – what lessons does KIPP’s success have for regular public schools – it’s important to clear up one misunderstanding. Assuming peer effects are a significant factor in KIPP’s success, this would not, in my view, discredit KIPP’s accomplishments, or diminish the lessons KIPP holds for our broader public school system.
The Mathematica researchers label me a KIPP “skeptic” for raising the issue of peer effects. But if having a favorable classmate environment were a part of KIPP’s formula for success, that finding would have powerful implications for school reform generally. If giving low-income students a positive peer environment has positive effects – as lots of other evidence suggests – then we should try to give more low-income students access to better peer environments, either in self-selected low-income environments, or, more generally, in middle-class schools. If peers matter to KIPP’s success, the favorable outcomes could be scalable through socioeconomic school integration.
As my colleague Halley Potter and I note in a forthcoming book, A Smarter Charter, a small but important number of charter schools – from High Tech High in San Diego to Community Roots in Brooklyn – are taking conscious steps to create socioeconomically and racially diverse school environments. Nationally, more than 80 public school districts are seeking to integrate their schools by considering socioeconomic status as a factor in student assignment.
KIPP appears to provide hard-working, striving low-income students with an environment in which they are surrounded by classmates who support student achievement. Acknowledging this likelihood doesn’t diminish KIPP’s success; it simply points to some important public policy solutions for public education generally.
-Richard D Kahlenberg
Richard D. Kahlenberg, a senior fellow at The Century Foundation, is coauthor, with Halley Potter, of A Smarter Charter: Finding What Works for Charter Schools and Public Education, to be published next month by Teachers College Press.
A response to this critique from the authors of the study appears here.
