Student performance in the nation’s capital has increased so dramatically that it has attracted significant attention and prompted many to ask whether gentrification, rather than an improvement in school quality, is behind the higher scores. Our new analysis shows that demographic change explains some, but by no means all, of the increase in scores.
The National Assessment of Educational Progress (NAEP), known as the “nation’s report card,” tests representative samples of fourth- and eighth-grade students in mathematics and reading every two years. NAEP scores reflect not just school quality, but also the characteristics of the students taking the test. For example, the difference in scores between Massachusetts and Mississippi reflects both the impact of the state’s schools and differences in state poverty rates and other demographics. Likewise, changes in NAEP performance over time can result from changes in both school quality and student demographics.
The question, then, is whether DC’s sizable improvement is the result of changing demographics, as some commentators claim, or improving quality. Our analysis of student-level NAEP data from DC, including students from charter and traditional public schools, compares the increase in scores from 2005 to 2013 with the increase that might have been expected based on shifts in demographic factors including race/ethnicity, gender, age, and language spoken at home. The methodology is similar to the one used in a new online tool showing state NAEP performance (the tool excludes DC because it is not a state).
It is true that the demographics of DC have changed substantially over this time period. The NAEP data show that the proportion of white and Hispanic students in DC has roughly doubled, while the proportion of black students has declined. Our analysis indicates that, based on the relationships between demographics and NAEP scores in 2005, demographic changes predict a score increase of four to six points between 2005 and 2013 (the data needed to perform this analysis on the 2015 results are not yet available).
But the actual score increases have generally far outpaced the gains predicted by demographic change alone. For example, in fourth-grade math, demographics predicted a four-point increase, but scores increased 17 points. The figure below shows predicted and actual score increases for all four tests for DC schools overall (including charters) and the traditional school district (DC Public Schools). Only in eighth-grade reading scores at DC Public Schools do demographic shifts explain more than half of the score increase.
To be sure, our analysis does not account for all potentially important demographic factors. In particular, we do not include any measures of family income. Though researchers often use eligibility for free or reduced-price lunch as a proxy for determining income level, changes to who is eligible make this measure unreliable. With the eighth-grade scores, we did attempt to use parents’ education as a proxy for socioeconomic status, but our results did not appreciably change: changes in demographics still did not account for changes in academic performance.
The bottom line is that gentrification alone cannot explain why student scores improved in Washington, DC, a conclusion that echoes previous analyses using publicly available data. DC education saw many changes over this period, including reform-oriented chancellors, mayoral control, and a rapidly expanding charter sector, but we cannot identify which policy changes, if any, produced these results.
And despite the large gains, DC NAEP scores still reveal substantial achievement gaps—for example, the gap between average scores for black and white students was 56 points in 2015; the gap between Hispanics and whites was 49 points.
In other words, much work remains to be done.
—Kristen Blagg and Matthew M. Chingos
This post originally appeared on Urban Wire.
