The closure of nearly every public school in the United States last spring, in the early weeks of the Covid-19 pandemic, was clearly going to disrupt student learning. However, despite speculation about the likely magnitude and distribution of these impacts, we still know very little about the actual impact of school closures and other Covid-related disruptions.
A number of private testing companies that provide diagnostic assessments to school districts have released their own estimates of learning losses, but these have a number of limitations. Experts have raised concerns about possible cheating when children take the exams from home, and a substantial number of students—disproportionately disadvantaged—did not take them at all this past fall. These limitations imply that available estimates may understate Covid-related learning losses.
In a new report, we analyze results from the fall administration of Ohio’s Third-Grade English Language Arts (ELA) assessment to address these limitations and document how the pandemic has affected student learning in our state. Unlike most state exams, which usually take place in the spring, this test is given several times per year and thus provides an early glimpse of how Ohio students are faring. Despite the disruptions caused by Covid-19, more than 80 percent of current Ohio third graders showed up to take the test, which districts administered on site and under the supervision of proctors in late October and early November of 2020. To (partially) address the problem of missing test scores for the remaining 20 percent of students, we used detailed student records—including information about their performance on reading diagnostics administered one year prior, when students were in second grade—to impute missing scores and to control for differences between students tested this past fall and those tested in the fall of 2019.
The results show that the achievement of third-graders declined significantly between fall 2019 and fall 2020. Widely used national benchmarks suggest that reading scores typically grow by about 0.6 standard deviations between second and third grade. We estimate that Ohio scores were down 0.228 standard deviations on average this year, suggesting that the achievement decline equates to about one third of a year’s worth of learning. The decline was more pronounced among economically disadvantaged children, and it was especially large for Black students. Indeed, Black students’ scores fell by 50 percent more than those of their White classmates–approximately half a year’s worth of learning.
Changes in Ohio third-grade ELA scores from fall 2019 to fall 2020, by race
Note: These achievement effects are calculated by imputing missing scaled scores and estimating regressions that control for students’ characteristics when they were in second grade.
Ohio Governor Mike DeWine let individual school districts decide how to reopen this past fall, with about one third of students coming back for in-person instruction, one third participating in some form of “hybrid” instruction, and one third participating in fully remote instruction. Declines in achievement were 50 percent greater among students whose districts started the year with fully remote instruction, as compared to students in districts with fully in-person learning. Achievement declines for students in districts with hybrid instruction was halfway between the two. These differences across modes of learning remain even after accounting for Covid health impacts (number of positive tests, hospitalization rate, and deaths per 100,000 population for the county in which each district is located) and unemployment shocks.
Changes in Ohio third-grade ELA scores from fall 2019 to fall 2020, by mode of instruction
Note: These achievement effects are calculated by imputing missing scaled scores and estimating regressions that control for students’ characteristics when they were in second grade. Mode of instruction in fall 2020 is determined based on weekly district reports for the period Sept. 10 through Nov. 5. Some districts with fully in-person instruction still provided virtual learning options for families.
It is important to reiterate that school closure is not the only reason student achievement took a hit during the pandemic. In addition to the challenge of adjusting to new and unfamiliar modes of instruction, some children also had loved ones fall seriously ill or even die due to complications from the virus. Many families have also faced financial hardships as unemployment reached record levels. Indeed, we find evidence that rising unemployment contributed to the drop in student performance, with larger test score declines in the parts of the state that experienced the sharpest job losses. Overall, our preliminary calculations show that test scores declined by 0.02 standard deviations for every percentage point increase in unemployment in the county where students resided, suggest that Covid-related unemployment explains approximately one-third of the decrease in average test scores statewide.
Our estimates are limited to just one grade and subject, so we caution against extrapolating our findings too broadly. Nevertheless, the differences in impacts across student groups is undeniable. Research suggests that racial achievement gaps, after narrowing in the 1970s and 1980s, have plateaued in recent decades. Covid appears to be expanding these gaps, reversing our progress.
National political leaders have already invested many billions of dollars to help local school districts purchase personal protective equipment and implement mitigation strategies needed to reopen schools. But bringing kids back to the classroom full-time is only the first step. It is important that we begin planning remediation efforts to help students make up lost ground, with special emphasis on student subgroups and schools that the pandemic affected the most.
Vladimir Kogan is associate professor in The Ohio State University’s Department of Political Science and (by courtesy) the John Glenn College of Public Affairs. Stéphane Lavertu is professor in The Ohio State University’s John Glenn College of Public Affairs..
