The unprecedented school closures caused by the Covid-19 pandemic in spring 2020 raised many questions about how student learning would be impacted in the short and long term. NWEA, a not-for-profit research and assessment provider formerly known as the Northwest Evaluation Association, took an initial look at those impacts on student achievement and growth in a new study released today. Our study examined three questions:
- How did students perform in fall 2020 in math and reading compared to fall 2019?
- How has student academic growth changed since schools physically closed in March 2020?
- How did fall 2020 achievement compare to NWEA’s widely cited April 2020 projections of the Covid slide?
We used a sample of 4.4 million students in grades 3-8 across 46 U.S. states who took the MAP® Growth™ assessment in fall 2020. Four key findings emerged:
- In fall 2020, students in grades 3-8 performed similarly in reading to same-grade students in fall 2019, but about 5 to 10 percentile ranks lower in math.
- In almost all grades, most students made some learning gains in both reading and math since the Covid-19 pandemic started. However, gains in math were lower on average in fall 2020 than over the same time period in prior years, resulting in more students falling behind relative to their prior standing.
- In fall 2020, students scored better than NWEA’s projections in reading, while math scores were in line with our projections for grades 4-6 and slightly above our projections in grades 7-8.
- It is too early to draw definitive conclusions about how impacts differed across racial and ethnic groups. Student groups especially vulnerable to the impacts of the pandemic were more likely to be missing from our data, either because they were not enrolled in school or did not participate in the tests. Thus, we have an incomplete understanding of how achievement in fall 2020 may differ across student groups and may be underestimating the impacts of Covid-19.
These findings highlight a need to address unfinished learning, especially in math. Given the sequential nature of how math is learned, diagnosing underdeveloped skills as a result of the spring school disruptions can shed light on areas where students need targeted instruction to get on track. Teachers will need an accurate measure of where students are, along with continued support and resources, to help the teachers prioritize interventions for students and deliver high-quality instruction. In addition, parents and caregivers should engage with their children and teachers to identify where students need support and how they can practice at home.
One important consideration in interpreting these findings is that most students in our sample were assessed remotely in fall 2020. According to MCH Strategic Data, 19 percent of districts offered in-person instruction to all students in fall 2020. Many school districts had to change their re-opening plans as a result of the ongoing pandemic. To investigate concerns around data quality in remote testing settings, my colleagues and I conducted an initial comparability study of remote and in-person testing in fall 2020. We found trends in test scores for remote and in-person tests for students in grades 3-8 were very similar, as were the tests’ psychometric properties. We did find that remote testing may make more of a difference for younger students in grades K-2, but they were not part of our study.
A second consideration is the large fraction of students who tested in fall 2019 but not in fall 2020. My NWEA research colleague Angela Johnson and I conducted a full attrition analysis to examine which and how many students were missing from the fall 2020 assessments. Nearly half of the students who tested in fall 2019 did not test in fall 2020. This is a sizable population, and it means that the achievement of some of the most vulnerable students is not reflected in the data. When interpreting the study results, we must acknowledge the unaccounted-for students and proceed with caution, so we do not underestimate the impact on our most underserved populations. Educators and policymakers should plan to provide ample support to students who have fallen behind and, when in doubt, err on the side of more service and outreach.
We are only scratching the surface in measuring the academic and non-academic impacts of the Covid-19 related school disruptions. These initial research findings shed light on student achievement following the spring school closures but cannot yet unpack differences by teaching modality (say, remote versus in-person) in the 2020-21 school year. Understanding these impacts both in the short and in the long term is essential to support students’ academic and social and emotional learning.
As part of the research study, we noted several considerations and recommendations along with our findings. We see a continued need for federal and state funding to school districts impacted by the pandemic. To understand the unmet needs of children and to target resources to those who need them most, a more robust effort is necessary to collect and share key data such as students’ opportunity to learn, academic achievement, and social and emotional wellbeing.
Megan Kuhfeld is a research scientist for the Collaborative for Student Growth at NWEA and lead author of “Learning During COVID-19: Initial Findings on Students’ Reading and Math Achievement and Growth.”
