The effectiveness of online learning in higher education is an open question: While some research finds that students who have lower GPAs or who are male, Black, or younger tend to perform relatively worse, other scholars have found that student learning can be just as successful in online classes as in in-person classes. In our recent work with James Berry, Austin Boyle, Thomas J. DiCiccio, Alex Rees-Jones, Tyler Ransom, and Jörg Stoye, we asked whether the challenge of the pandemic-induced switch to online teaching negatively affected student learning, and, if so, whether certain demographic groups were affected more than others. Further, we wanted to see if there were factors, such as particular pedagogical approaches, that could mitigate potential negative effects.
To do so, we examined students’ performance on standardized assessments, developed as part of the Active Learning Initiative at Cornell University, in seven intermediate-level economics courses at four PhD-granting institutions in semesters before and during the pandemic: Spring and Fall 2019, and Spring 2020. Not all teaching during the Spring 2020 term took place online, but each of the questions on these assessments is associated with specific course learning goals, so we were able to calculate separate assessment scores for material learned before and after the switch to remote learning.
We found that, on average, students tended to perform worse during the pandemic semester than the pre-pandemic semesters. The sub-score for the material learned during the remote portion of Spring 2020 was nearly a tenth of a standard deviation lower than this same sub-score in previous semesters. It is difficult to tell how much of this effect resulted from difficulties with remote learning and how much resulted from pandemic-induced stressors and loss of focus.
We were pleasantly surprised, however, to find little evidence of disparate effects on different demographic groups. Though we found that women and underrepresented-minority students performed worse than non-underrepresented-minority men in both semesters, this preexisting disparity was not exacerbated further during the pandemic semester. We did find that first-generation college students had a relatively larger drop in scores during the pandemic semester. However, while this decrease was statistically significant for the overall assessment score, it was not for the sub-score of the material that was covered during the remote portion of the Spring 2020 term.
Though it does not appear that impacts of the pandemic differed substantially across most demographic groups, they did differ across courses. We surveyed course instructors about the material they covered and the teaching methods they used before and during the pandemic, along with their prior experience with online teaching. We were particularly interested in teaching methods that fall into the category of “active learning,” such as regularly polling students on recently learned material or promoting peer interaction through preplanned think-pair-share or small group activities.
We found evidence that instructor experience with online teaching did make a difference. On average, students in classes taught by instructors with experience teaching online had sub-scores that were 0.625 standard deviation higher on the material covered during the remote portion of the semester than those in classes with instructors who had no online teaching experience. This finding bodes well for students now learning in remote classrooms, since most instructors have been steadily accumulating experience teaching online since the start of the pandemic.
We also found that certain active-learning practices make a difference. Specifically, students tended to weather the pandemic better in courses that promoted peer interaction through think-pair-share activities, small-group activities, preassigned small groups encouraged to work together outside of class, or collaborative exams. Students in such courses scored nearly one third of a standard deviation higher on average on the purely remote material than students in courses without such peer-interaction strategies.
An alleviation of the practical and mental difficulties caused by the pandemic appears to be on the horizon. We hope that some of the lessons learned during the pandemic, however, will stay with us and inform teaching practices in the future.
Anna McDougall is an undergraduate research assistant at the Cornell University Department of Economics, where Douglas McKee is senior lecturer and George Orlov is postdoctoral associate.
