Online courses have the potential to improve instruction at every level of education. Adaptive online courses can allow students to learn at their own pace, with material adjusting to fit the needs of both advanced and remedial learners. Online courses can also open up more curricular offerings in schools that lack specialists, such as those in rural areas.
Online courses are particularly attractive to school and district leaders looking for ways to trim costs. Teacher salaries are the key driver of instructional costs at every level of education, so any technology that allows a teacher to instruct more students can free up funds that can be used for other purposes. Whether cost savings are realized depends on production costs, which can be high for universities that are rolling out their own digital content. [1]
Are online courses fulfilling their promise? In a June 2017 Evidence Speaks post, Eric Bettinger and Susanna Loeb of Stanford University showed that in a large, for-profit college, online courses are a poor option for the least prepared students. [2] Online students did substantially worse than students in the same face-to-face course: They earned lower grades, were less likely to succeed in subsequent courses, and more likely to drop out.
This is particularly bad news because students who enroll in online classes tend to, at the outset, face more challenges than their face-to-face peers: they are older, more likely to work full-time, and more likely to be a single parent. [3] While online courses are certainly convenient for such non-traditional students, the existing evidence suggests they are a poor fit for those who are academically behind their peers.
One study cannot tell a complete story of online learning. Does this finding replicate in other postsecondary settings? And, what are its effects among younger students, such as adolescents in middle and high school?
Maya Escueta and colleagues have recently completed a thorough review of studies that measure the causal impact of online courses on both learning and course access. [4] The review focuses on papers that make use of randomized trials and regression discontinuity designs, two approaches that let us be particularly sure that we are nailing the causal effect of the online “treatment.” While there are still questions to be answered and studies to be done, the review shows progress in our understanding of which students benefit from online courses.
Online learning comes in two broad categories: purely online courses, in which a student is never in the same room as an instructor, and “blended courses,” in which students spend time in a physical classroom with an instructor, and, also, time online with instructional videos and digital content.
Overall, the body of research suggests that learning suffers with no face-to-face instruction. Students in blended courses appear to do about the same as those in fully face-to-face courses. If a blended course frees up teachers’ time, that time can be transferred to additional courses, or to extra attention to students who are struggling.
The research discussed above takes place mostly in colleges. A pair of recent studies has examined the effect of online education among middle- and high-school students. The pattern of effects in these papers echoes that of the postsecondary findings. Both papers examine the effect of an online algebra class, but to very different populations.
The first paper evaluated a program to expand access to algebra courses among middle-school students in Maine and Vermont. [5] In small, rural schools, there are relatively few specialized courses for students who are working ahead of their grade. The economics of a small schools simply cannot support the range of specialized teachers that a large, urban middle school can. The online format can therefore open up curricular options that otherwise would be denied to students.
In the evaluation, eighth graders whose academic performance made them eligible for Algebra I and who were attending schools that did not offer an eighth-grade algebra course were randomly selected and given that course in an online format. The students in the control group took their schools’ standard, face-to-face general math course in eighth-grade.
The students taking the course online did substantially better on assessments of algebra knowledge at the end of eighth grade, scoring 0.4 standard deviations higher than students in the control group. This is a substantial effect, especially for a one-year intervention. The treated students were also twice as likely to complete advanced math courses in high school, competing at least Algebra II by tenth grade (26 percent in the control group vs. 51 percent in the treatment group).
Note that this study tested a mixed treatment: exposure to Algebra I in eighth grade and enrollment in an online course, relative to exposure to general math in eighth grade in a face-to-face course. We can’t tell which of these aspects of the treatment is producing the effect we observe. Treated students may have learned even more had they learned algebra in a face-to-face course, rather than online. From a scientific perspective, the findings are therefore a bit unsatisfying: we can’t separate these two channels of the treatment’s effect. From a policy perspective, however, the findings are quite satisfying: online math courses can provide a productive learning experience for academically proficient adolescents in eighth grade who otherwise would not have access in that grade to that content.
The second study also tested the effect of online algebra, but on a very different population: high school students in Chicago who had already failed a face-to-face version of this math class. [6] Online platforms are increasingly used for such “credit recovery,” in which students repeat failed coursework.
In seventeen Chicago high schools, students who had failed algebra were enrolled in a summer recovery course. Once they had showed up for a few classes, they were randomly assigned to an online or face-to-face format. In this case, students in the online courses did substantially worse in end-of-course tests, scoring 0.2 standard deviations lower than students in the face-to-face classes. The online students were substantially less likely to pass the course: 66 percent vs. 78 percent.
Two randomized trials of online coursework among adolescents are not enough to set policy. But in combination with the postsecondary studies, a clear pattern emerges: academically challenged students do worse in online than in face-to-face courses. The existing evidence suggests that online coursework should be focused on expanding course options or providing acceleration for students who are academically prepared, rather than shoring up the performance of those who are lagging.
— Susan Dynarski
Susan Dynarski is a professor of public policy, education and economics at the University of Michigan, where she holds appointments at the Gerald R. Ford School of Public Policy, School of Education, Department of Economics and Institute for Social Research and serves as co-director of the Education Policy Initiative.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
Notes:
1. https://wcetfrontiers.org/2017/02/16/distance-ed-price-and-cost/
2. https://www.brookings.edu/research/promises-and-pitfalls-of-online-education/
3. Deming DJ, Goldin C, Katz LF, Yuchtman N. “Can Online Learning Bend the Higher Education
Cost Curve?” American Economic Review. 2015;105 (5) :496-501. https://scholar.harvard.edu/files/ddeming/files/aer.p20151024.pdf
4. https://www.nber.org/papers/w23744. The paper also reviews evidence on the effects of access to computers, adaptive learning, and behavioral nudges that make use of technology.
5. https://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/REL_20124021.pdf This paper is not discussed in the review. A What Work’s Clearinghouse review of this study is available at https://ies.ed.gov/ncee/wwc/Docs/QuickReview/algebra_032712.pdf. The study meets WWC standards without reservations.
6. http://www.tandfonline.com/doi/abs/10.1080/19345747.2016.1168500?journalCode=uree20
