Matt Chingos of Brookings and Guido Schwerdt of the University of Konstanz are out today with a new Program on Education Policy and Governance working paper that provides, to my knowledge, the first credible evidence on the effects of online courses on student achievement in K-12 schools. The bottom line is that Florida high school students taking Algebra or English I online perform at least as well on state math and reading tests as do students taking the same courses in a traditional format. That is, their results do not demonstrate the superiority of fully online instruction for promoting student achievement, at least in its earliest iterations. But the results should nonetheless be encouraging to proponents of the potential for online learning to open up new, high-quality educational options for American students.
Florida Virtual School (FLVS) is the nation’s largest statewide virtual school. Indeed, its 462,000 course enrollments in 2013 represented more than half of all enrollments in state virtual schools nationwide. (For the backstory on how FLVS managed to grow so quickly, see Bill Tucker’s analysis here.) More than 95 percent of FLVS students are part-time students who take their other courses in another school.
Chingos and Schwerdt first demonstrate the extent to which the increased choice created by FLVS has enabled students to access educational content that would not otherwise have been available to them. For example, they show that in 2008-09, at least 1,384 FLVS Advanced Placement courses were by 916 students attending high schools that did not offer the same course.
In this respect, their analysis follows the logic of Clayton Christensen’s theory of disruptive innovation, which posits that new technologies first gain a foothold by offering a product or service to consumers who would not otherwise have access to the same product or service. That is, rather than competing directly against existing technologies, they compete against non-consumption. This provides space for the new technology to improve in quality to the point where it matches or even exceeds the quality of existing technologies.
How does the quality of FLVS virtual courses now compare to that of traditional courses? To address that question, Chingos and Schwerdt zero in on Algebra and English I – courses that are available in both formats to virtually all Florida students. First, they compare the 10th-grade test scores of students with similar 8th-grade test scores and demographics, some of whom took the algebra and English courses online with FLVS and others who took the same courses in person at their local public school. But their strongest evidence comes from analyses that identify students who took one of those courses online and the other in person, and ask whether a given student’s 10th-grade test scores were higher or lower in the subject he or she took online. Both analyses show that students do as well – and perhaps a bit better – in courses taken through FLVS as those taken in a traditional format.
One limitation of the analysis is that the data Chingos and Schwerdt examine only ran through 2008-09. On one hand, this means that their results tell us about the effectiveness of virtual courses for “early adopters” who may be particularly well-suited for online learning. On the other hand, it also means that their results do not capture any more recent innovations at FLVS; five years is a long time in the rapidly evolving ed-tech space. And, of course, the study is also not a randomized experiment, the gold standard in education research.
But, in my view, these “do-no-harm” findings support continued experimentation with technology-based innovation in K-12 education, ideally with those efforts subjected to even more rigorous and timely evaluation.
– Marty West
