The First Hard Evidence on Virtual Education

By 09/12/2014

1 Comment | Print | NO PDF |

Matt Chingos of Brookings and Guido Schwerdt of the University of Konstanz are out today with a new Program on Education Policy and Governance working paper that provides, to my knowledge, the first credible evidence on the effects of online courses on student achievement in K-12 schools.  The bottom line is that Florida high school students taking Algebra or English I online perform at least as well on state math and reading tests as do students taking the same courses in a traditional format.  That is, their results do not demonstrate the superiority of fully online instruction for promoting student achievement, at least in its earliest iterations.  But the results should nonetheless be encouraging to proponents of the potential for online learning to open up new, high-quality educational options for American students.

Florida Virtual School (FLVS) is the nation’s largest statewide virtual school.  Indeed, its 462,000 course enrollments in 2013 represented more than half of all enrollments in state virtual schools nationwide. (For the backstory on how FLVS managed to grow so quickly, see Bill Tucker’s analysis here.) More than 95 percent of FLVS students are part-time students who take their other courses in another school.

Chingos and Schwerdt first demonstrate the extent to which the increased choice created by FLVS has enabled students to access educational content that would not otherwise have been available to them.  For example, they show that in 2008-09, at least 1,384 FLVS Advanced Placement courses were by 916 students attending high schools that did not offer the same course.

In this respect, their analysis follows the logic of Clayton Christensen’s theory of disruptive innovation, which posits that new technologies first gain a foothold by offering a product or service to consumers who would not otherwise have access to the same product or service.  That is, rather than competing directly against existing technologies, they compete against non-consumption.  This provides space for the new technology to improve in quality to the point where it matches or even exceeds the quality of existing technologies.

How does the quality of FLVS virtual courses now compare to that of traditional courses?  To address that question, Chingos and Schwerdt zero in on Algebra and English I – courses that are available in both formats to virtually all Florida students.  First, they compare the 10th-grade test scores of students with similar 8th-grade test scores and demographics, some of whom took the algebra and English courses online with FLVS and others who took the same courses in person at their local public school.  But their strongest evidence comes from analyses that identify students who took one of those courses online and the other in person, and ask whether a given student’s 10th-grade test scores were higher or lower in the subject he or she took online.  Both analyses show that students do as well – and perhaps a bit better – in courses taken through FLVS as those taken in a traditional format.

One limitation of the analysis is that the data Chingos and Schwerdt examine only ran through 2008-09.  On one hand, this means that their results tell us about the effectiveness of virtual courses for “early adopters” who may be particularly well-suited for online learning. On the other hand, it also means that their results do not capture any more recent innovations at FLVS; five years is a long time in the rapidly evolving ed-tech space.  And, of course, the study is also not a randomized experiment, the gold standard in education research.

But, in my view, these “do-no-harm” findings support continued experimentation with technology-based innovation in K-12 education, ideally with those efforts subjected to even more rigorous and timely evaluation.

– Marty West

Comment on this article
  • Scott Petri says:

    Nice piece, Martin.

    Means et al (2010) conducted a meta-analysis for the U.S. Department of Education and found only five experimental or controlled quasi experimental studies published between 1994 and 2008 that compared the learning effects of online versus face-to-face instruction for K-12 students. The analysis revealed that students in online courses performed modestly better than those learning the same material in face to face instruction. Johnson & Mejia (2014) conducted a more recent analysis using ten years of California Community College data to examine 750,000 student records. This effort revealed that online students did worse than face-to-face students in traditional courses.

    Many of my teaching colleagues feel that schools and districts under pressure from NCLB are pushing their weakest students out of traditional public schools and into online charters and credit recovery programs where they are not likely to persist to a diploma. Credit recovery is the fastest growing area of online learning (McCabe & St. Andrie, 2012). These programs are highly unregulated and there is minimal information on their enrollments or effectiveness. More research, please.

  • Comment on this Article

    Name ()


         1 Comment
    Sponsored Results

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform