Failure to Disrupt: Why Technology Alone Can’t Transform Education
by Justin Reich
Harvard University Press, 2020, $27.95, 336 pages.
As reviewed by Michael B. Horn
In Failure to Disrupt, Justin Reich aims to reset the hype around education technology and replace it with a more realistic set of expectations for what such technology can and cannot accomplish. Arriving amid the coronavirus pandemic, the book has heightened relevance, although it does not specifically focus on the abrupt transition to distance learning occasioned by the crisis.
At its core, the book offers a helpful framework for evaluating the likely impact of new edtech products. That framework entails asking four questions about any new education technology: 1) How will existing stakeholders use the technology, and will it help them extend their current practices? 2) What kinds of learning can and cannot be assessed with it? 3) How will learners from different backgrounds and circumstances access it? 4) How could research and experimentation improve the product?
Reich, who is director of the Teaching Systems Lab at MIT, evaluates a range of recent education technologies through this framework. He then concludes that the answers to his questions suggest four corresponding limitations on technology’s potential to bolster learning at scale: 1) Most educators use technology in familiar rather than innovative ways, a practice that replicates current outcomes rather than transforming them. 2) Routine assessment, which is all that machine-based learning systems can perform, measures only learners’ knowledge and their ability to complete rule-based tasks. Technology can’t measure more-complex types of learning at scale. 3) Those with greater resources benefit more from new products than those without such resources, who in turn fall further behind in their learning—a dynamic Reich calls the “EdTech Matthew Effect.” 4) Data and experimentation hold great potential to improve new technologies, but concerns about privacy and “experimenting” on children often keep this from happening.
The most useful observation to come from these insights is that by analyzing how any new edtech product is likely to be used in teaching, we can use research on past products with a similar profile to predict how effective and transformative the new technology will be. Past will be prologue, in other words.
Massive open online courses, or MOOCs, are simply a vehicle for traditional teacher-directed “instructionism,” Reich argues. That means they were fated to have low completion rates—particularly for students in poverty-impacted communities.
Adaptive-learning products, through which individual students work at their own pace, are similar to tools that date back to the 1960s. Reich dives into the meta-analyses of early adaptive-learning tools, as well as research on newer ones, to argue that we shouldn’t expect too much from such products.
And peer-guided learning has consistently shown that it can ignite passion in some learners, but it is also likely to leave many individuals behind. Reich suggests that such approaches should be evaluated based on the depth of learning for the specific individuals who are able to explore their passions, not the numbers of students advancing toward mastery. “Having a learner leave the Scratch [a platform for learning computational thinking] community or pass through with only a light touch isn’t necessarily a loss or a concern,” Reich writes.
In this vein, Reich makes a commonsensical appeal for “methodological pluralism” in education, that is, for an approach that draws on both teacher-driven instructionism and student-driven constructivism. In learning theory, constructivism holds that learners “construct” their own knowledge, while instructionism emphasizes the teacher’s role and sees learning as a behavioral change. He writes, “We need our entire population to have fundamental skills in reading, writing, numeracy, civics, science literacy, and communication; in these domains, we need to take the entire distribution of learners and help them move toward mastery. We also need learning environments that let young people discover their interests and explore them deeply, much more deeply than might be allowed if the environment were equally concerned with bringing along the unenthusiastic with the enthusiastic.”
So far, so good. But Reich’s argument stops short here. On constructivism, he fails to ask: what if learners are failing not because they lack interest but because the environment is poorly constructed or alienating? What if, as he suggests for constructivist learning, we were to evaluate instructor-driven technologies like MOOCs based on the number of people who were turned on to a subject? Would that number be equal to, less than, or more than the number for constructivist learning methods? And if it were equal, what would that say? Similarly, the considerations he offers for designing education technology for equity sound nice and offer good vignettes, but they lack data to support them. Are we to use data when it suits us but ignore it when it’s inconvenient? Reich doesn’t explore these questions.
Reich does concede that applying his framework isn’t as straightforward as it might seem, as he shares his own missteps with creating MOOCs in the MIT Teaching Systems Lab, where his team’s “progress is mixed.” For example, in offering a professional development course for teachers, the team hoped to serve educators in less-affluent schools but found instead that the teachers who participated were “disproportionately likely to come from independent or suburban schools serving affluent students.”
Reich concludes, “For MOOCs to serve populations beyond the already educated, there will need to be substantial support for the social elements of learning—coaching, advising, peer support, and so forth. All of these efforts will require seeing MOOCs not as a technological solution to a complex social problem, but as one element of a comprehensive solution.”
Sensible as it is, this conclusion begins to show how Reich’s narrative overreaches. In Reich’s telling, Harvard Business School professor Clay Christensen, who died in January 2020—and by extension Christensen’s allies and colleagues like myself—are the central bogeymen. Reich depicts us as cheerleaders for a disruptive future who are mistaken in our predictions and fail to understand the underlying historical research on the technology-based pedagogies we advance.
Actually, many of us are more bearish on traditional MOOCs than Reich. We were skeptical of the first generation of MOOCs because they replicated the passive pedagogy of the college classroom without support—not too dissimilar from Reich’s argument. I have also argued that most MOOCs suffer from other limitations, such as the lack of sound instructional design. That constricts their ability even to form a component of a more holistic solution, let alone serve as a robust standalone offering. It also limits the ability of designers and researchers—even with data and controlled experiments—to improve MOOCs, given their flawed starting points. What’s more, MOOCs as launched originally failed the test of being a disruptive innovation because they lacked a coherent business model and a technological driver that allowed them to improve.
Ironically, as MOOC platforms like Coursera and edX have pivoted to manage some universities’ online programs—evidence, Reich asserts, that MOOCs are becoming mainstream rather than disruptive—they are more likely to contribute to the demise of mid-tier universities that depend on pricey master’s programs to subsidize their expensive-to-run undergraduate offerings. Although Christensen and I did not argue, as Reich mistakenly asserts, that online learning would be the main driver of colleges closing, this move by the MOOC providers likely makes them more threatening to many universities and their troubled business model, not less.
Reich also attacks a big prediction that Christenson, Curtis W. Johnson, and I made in our 2008 book Disrupting Class—that by 2019, 50 percent of all high school courses would be delivered online. But Reich calculates the number of students in full-time virtual schools, a data point that is irrelevant to our prediction, as we anticipated that over 90 percent of the online learning would occur in physical schools. Our prediction centered on the extent to which digital materials would replace analog, offline curricula. Based on information I gathered from digital curriculum providers, at least 13 million K–12 students, or about 25 percent of them, were learning through digital curriculum for at least a portion of their day a couple years before the pandemic hit. According to a representative Digital Promise survey from April 2019, 35 percent of the responding K–12 public school teachers report using edtech daily and another 23 percent use it most days—adding up to more than 50 percent of such educators. To be clear, that hardly confirms that our prediction was correct, but it signals that digital learning has grown rapidly. That growth has only accelerated amid the pandemic. And although we were wrong about how much the costs of online learning would drop—Reich notes that we predicted they would fall by two-thirds—today, a class on a platform such as Outschool often costs less than half of what a provider would have charged for a comparable course in 2009. We were correct in forecasting the downward price trend, but wrong in our estimate of its magnitude.
What’s more, Reich asserts that in Disrupting Class we were cheerleaders for certain technologies that we in fact did not champion. As he dissects the research behind computer-adaptive instruction, he claims that we said adaptive online learning would come to dominate K–12 schools. But the word “adaptive” never appears in our book. We wrote that, for education technology to customize to individual learners, a facilitated-network model—in which students and teachers would teach each other—would need to emerge. Such a model would more closely resemble the peer-guided learning models Reich analyzes, not adaptive courseware.
Salman Khan, the founder of the free online-education platform Khan Academy, is another of Reich’s bogeymen. Reich misrepresents Khan’s recommendations on how learners can most effectively use the platform. In Reich’s telling, Khan says in regard to math learning that “the proper first step toward deeper learning is learning mathematical procedures and facts that might eventually lead to doing interesting collaborative projects.” Yet that is the inverse of how students in Khan’s own physical school—the Khan Lab School—typically use his platform. In that setting, students often study math through interest-based projects in which they call upon procedures and facts as needed and in turn have added intrinsic motivation to learn them. Here again, there’s more agreement than disagreement.
Reich is correct that our larger hope in Disrupting Class has failed to come to pass—that is, the hope that the U.S. education system would undergo transformation into a student-centered system in which young people would have more opportunities to build on their passions and fulfill their potential. After advancing one explanation for this disappointment—that many of the new education technologies introduced were based on well-studied pedagogies that could have been predicted to have limited impact—Reich turns to another reason: that schools are complicated.
In citing Stanford pro-fessors Larry Cuban and David Tyack, he turns to the same source we did in explaining why it would be difficult to transform schools by working within existing classrooms. A central tenet of the theory of disruptive innovation is that technologies that succeed in transforming an industry get their start not by challenging the industry’s dominant players directly, but rather by serving segments of the market the dominant players have ignored—that is, by competing against nonconsumption. As we wrote in Disrupting Class, “When disruptive innovators target nonconsumption for their foothold applications, they have a good chance of succeeding. But if those applications are then ensconced within a value network—a chain from suppliers to customers whose definitions of quality and profitability were honed in the established way of doing things—the disruption won’t fly unless it conforms to the rest of the players’ needs and expectations. That typically limits the scope of the innovation. And it is expensive. It is for these reasons that disruptive growth is truly unleashed only when the new technology is taken to the market not only through a disruptive business model, but also by utilizing a disruptive value network—from suppliers through distributors—whose economics are consonant with the disruption.”
Much of the disruption we wrote about in Disrupting Class has occurred within a larger system of existing schools and districts with a multiplicity of stakeholder interests and regulatory rules and practices. Dis-rupting from within can result in a change of modalities without a change of the larger rules of the game—and therefore create the incentives for technologies to fall in line with existing practice rather than change it. In the case of schools, that means technologies serve to perpetuate the traditional model of students advancing in their education at a uniform pace according to a content guide, for example. The transformative effects we had hoped for have, alas,
been limited.
Unfortunately, Reich doesn’t probe more deeply here toward finding a path forward. He misreads the most recent research on Teach to One, a software-based, adaptive, personalized math program. Reich cites one randomized evaluation that found that the program fails to produce improved student results on state tests. But he fails to note that a second study showed that, in those places where school accountability systems measured student growth and therefore incentivized teachers to address the gaps in student knowledge rather than just teach grade-level material, Teach to One produced significant gains on benchmark assessments (See “The Grade-Level Expectations Trap,” features, Summer 2020). In other words, tests that dictate a school’s priorities matter and can help us understand what pedagogies and practices can or cannot be successfully adopted.
Reich doesn’t grapple with these more nuanced parts of the theory of disruptive innovation. His writing also reveals a mistaken assumption: that anyone touting the power of disruption must see technology as the most important ingredient. We devoted the entire third chapter of Disrupting Class to showing that the model in which a technology is used carries far more weight than the technology itself, which helped explain why computers had not had a substantial impact in schools. Understanding this point—that the model in which something operates is far more important than the technology itself—is central to the theory of disruptive innovation.
With this insight, one realizes that much of Reich’s ultimate argument isn’t necessarily in conflict with ours. The two are consistent with each other.
His argument that community matters more than technology is similar to, although less precise than, our claim that the model—comprising the school resources, processes, priorities, resulting culture, and incentives for revenue—matters more than the technology. When he argues that “creative educators [should] find more spaces where peer-guided large-scale learning can be woven into the periphery of schools—in electives, extracurriculars, and untested subjects—so that learners can have some practice in navigating these new networks,” he’s making the same argument we made in Disrupting Class. Innovators seeking to introduce new technologies should start in areas of nonconsumption; only then will they have the freedom to create the proper model for what they want to achieve. Many education innovators have ignored this advice, given their short time horizons for achieving financial sustainability—perhaps another reason why our larger hopes from Disrupting Class have not materialized.
If Reich’s hope for MOOCs—that they become embedded in novel educational models with wraparound supports to help learners succeed—came true, it would be a showcase for our work, with digital curriculum replacing print and a comprehensive model with the power to displace the status quo. This points to perhaps the biggest problem with Reich’s book, which is the title itself: “Failure to Disrupt: Why Technology Alone Can’t Transform Education.” It is common sense, but it is also a straw man.
Michael B. Horn is co-founder of the Clayton Christensen Institute for Disruptive Innovation and an executive editor of Education Next.
This article appeared in the Spring 2021 issue of Education Next. Suggested citation format:
Horn, M.B. (2021). Effort to Debunk Education Technology Falters by Overstating Its Own Case: A sensible, but hardly novel, appeal for “methodological pluralism.” Education Next, 21(2), 83-86.
