Evidence In Education: A Look to the Future

By 04/16/2014

3 Comments | Print | NO PDF |

The following is an excerpt from What Lies Ahead for America’s Children and Their Schools, a new book edited by Chester E. Finn, Jr. and Richard Sousa for Hoover Institution Press. This excerpt comes from a chapter called “Relying on Evidence” by Grover J. “Russ” Whitehurst.

The transformation of education from a field based on intuition, historical practice, and fad and fancy to a field based on evidence has been thwarted until recently by an inadequate supply of rigorous and relevant research. The supply side of evidence-based education has advanced rapidly in the last decade. But demand is still weak.

The essential question for those interested in the advance of evidence-based education is whether demand is weak because there is something wrong with the research that is being provided or because there is something about the way the field of education is organized that suppresses the market for evidence-based approaches. The problem lies in both areas.

The research and development enterprise in education needs to invest more deeply and systematically in process innovations that will serve the practical needs of school districts and schools. We are unlikely to get dramatically better at educating students until we have a cadre of researchers whose job is to engineer more efficient and effective processes for carrying out the work of schools. Education has an increasingly strong research community, but it lacks more than a few people trained and employed to improve the workaday processes of delivering education through systematic experimentation.

If Harrah’s intends to increase the number of its mid-week customers from Southern California through the mailing of a discount offer, the person responsible for designing the solicitation will lose his job if there isn’t a control group. If Google wants to find the display size for search results that generates the most click-throughs on smart phones, it will systematically vary that parameter across different randomly selected groups of users. It has employees and contractors to design the intervention and analyze the results. Its business success depends on finding the best answer.

In contrast, if a large school district wants to redesign its processes for recruiting new teachers by changing when applications are due and offers of employment are made, it would be exceedingly rare if it either had anyone on staff or could find anyone in a local university who would be interested and able to carry out an experiment on the issue. The education research community, which is predominantly comprised of academics, is not interested in such atheoretical, small-bore questions. But these are the types of issues that education administrators address, whereas broad questions of education policy seldom are within their bailiwick. And because the managers of schools and school districts have rarely if ever been supplied with research that directly addresses the decisions they have to make, they have not had the opportunity to develop an appetite for evidence-based education.

Those who have responsibility for the supply of education research, including universities and funding agencies, need to create a pipeline that is primed with practical research of immediate relevance to everyday education decisions. This will require not only a redirection of the goal of much education research but also much better access by the research community to the administrative data at the state, district, and school level on which the research would draw. It will also require a new channel of federal funding for short-term projects that are of immediate practical significance and that can be reviewed and funded within a few months. This is in contrast to the current modus operandi in which applications for research grants can take a year or more to make their way through the review system and are typically for multiyear projects.

Whether a supply of immediately practical research findings will increase demand for evidence-based education is an empirical question. I expect it would be useful but that its impact would be muted by the same factors that suppress that uptake of evidence that already exists on the impact of broader policies and programs.

The reason that businesses such as Google, Harrah’s, and CapitalOne have an appetite for evidence of what works is that avoidable errors in their business decisions go directly to their bottom line, for which managers at many levels and the CEOs are accountable. Google needs to figure out how to maintain its search dominance on mobile devices or others will take its market share. There are lots of casinos in Las Vegas. Harrah’s success depends on competing successfully for customers. And so on.

Education, in contrast, is by and large a public monopoly. A recruitment process for new teachers that is much less effective than it might be does not result in the school district losing students or revenue, at least not within a time span or through a series of events that would make the connection discernible. A truly dysfunctional management process may call attention to itself and the administrator responsible for it, but there is no incentive built into the system to experiment with improvements in processes that seem to be OK.

The most powerful way to incentivize evidence-based decision-making in education would be a system of delivery in which schools compete for students and their funding and in which the jobs and compensation of school employees and managers are conditional on their success in attracting and retaining students. Until errors in decision-making have palpable consequences for those responsible for those decisions, the demand for evidence that will enable better decisions will be weak.

In short, the education research community needs to prime the pump of evidence-based education with a supply of research findings that are of immediate relevance to workaday decision-making, e.g., recruiting tools that enhance the effectiveness of the workforce; ways to increase the productivity of the central office; and differences in the impact of available curriculum materials for particular types of teachers and students. If this is to have more than marginal impact, it will need to be accompanied by a redesign of the delivery of education services such that schools and those who work in them are subject to market forces.

The nation can no longer tolerate vast differences in the quality of its schools and classrooms. Residential geography and quirks of school choice and classroom assignment cannot continue to define the education destiny of individual students. We need for all of our schools to be good enough to do the job that is expected of them. This will require nothing less than a relentless effort to engineer processes that assure the best possible outcomes and that result in continuous improvement. Much of this work will be down in the weeds and the results of any single effort will be incremental. Examined within a short time frame, those results may not look like they are going very far. But it is the accumulation and progression of those incremental improvements that will ultimately be transformational for student achievement and the nation’s future.

Evidence-based education has shown progress over the last decade that seemed unimaginable twenty years ago. A foundation is in place for the kind of explosive growth in knowledge and application of what works that has been seen in other fields. Improvements in the relevance of the supply of research and the incentives for educators to make the best possible decisions are the necessary ingredients for the next stage of the reform of our education system.

Russ Whitehurst is the Herman and George R. Brown Chair and director of the Brown Center on Education Policy at the Brookings Institution.

Reprinted from What Lies Ahead for America’s Children and Their Schools, edited by Chester E. Finn Jr. and Richard Sousa, with the permission of the publisher, Hoover Institution Press. Copyright © 2014 by the Board of Trustees of the Leland Stanford Junior University.

Comment on this article
  • John D'Auria says:

    The divide between university research and craft knowledge developed by practitioners also contributes to the problem you eloquently describe in this article. Practical knowledge is not valued within the university research community. AERA is something that most principals and teachers don’t know about and it is not an arena where they would not be welcome. Practitioners are developing insights and solutions daily but often lack the time, not the ability, to communicate and publish. Universities have professors who have the time and the ability but they do not experience the day to day contextual pressures that school personnel face. If we could rotate each group into the other’s context for a year, we might see more practical research. @jdauria

  • Erin says:

    It would be lovely if schools would use evidence to improve student learning. But I think that you have completely missed why schools do not currently use evidence to inform practice.

    You assert that the reason that schools have not used evidence is because there is “an inadequate supply of rigorous and relevant research.” This is completely false.

    Schools do not use evidence because school administrators, curricula directors and teachers do not believe that evidence matters. For the most part, educators believe other factors (relevance, interest, social dynamics, familiarity, sales presentations, etc…) are tremendously more important than evidence of effectiveness. And for the most part, parents agree and support this perspective.

    In the private school world, evidence of effectiveness is not a selling point. Private schools (that do compete with each other for students) sell their programs based upon a multitude of factors (class size, arts, sports, extracurricular, exclusivity, location, facilities, etc.). Evidence is not one of these factors. So if the private schools do not sell themselves based on evidence, why would public schools use this topic as a selling point?

    Instead of using the competition in commerce (Google, Harrah’s et al.), a better model for improving education may come from the medical fields.

    In the early 20th century, health care went from “a field based on intuition, historical practice, and fad and fancy to a field based on evidence” (to quote your description of education, as it exists today).

    So why did medical practice change (to produce an outstanding health care system), when education persists in mediocrity?

    The single most important factor in improving (and continuing to improve) health care was/is the regulation of therapies by the FDA. The FDA requires therapeutic companies to demonstrate effectiveness before their products can be sold. The FDA does not generate the research; the companies generate that evidence. The FDA then evaluates their research to determine if the products are statistically better than the current standard of care. This product evaluation process has been essential for the continual improvements in health care.

    To translate this approach to education: States would insist that publishing companies demonstrate evidence of effectiveness before their products (textbooks, workbooks, teacher’s manuals, professional development, etc…) could be bought by schools with public monies.

    While we may both agree that schools should use evidence to inform their practice, the educators that run our schools emphatically do not. So even if academic research generates volumes of data, educators will never use those findings unless they value that information, which currently they do not.

  • Bonita says:

    Re: ” it lacks more than a few people trained and employed to improve the workaday processes of delivering education through systematic experimentation.”

    What field of study would provide for people trained to improve the workaday processes of delivering education beyond the research itself?

  • Comment on this Article

    Name ()


    Sponsored Results

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform