Two education bills from George W. Bush’s first term are long overdue for reauthorization. One, of course, is the No Child Left Behind Act (NCLB), passed in late 2001. The other is the Education Sciences Reform Act (ESRA), which in November 2002 replaced the Office of Educational Research and Improvement (OERI) with a new Institute of Education Sciences (IES).
The first bill is already iconic: a Lexis-Nexis search covering NCLB since its passage quickly overloads and shuts itself down. No such difficulties, however, hamper a hunt for stories about IES. Half of the meager fourscore hits produced by the agency’s title turn out to be for a similarly named organization in Angola. Searching instead for IES director Grover “Russ” Whitehurst only confirms that many high school sports teams have a Whitehurst on the roster. It seems safe to say that IES is operating well under the radar of mainstream media attention.
But as the organization responsible for putting the “scientifically based” in the scientifically based research mandated by NCLB—not to mention the home of the National Center for Education Statistics (NCES) and the National Assessment of Educational Progress (NAEP)—IES deserves attention in its own right. With Whitehurst moving to the Brown Center on Education Policy at the Brookings Institution after his term expires this month, and as drafts for IES reauthorization begin to make the Beltway rounds, it is time to assess the contribution of IES to the history of federal education research and look ahead to its future.
At best, education research is hard to do well: it studies a field in which all else is too rarely equal, one often driven not by facts but by values and aspiration. “Scientific” study of education at the federal level is thus a difficult mission. Achieving it has been made harder still by the inability of a succession of research agencies to withstand pressures to politicize or gain significant funding. Lawmakers constantly tout the virtues of the National Science Foundation (NSF) and the National Institutes of Health (NIH) when discussing what they want from education research, but they have just as constantly failed to endow that vision with either firewalls or cash.
IES, however, has a structure that heightens its autonomy from political interference. Whitehurst has used the opening provided to promote a new culture of favored research that, while sometimes narrow in methodological scope, has changed the conversation in the research community about rigor and relevance—and thus changed, at least at the margins, the research enterprise itself.
Still, has education research managed to move from tracing a vicious circle to a virtuous one? With new sets of policymakers—in the White House, Congress, and IES itself—taking office after the 2008 elections, the real tests may be yet to come.
Truth vs. Partisanship
President Hoover’s 1931 advisory committee on education grandly proclaimed the battle of “Truth vs. Partisanship.” Education policy “cannot hope to rise above partisanship,” it argued, “…unless mere differences of opinion, tenaciously held, are dissolved by revelations of pertinent facts established by scientific method and presented in understandable terms.” The Hoover committee thus summed up two truths about federal education research. First, rhetoric elevating the methodical quest for the scientific truths underlying education and its outcomes would echo repeatedly throughout the decades that followed. Second, its reality would fall far short. Even though a desire to collate and diffuse “statistics and facts” about education motivated the creation in 1867 of the earliest national department of education, research neither before Hoover nor since has ever achieved the status his advisors envisioned.
Indeed, at the turn of the 19th century, the first Office of Education (OE), by virtue of its placement in the Interior Department, spent more effort administering the breeding of Alaskan reindeer than driving or disseminating education research. (The reindeer would depart for greener bureaucratic pastures in 1907.) Periodic efforts to bolster OE’s clout did not take hold until after World War II. The faith in scientific research inspired by the war, afterward institutionalized in the likes of the NSF and the NIH, soon spread to education. In the 1960s, a series of research-and-development centers and then regional educational laboratories was rapidly established. But soon a grander proposal was in play—a plan, as Lee Sproull and her coauthors describe in Organizing an Anarchy, born “out of frustration with the failure of OE and belief in the power of scientific research.” The driving force was Nixon aide Daniel Patrick Moynihan, who had realized (he wrote in 1969) that “almost nothing is known about education.” OE had inadequate resources or autonomy to do much about this, and Moynihan convinced the president to seek creation of a powerful independent research organization. Nixon’s subsequent message to Congress stressed the application of “science…to the techniques of teaching” and outlined an independent National Institute of Education (NIE) that would hire a “permanent staff of outstanding scholars” from across the sciences, contract with other researchers, and serve as “a focus for educational research and experimentation.” As such, the president anticipated a quarter-billion-dollar annual appropriation. The Budget Bureau’s Emerson Elliott, who became acting director of NIE, says it aimed even higher: “it was formulated by Moynihan as a billion dollar agency.”
Things didn’t work out that way. Michael Timpane, NIE director in the Carter administration, later said the agency “inherited at best a mixed bag of research projects, was slow to organize, inexpert at explaining itself, and soon fell from political grace.” As an indicator of the latter plunge, consider that Nixon’s $250 million agency netted $70 million in fiscal 1975 and barely $50 million 10 years later. Across 13 years NIE had six directors plus five acting directors, and was subject to constant reorganization and political intrusion. President Reagan fired its entire advisory board, despite its members’ statutory fixed terms, and his first director was most noted for decrying the “false” premise “that education is a science, whose progress depends on systematic research…”
In 1985, however, Secretary of Education William Bennett merged NIE and NCES into OERI. OERI head Chester Finn, as an aide to Moynihan, had helped create NIE; now he helped put it out of its misery.
Finn’s greatest success was in revamping NCES as an authoritative source for neutral data (see “Troublemaker,” features, Spring 2008). But the labs and centers, with which Finn waged war over earmarking, returned those attacks with political interest. And over time, the OERI suffered from some of the same problems as NIE. It had frequent changes of leadership; new monies it received were encumbered by new duties. The agency remained politically permeable and legislatively micromanaged.
On cue, the rhetorical cycle began again: OERI needed to be “depoliticized,” Rep. Major Owens (D-New York) argued in 1991, “so that… research activities can gain the kinds of credibility and support they merit.” Owens favored the restoration of a powerful independent policy board; others, gazing at the NSF/NIH grail, wanted to organize around distinct “institutes” within the agency, each focused on a different educational ill. Five of these were ultimately created, but not supported. Bush OERI director Christopher Cross argued that the institutes would need $50 million—each. But funding in fiscal year 1996 totaled just $43 million.
By the time reauthorization came due again, OERI was in trouble. The Clinton administration decision to use OERI to develop and oversee a system of voluntary national tests drew renewed accusations that the agency had been politicized for the president’s policy purposes, and the GOP-led Congress finally forced the issue out of the agency’s jurisdiction; Rep. Michael Castle (R-Delaware) would soon charge OERI with a long list of familiar sins, including “the creeping influence of short-lived partisan or political operatives, the funding and dissemination of questionable studies…and…no real sense of mission, mired by duplicative programs and competing interests.” In the fight between truth and partisanship, partisanship had won yet another round.
The Education Sciences Reform Act of 2002
Unexpectedly, though, as a new iteration of the research function took shape, what political scientist Terry Moe calls “the politics of bureaucratic structure” actually worked to promote agency insulation. There were various reasons for this, among them the fruitful foray into the study of reading by the National Institute of Child Health and Human Development and the centrality of such “scientifically based research” in the concurrent debate over the Elementary and Secondary Education Act. President Bush praised ESRA as “an important complement” to NCLB that would “substantially strengthen the scientific basis” of classroom teaching. Likewise, the Democratic legislators so crucial to the passage of NCLB made ESRA bipartisan as well. As research became salient, so did “good” research, and as the congressional debate over education became inseparable from a broad endorsement of “accountability,” an autonomous IES became a marker of members’ commitment to that cause.
Accordingly, Castle’s first draft of ESRA in 2000 envisioned a National Academy for Education Research, Statistics, Evaluation, and Information independent of the Department of Education, run by a director serving a fixed term. Each of its three centers (for research, statistics, and evaluation) would be headed by a presidential appointee also serving a fixed term and under statutory stricture to guarantee that agency activities were “free from ideological agendas and undue political influence.” Intriguingly, what constituted valid research was to be mandated in law. Quantitative hypotheses, for instance, had to be “evaluated using experimental designs in which individuals, entities, programs, or activities are assigned to different conditions with appropriate controls to evaluate the effects of the condition of interest through random assignment experiments, or other designs to the extent such designs contain within-condition or across-condition controls.”
The end product retained key features of this first draft. But a series of changes was made over time. The academy was moved back into the department, becoming the Institute of Education Sciences, but retained its independent director, who gained appointment powers over the centers’ commissioners—except, notably, at NCES. The definition of “scientifically based research standards” for education wound up less proscriptive than in earlier drafts, but still limited causal claims to research designs that could bear their weight. A new advisory board (the National Board for Education Sciences, or NBES) was given formal approval powers over the institute’s long-term research priorities and its mandated strengthened peer-review process. And for the first time, a majority of the board was to be composed of researchers, rather than practitioners or other consumers of research (see Figure 1).
The first director of IES, moving over from OERI, was Russ Whitehurst. Whitehurst’s strong academic credentials broadcast his methodological predilections. He came to the Bush administration from the quantitatively minded psychology department at SUNY-Stony Brook with a long scholarly record that included stints editing research journals and serving as academic director of the Merrill-Palmer Institute. Per the mission statement of the latter, IES would be led by someone dedicated to “evidence-based programs and interventions” and the models that probed them.
The First Authorization Cycle
In the fall of 2006, IES received what an Education Week survey termed “mixed, but mostly positive, grades”; and the follow-up here likewise found widespread praise, leavened with caution. All sides agree that the six years since ESRA was passed have been consequential ones for education research.
Fans of IES tend to praise a new sense of agency independence and, not unrelated, a renewed emphasis on scientific rigor. IES has more structural autonomy than its predecessors; and below the surface the agency’s authority is also much improved, former OERI head Kent McGuire says, in ways that are “not sexy but critical.” In one telling shift, the director shot up the bureaucratic ranks to Level II on the executive schedule. That placed the position just one rung beneath the secretary and, by no coincidence, at the same rank as the director of NSF. (Assistant secretaries, by contrast, are normally at Level IV.) The ability to hire personnel outside civil service constraints and to roll funds over multiple fiscal years has been equally helpful. So too has language in ESRA that gives the director explicit authority to prepare and publish research and evaluation reports “without the approval of the Secretary or any other office of the Department.”
Thus if, as Finn argues, “three things matter in Washington, in terms of autonomy: the ability to manage your own money; to manage your own people; and to manage your own words,” IES has made important strides on all three fronts. One insider notes the dramatic contrast between the “most partisan Department of Education I have seen since 1988, and the most independent research and stats agency.”
IES has taken advantage of this independence to drive changes in expectations and values in the research community. The agency’s grant-review process was revamped to one Whitehurst says was “explicitly modeled on NIH,” emphasizing separation of the peer review and contracting processes and requiring agency reports to be peer reviewed before publication. The change in vocabulary has been striking: loud language touting the “rigorous scientific standards” set forth in statute emanates constantly from IES, relentlessly pressing the importance of randomized, controlled trials as the “gold standard” for judging whether and how a given intervention is effective. The online What Works Clearinghouse (WWC) judges “what works” with regard to curricular interventions on this basis alone. Blasted by high-volume endorsements of experimental design, researchers may be forgiven for feeling like Manuel Noriega in his 1989 Panama compound.
Some, of course, are clapping along to that beat; the quantitatively minded choir to which IES could preach has been a growing part of the education research community. But there are many others dubious of what they see as the myopic focus of IES. For them, the IES mantra of “transformation” via “evidence-based intervention” amounts (ironically enough) to a sort of faith-based program, an unrealistically dogmatic view of what should count as research. Instead, they argue, as former OERI head Sharon Robinson puts it, “you have to have a range of methods and rigor in all methodologies” and that an exclusive approach created mistrust in large segments of the education establishment, instead of “enlarging the political army.” Further, Knowledge Alliance’s Jim Kohlmoos argues that “the irony is that as you insulate [the agency]—which is a positive thing, in my opinion—you run the risk of becoming less relevant” to the world of practice. This connection, of course, is one of the things that WWC is designed to enhance; that it is not uniformly seen as doing so remains a concern. (Recently a parallel, and peppier, site titled Doing What Works has arisen in the department—not in IES, though it uses IES research.)
Suggesting these views had at least some traction on the Hill, early House drafts of NCLB reauthorization tinkered with the standard for scientifically based research, weakening its focus on experiments. A recent proposal for reauthorization approved by the IES advisory board, the National Board for Education Sciences, tweaks the language back again to reemphasize that causal claims can be made “only in well-designed and implemented random assignment experiments, when feasible.”
Even those who resent that focus must concede that IES practices what it preaches. The agency has rarely parroted the lines scripted by its political bosses or industrial heavyweights: Whitehurst was careful to cast caveats upon education secretary Margaret Spellings’s claims that rising student test scores in 2005 offered “proof” of NCLB’s effectiveness, adding in 2007 that widely divergent state standards for “proficiency” in fact made it hard to judge NCLB’s impact. While the administration has strongly pushed phonics-based reading curricula, IES gave positive marks to the rival Reading Recovery program and later reported ambivalent if still preliminary results for Reading First (a broader report on this topic is due in late 2008). And WWC found in 2007 that expensively touted math texts and test-preparation software had little effect on student outcomes: textbook superpower Houghton Mifflin’s elementary mathematics curriculum, for instance, was “found to have no discernible effects on…achievement.”
Such insulation is good for the perceived integrity of the institute. Even congressional Democrats, angered by what they see as the gross politicization of science agencies under the Bush administration, make an exception for IES. But being apolitical takes special political skill. Otherwise—as with the initial Reading First report—it runs the risk of antagonizing potential allies, and of cooling ardor for institute initiatives and departmental appropriations (see Figure 2). (Reading First funds have been cut from $1 billion to $393 million to, possibly, zero in 2009, and IES’s own funding has been flat since fiscal 2004.) It can mean, too, that the institute does not participate in shaping legislation with important ramifications for research, such as the NCLB reauthorization.
Over time, however, as Whitehurst’s tenure lengthened vis-à-vis latecomers to the administration, his role of honest if prickly broker has garnered more deference. Time, too, has helped soften the edges between Whitehurst’s focus on quantitative experimental methodology and those pushing more immediate “relevance” over his brand of “rigor.” Sharon Robinson concedes, “They’ve come a long way toward lowering the guard and righting the rhetoric,” and Whitehurst himself seemed to be channeling his inner Robinson in February, when he told an American Enterprise Institute conference that education is “richly contextual and multivariate” and that research should encompass a “panoply of existing techniques.”
Still, moving toward reauthorization, IES must deal with concerns from at least two corners of the education research constituency. On one side are partisans of the regional centers and labs first created in the 1960s. Under IES the centers have seen their funding cut and the scope of their research tightened; the labs have been pushed to supplement with experimental research their traditional role of local dissemination and “rapid response” technical assistance. Whitehurst argues that while Congress should determine what it wants to know, IES should determine how to find that out. But the labs, especially, originally set up to maximize local political support, have always done so; and their customers, officials used to receiving quick customized help, have not always been pleased with the changes at IES. Indeed, the Democratic addendum to ESRA notes that protecting the regional system was “a critical priority.” Those members are now in the majority.
Coming from another angle are those who fear for the independence of the National Center for Education Statistics. Recall that the head of NCES differs from other IES commissioners in being a Senate-confirmed presidential appointee with a fixed term. The idea is to give the job additional status and independence, even from IES, placing yet one more barrier between politics and data. As a guardian of statistical information, NCES holds its main aim to be the integrity and credibility of that data; it should have little to say about its analysis.
But the managerial messiness of this has never pleased Whitehurst, and IES has sat rather heavily astride both NCES inputs (especially hiring) and outputs (its publications). NBES’s proposed reauthorization language goes further, making the NCES commissioner an appointee of the IES director, though dismissible only for cause. Advisory board chair Robert Granger argues that this is “a more rational model,” creating efficient lines of accountability and ensuring that the director and commissioner will be compatible. Whitehurst suggests that having a presidential appointee at NCES could prove more problematic, not less, in terms of partisan influence, without the director to buffer the commissioner from political winds.
One could foresee circumstances in which this would be true. But in general the proposal runs counter to the long-standing recommendations of the National Research Council and its guide to Principles and Practices for a Federal Statistical Agency. Certainly the heft of a presidential appointment is useful to the NCES commissioner in bureaucratic dealings. More broadly, statistical agencies, in this view, are qualitatively different, not to be treated, as Elliott puts it, as “just another government agency.” Even such a fan of managerial efficiency as the Office of Management and Budget, whose program evaluation gives both IES and NCES coveted (and rare) grades of “effective,” feels that “real and perceived independence” is best served by the current system.
The drive by IES for managerial (and budgetary?) control also seeks to alter the complicated relationship between NCES and the National Assessment Governing Board (NAGB), which oversees the National Assessment of Educational Progress (NAEP). NAGB has autonomy regarding the conduct and presentation of NAEP, but proposed changes would give most of that power to NCES, which would itself be under the thumb of IES. NCES would be happy to get the envisioned authority in the proposed language, if limits on its own autonomy were not part of the deal, but the current system is intentionally uncomfortable, dividing authority in a way that is itself insulating.
Looking Ahead
It seems unlikely, given current sentiment on the Hill, that these changes will receive legislative sanction. But there will be plenty of time for the debate to play out. Few expect much progress on ESRA reauthorization even in 2009, pending movement of the equally late NCLB reauthorization. As that approaches, it is worth remembering Whitehurst’s hopeful testimony in 2002 that “we are close to a point where the right investment in the right structure could get us close to a tipping point, where education becomes an evidence-based field.” Has that occurred?
The quick descent to technical minutiae in the discussions already occurring on reauthorization actually bodes well on this point. Early murmurings suggested the new ESRA could be passed, as a noncontroversial measure, in order to clear the congressional decks for NCLB. This didn’t happen, of course, but even informal designation as “noncontroversial” suggests IES has come a long way. There are few questions about its basic structure. Legislators seem serious about avoiding confirmation of a partisan hack to the IES directorship, when that job opens up. So far, insulation remains good politics.
But politics lurk close. In a months-long tussle between IES and Congress over the evaluation of Upward Bound, IES insisted that a robust test of whether the program made a difference to students required a control group—students eligible for but not served by it. This meant some students would be recruited for Upward Bound but denied participation. And lawmakers decried that as unethical and “discriminatory,” even comparing it, ominously (if inaccurately), to the infamous Tuskegee syphilis experiments. In early 2008 the evaluation was canceled.
Whitehurst called the legislative intervention “a terrific mistake.” On scientific grounds, he was right. But as he conceded, “We produced some of these problems for ourselves,” in part by centering on a single kind of evaluative procedure and by failing to consult with the advocacy community and Congress in advance before grappling with a popular program.
Will “the Upward Bound problem…be contained to Upward Bound,” as Whitehurst later insisted? Probably so, but it nonetheless showed how easily breached is agency autonomy when its exercise roils partisan waters. Further, if a director as experienced as Whitehurst could stumble into this sort of controversy, how will his greener successor fare? Some of Whitehurst’s effectiveness, and leeway, is a matter of longevity and related learning (on all sides) that will have to be repeated after the transition. Given ESRA’s tight association with NCLB, the crumbling consensus around the latter will bring new pitfalls to navigate.
Whatever November’s outcome, January 2009 will bring major changes to Washington. It is not clear that it is worth betting the structural house on things having changed for good. Indeed, in purely practical terms, the IES budget is not big enough to tip the field of education research on its own. Structure matters, but so do resources; and the politics of knowledge is no different from politics as a whole—defined, as Harold Laswell put it long ago, by who gets what, when, and how. By this standard, IES is distinctly undercapitalized (see Figure 3). The institute’s funding (for research and dissemination) is up by a third compared to the final days of OERI, to $160 million or so; that said, that level was achieved in fiscal 2004 and has hardly budged since. The bipartisan NCLB commission proposed doubling that amount. And in its draft reauthorization bill, NBES proposes that IES become a $1 billion agency as of fiscal 2009, finally meeting Pat Moynihan’s 1970 vision. Of course, a billion 1970 dollars equates to some $5.3 billion today. Some advocates argue that to parallel corporate research-and-development budgets the figure should really be 2 percent of overall investment, or something like $10 billion.
That those numbers sound utopian speaks to the need for new political dexterity at IES that also maintains its scientific integrity. Still, Whitehurst and IES can take pride in having made a solid start. If the randomized-controlled-trials approach was oversold, this has served to move the field by force, further and faster, toward recognizing the need for rigorous research standards generally. The unstinting focus on promoting scientific rigor, even through a single lens, must thus be viewed as broadly positive. And if these changes are, ultimately, at the margins, in American politics that is where most change occurs. The institution is in place: the next administration, next Congress, and next director will have the chance to institutionalize it.
Andrew Rudalevige is associate professor of political science at Dickinson College. This article is adapted from “Structure and Science in Education Research,” in Frederick Hess, ed., When Research Matters (Harvard Education Press, 2008).
