Federal funding and direction routinely lead to research and development breakthroughs in science, medicine, and defense, among other fields. Witness the recent debate over federal funding of stem-cell research, during which supporters of stem-cell research emphasized that federal funding tends to draw the best scholars to a field. So why do research programs funded by the U.S. Department of Education continue to disseminate research that is barely respected by serious scholars? Many academics and policymakers consider the education research sponsored by the National Science Foundation or the National Institutes of Health to be more rigorous and scientifically sound than the research emanating from the Department of Education’s Office of Educational Research and Improvement (OERI). Likewise, the department’s Planning and Evaluation Service (PES), tasked with evaluating the performance of federal education programs like Title I, has a mixed record of developing sound and timely information on program quality.
As policymakers look to redesign or replace many of the basic federal education programs established during the 1960s, they face mounds of Department-produced research to guide them, but much of it is of limited value. This is a problem that has cut across the decades, during both Republican and Democratic administrations. With OERI up for reauthorization by Congress, this seems like the perfect time to revisit the federal role in research and development.
The Centers and Labs
Federal education research and development has been fragmented and disorganized for most of its history. In an effort to address these problems, in the mid-1960s the government created two complementary sets of institutions intended to promote more coordinated, long-term, large-scale education research and development: the Research and Development Centers and the Regional Educational Laboratories. However, most of their work turned out to be (and continues to be) short term and small scale. They tend to focus on small, individual research projects rather than supporting large-scale development and evaluations of alternative techniques and programs for helping all children to succeed in school.
The value of their research and development efforts has been routinely criticized, yet the share of research funding devoted to the centers and laboratories rose under both the National Institute of Education and its successor, OERI. The labs used their lobbying arm, the Council for Educational Development and Research, to protect their budgets even as funding for research proposed by individual scholars and other initiatives was disappearing or was dramatically curtailed during the Reagan years. As funding for OERI grew through the 1990s, a large share continued to go to the centers and labs or to activities only loosely related to research or development, such as distributing funds for introducing more technology in the classrooms (see Figure 1).
Neither the recent OERI third-year assessment of the centers nor the PES third-year review of the labs was a thorough or totally objective evaluation of the R&D contributions of these institutions. The results were also not made available to the public or to policymakers in a timely or convenient manner. Yet in the closing months of the Clinton administration, OERI made a last-minute decision to fund some of the existing centers for another five years and offered new five-year contracts to the labs-when the agency hasn’t even been reauthorized by Congress yet. It might make some sense to extend existing center or lab grants for another year while awaiting OERI reauthorization, but the decision to make long-term commitments to these institutions, based on inadequate evaluations and before Congress has even signaled its willingness to continue to fund them, seems quite misguided.
To be sure, the sheer paucity of funding places a serious limit on the quality of federally funded education research and evaluation, yet this hardly explains all the problems of education research and evaluation today. OERI has frequently suffered from unstable and intellectually weak leadership and the absence of a distinguished research staff. After Sharon Robinson completed three-and-a-half years as head of OERI, the agency went through four different heads in less than a year. During the Clinton administration, OERI personnel were cut by 25 percent, worsening the agency’s shortage of capable, innovative researchers. Furthermore, the agency has not taken advantage of its opportunities to hire qualified researchers. In the waning days of the Clinton administration, OERI appointed full-time directors for its five research institutes, but only one of these individuals has extensive research training and scholarly experience.
As a result, neither OERI nor Planning and Evaluation Service is providing a significant number of scientifically sound and educationally relevant program evaluations. The education marketplace is crowded with competing “whole school reform” models, such as Success for All, Accelerated Schools, and Core Knowledge, all promising remarkable gains in student achievement, especially among disadvantaged students. If one were to define the federal government’s role in education, wouldn’t one of its core functions be studying whether any of these programs actually live up to their claims? OERI has yet to systematically examine which models of education interventions are especially effective at improving student achievement in different settings. Nor has PES, which has the primary responsibility for conducting program evaluations, been able to accomplish this important task. Consequently, school districts lack the kind of research-based evidence necessary to choose among such programs; often the only information they have comes from the program developers themselves. Even worse, the federal government, through the Obey-Porter legislation, is giving aid to districts to implement many of these programs even though few of them are backed by any solid research.
Likewise, a variety of proposed reforms-charter schools, expanded standardized testing programs, class-size reduction, and school vouchers, to name a few-are currently being considered by legislators nationwide. Yet researchers all too often must turn to private foundations to secure enough funding to evaluate these reforms; OERI has almost no presence in many of these debates. What is the purpose of the Department of Education if not to provide scientifically reliable research information to guide legislators on the merits of various policy choices?
Protecting Research Independence
Federally funded education research needs to be objective and trustworthy, yet periodic political intrusions by Congress and the White House continue to muddy some of the key operations of OERI and of Planning and Evaluation Service. For example, congressional mandates specifying how OERI must spend its R&D funds hamper the agency’s ability to operate efficiently and effectively. Since the mid-1970s a few members of Congress from both parties have allied themselves with the largest beneficiaries of federal research contracts (the labs and centers) and ensured that these organizations receive a substantial portion of federal education funding for R&D.
The executive branch is equally guilty of periodically interfering with the research process. For instance, Pascal “Pat” Forgione was widely regarded as a conscientious and effective commissioner of the National Center for Education Statistics (NCES), the statistical branch of OERI. The national center’s Advisory Council on Education Statistics had strongly urged the secretary of education to reappoint him. Yet he was not renominated, and some speculate that the real cause of Forgione’s departure was his public protest of then-vice president Al Gore’s inappropriate politicization of the Department’s release of reading scores from the National Assessment of Educational Progress (NAEP). Rather than wait for NCES to release the NAEP results objectively, as was the normal and expected procedure, Gore decided to use the occasion to provide his own interpretation of the data-an interpretation that many observers thought was too slanted toward the administration’s own policy interests. Likewise, the Department of Education’s attempts during the late 1990s to use OERI staff and resources in developing the controversial voluntary national tests in reading and math, without congressional authorization, led some legislators to question the research agency’s independence.
| At times the Department of Education has delayed much too long in releasing unfavorable results that might have challenged current initiatives and priorities. |
The Department of Education itself is no stranger to politicking. When it suits its policy purposes, Planning and Evaluation Service has publicized evaluation results early-before the actual reports have even been made public. In 1993, PES published selected excerpts from the preliminary results from Prospects, the national evaluation of Chapter 1 (later renamed Title I), before the full results of the preliminary assessment had been made public. At other times, the agency has delayed much too long in releasing unfavorable results that might have challenged current initiatives and priorities. For instance, the $9 million Longitudinal Evaluation of School Change and Performance was designed to evaluate the effectiveness of the current Title I program. Though they could have, the Department of Education and its contractors did not provide those results in time for deliberations during the 106th Congress, when Title I was up for reauthorization.
The political independence of OERI needs to be safeguarded. Research and evaluation need to be carried out objectively and reported accurately-even if the results challenge some of the policies of the current administration. One solution many analysts have suggested is to create an altogether independent agency. This would be a wise first step. There are some drawbacks to having the agency responsible for education research located outside the Department of Education, but they are outweighed by the benefits of freeing the research unit from political interference and enabling it to develop more rigorous and scientifically sound research practices. Therefore, an independent agency, reporting to the President, should be created to handle education research, development, evaluation, and statistics-something similar to the National Endowment for the Humanities or the National Science Foundation. The proposed organization would house a series of quasi-independent and separate agencies, such as the National Center for Education Research, the National Center for Development and Evaluation, the National Center for Education Statistics, and the National Assessment Governing Board.
The National Center for Education Research would be led by a commissioner of research appointed to a six-year term. It is imperative that the commissioner of research have a distinguished background in research, development, or evaluation. The size of the center’s staff should be restored to at least the 1992 levels of OERI staff, and additional distinguished research and development professionals should be recruited. The R&D centers, located in the new agency, need to grow and become more focused if they are to be relevant sources of research. Some centers survive on an annual budget of only $1.5 million to $2.5 million, hardly enough to support a legitimate large-scale program of research. An individual R&D center’s budget should be at least $5.0 million annually. Moreover, the centers need to develop a coherent, focused, coordinated five-year program of research. They should not, as is now the case, have 20 to 30 different small-scale, uncoordinated projects scattered among a half-dozen different centers across the nation.
Another independent agency needs to be created to initiate and oversee serious efforts to develop effective educational strategies and evaluate federal education programs-a National Center for Development and Evaluation, overseen by a commissioner appointed for a six-year term. Large-scale, rigorous development programs have not been a part of the department’s agenda. Likewise, the Department of Education has not conducted many scientifically sound, nonpartisan, objective program evaluations during the past two decades.
The program development and evaluation effort should be overseen by an independent, objective group of experts who will not only provide technical support, but will also ensure that the design, implementation, and interpretation of the work is scientifically sound as well as useful to educators and policymakers. Moreover, they will help to ensure that the national center’s development projects and evaluations will be readily and equally available to everyone-and not just to those who control the Department of Education at the time.
The new National Center for Development and Evaluation should solicit and implement large-scale, systematic development projects-a role that has been largely missing from the Department of Education’s agenda in recent decades. This initiative might focus its energies initially on three to five long-term projects in areas such as developing reading improvement programs or helping at-risk children make successful transitions from early-childhood programs into regular classrooms. Any research outfit, including the federally funded centers and labs, could compete for these demonstration projects. The open competition would not only spur existing education research and development providers to develop better proposals, but it might also attract interest from other major social science research organizations, such as the Manpower Development Research Corporation, the RAND Corporation, or the Urban Institute.
The new center would also be responsible for interpreting the effectiveness of the various strategies. The program evaluations will vary according to the types of information analysts are looking for. The most rigorous and statistically reliable studies will use randomized-assignment control groups-though the much higher costs associated with randomized experiments will limit the number of these studies.
The collection, analysis, and distribution of education statistics also need to be improved and protected from undue political interference. The National Center for Education Statistics should be continued, protected, and expanded as an independent entity with a commissioner of education statistics appointed for a six-year term. And the National Assessment Governing Board (NAGB), which now shares responsibility for overseeing NAEP, should be continued as an independent group and given additional resources to improve the quality of its work. In the past there has been some tension and confusion between NAGB and NCES over who administers the NAEP contract, how the NAEP scores are presented in terms of achievement levels, as well as when and how the results are disseminated to the public. The National Assessment Governing Board should be given the responsibility not only for setting NAEP policies, but also for administering and monitoring the NAEP contract, and releasing the results to the public. Currently, the National Assessment Governing Board sets NAEP policies, but does not monitor the contract for the program, and it should be given more control over the administration of NAEP so that the oversight and coordination of these vital assessments can be improved.
The Department of Education should continue to provide information and technical assistance to states, school districts, and local schools by funding education experts to help educators improve their overall operations or classroom teaching. But this important technical assistance work of OERI’s labs and the comparable and overlapping work of the recently created comprehensive regional assistance centers (currently overseen by the Office of Elementary and Secondary Education) should be merged and transferred to a new office of reform assistance and dissemination under an assistant secretary for planning and dissemination in the Department of Education. Moreover, states and local districts ought to be given more flexibility in deciding how to spend federal funds designated for technical assistance. They should be able to choose from among a variety of providers rather than being forced to receive those services from either the labs or the comprehensive regional assistance centers.
Same Old Song?
There has been no shortage of thoughtful suggestions for reforming federal R&D over the past three decades. Almost all reformers seem to call for more research funding; researchers with more training; more permanent and distinguished OERI leaders; more strategic planning to meet the needs of classroom teachers and students; more long-term, scientifically sound R&D projects; and preservation of the intellectual and political independence of the agency. Sound familiar? Indeed, most of these recommendations have made their way into the language of OERI’s periodic congressional reauthorizations.
Yet little seems to have changed. Structural weaknesses in the design of OERI, inadequate funding, and excessive congressional micromanagement all contribute to the inability of OERI to transform itself. But the agency’s leadership over the past 25 years must share some of the responsibility for its shortcomings; OERI directors have not always tried to recruit distinguished researchers or insisted on high-quality work from all of the agency’s grantees and contractors. Nor have all members of the education research community been sufficiently committed to making OERI a distinguished agency-especially when it meant subjecting their own federally sponsored work to rigorous evaluations or facing more frequent competitions for their funding.
Many in the education community are becoming impatient with the recitation of familiar promises to improve research and development in the near future. Unless policymakers transform OERI into a first-rate, high-quality R&D operation, some members of Congress might consider shifting some of the funding and responsibilities currently under OERI’s watch to other research and statistical agencies.
There are promising success stories in the remaking of federal efforts in education research. In the mid-1980s, the Department of Education commissioned the National Academy of Sciences to undertake a thorough review of the National Center for Education Statistics. The review panel was so disappointed with the NCES’s work that it recommended the center’s dissolution if corrective measures were not taken immediately. A few dedicated and talented individuals rose to the challenge. Working closely with the appropriate OERI staff as well as with several influential members of Congress, they managed within the space of only a few years to create an organization that is now acknowledged as a distinguished and effective federal statistical agency. Revamping OERI will take no less commitment and effort, but the result could be a rigorous and effective research agency that is able to really improve education for all of America’s children.
Maris A. Vinovskis is a professor of history at the University of Michigan and the author of Revitalizing Federal Education Research: Improving the Regional Educational Laboratories, the R&D Centers, and the “New” OERI (University of Michigan Press, 2001).
