During the 1970s, nearly every state in the nation began instituting tests of basic skills for high-school students as the leading edge of the so-called “first wave” of education reforms. These reforms were a response to the widespread impression that test scores and the quality of public schooling were in decline. According to critics, the high-school diploma, once a true accomplishment, had been debased in an era of social promotion and low standards—to the point where it held no real meaning for postsecondary institutions or potential employers.
The pace of reform was greatly accelerated with the release in 1983 of the blue-ribbon report A Nation at Risk. A chief cause of the nation’s educational decline, the report ventured, was the “cafeteria-style curriculum” that allowed students to pursue a diffuse and unchallenging course of study. The report recommended that states require students to take a minimum number of courses in core academic subjects in order to graduate from high school. As a result, by 1992 nearly every state had increased its graduation requirements in the core academic areas. However, only three states, Florida, Louisiana, and Pennsylvania, had met the standard recommended by the Risk report: four years of English and at least three years each of social studies, science, and math.
Such reforms, with their focus on testing and higher academic standards, are the precursors of today’s controversial accountability movement. Yet there has been surprisingly little study of their consequences. For instance, did the requirement that all students pass a minimum-competency test in order to graduate from high school encourage more students (in particular, minorities) to drop out of high school, as many critics feared? Did such exams, as supporters hoped, make the high-school diploma more valuable, thereby improving the job prospects of graduates?
These questions continue to be relevant as the states ramp up their testing programs and impose ever-harsher sanctions on failing schools and low-performing students. Judging from their continuing prevalence in policy debates, such questions also remain mostly unanswered.
Did requiring all students to pass a minimum-competency test in order to graduate from high school encourage more students to drop out?
Controversies
Minimum-competency exams were by far the more controversial of the two major first-wave education reforms. In the end, few critics object to tougher course requirements. The controversy arises when states attempt to test whether students have met a certain standard—and especially when they attach consequences to the results of such exams. In the beginning, most minimum-competency exams were intended simply to identify low-performing students and to direct them toward sources of remediation. However, several states also mandated that students pass the minimum-competency exam in order to graduate with a standard high-school diploma. By 1992, graduating high-school seniors in 15 states were required to pass a basic-skills test.
Typically, students would first sit for these exams in the 9th or 10th grade and enjoy multiple opportunities for retests. The conventional wisdom has been that these tests were “legislated as a lion but implemented as a lamb.” To wit, the exams were usually set at only the 8th- or 9th-grade level. The standards were sometimes set even lower in the face of politically unacceptable failure rates. As a consequence, the ultimate pass rates among high-school seniors were extremely high. The still-open question is whether these pass rates were artificially high, given that some students may have chosen simply to drop out if they thought they had little chance of passing the test.
Placing students at increased risk of dropping out is one of the major objections that critics lodge against any kind of testing regime that imposes harsh penalties on students. The intent of these reforms is both to raise academic standards and to give students the incentives to meet them. But critics say that standards-based reforms may simply exacerbate existing inequalities if sanctions are applied to low-performing students without giving them the resources and help they need to succeed. Furthermore, standards could lower the achievement of high-performing students if they signal that learning for its own sake is not worthwhile.
Economists Julian Betts and Robert Costrell have argued that students whose prior academic record provides a clear indication of how they will perform on the test (pass with flying colors or fail miserably) face little incentive to change their study habits. It is students at the margin who may feel most of the impact of a test- or curriculum-based sanction. Some will respond by redoubling their efforts to ensure that they pass the test or accumulate the proper number of credits in core academic subjects. Others may grow discouraged and eventually give up.
So the marginal students who choose to work harder may benefit from higher testing and curricular standards. High standards, argues Cornell economist John Bishop, may also shield bright students from the harassment and peer pressure that often accompany excelling in academics (as opposed to, say, athletics). After all, if all students must meet a defined external standard, there may not be much glory in failing and being subject to a consequence as harsh as not graduating. High standards may also generate broader educational gains as schools, teachers, and students struggle to attain them. Even students who fully expect to drop out may expend more effort in the short term as they are forced to progress through a state-mandated set of core academic courses. Thus both dropouts and graduates may reap the rewards of higher standards, if the standards signal to employers that the average ability of all students has increased.
To shed light on these debates, I examined national data on a variety of outcomes for students to assess the impact of the first wave of education reforms. The outcomes include both school-based achievements, such as students’ course-taking and their level of educational attainment, and students’ later experiences in the labor force.
Did such exams make the high-school diploma more valuable, thereby improving the job prospects of graduates?
Contradictions
The available evidence on the influence of minimum-competency exams and higher curricular standards on educational attainment and employability is not only scant but often contradictory. One possible explanation for these conflicting results is that almost all previous empirical studies have relied on comparisons of students in states that adopted these reforms with students in states that did not. But test scores and other outcomes related to education vary dramatically from state to state, often for cultural, socioeconomic, and political reasons that are difficult for researchers to measure directly. In addition, these factors, which are unique to each state yet difficult to account for, may influence a state’s decision to adopt reforms like minimum-competency tests and higher curricular standards. That makes it quite difficult to credibly isolate the effects of such reforms using traditional comparisons across the states.
To address these concerns, I used 1990 census data (the Public Use Micro-Data Sample), which allowed me to construct before-and-after comparisons of students from within the same states. I compared individuals of different ages, some of whom attended high school before the first-wave education reforms were implemented and others who attended afterward—and were thus required to take minimum-competency exams and more courses in academic areas in order to graduate. I made these comparisons by matching each state’s graduating class to the policies that were in effect the year they graduated—whether students had to take a minimum-competency exam and whether they faced high course requirements in order to graduate. High course requirements were defined as a high-school curriculum that includes at least three years of English, two of social studies, one of science, and one of math. In the initial analysis, I assessed the impact of these reforms on the chance that students would graduate from high school and enter college. I then examined their effects on subsequent participation in the labor market and average weekly wages as adults.
In order to eliminate the influence of other national trends, I compared the changes within states that introduced first-wave reforms with the contemporaneous changes in states that did not. I also adjusted the data in both analyses to account for a variety of other within-state factors that changed over time and could have influenced student outcomes. For instance, first-wave reforms were sometimes part of omnibus education bills that included other policy changes, such as increased spending on K–12 education. To address this, I controlled for real per-pupil spending in the state when the survey respondents were 16 to 17 years old (in other words, respondents who were 18 years of age in 1980 were matched to their states’ school spending during the 1978–79 school year). My results are also adjusted for the state unemployment rate when respondents were 17 years old, since high unemployment rates can provide an incentive for students to stay in school.
There is also evidence that educational attainment can be influenced by the relative size of a birth cohort. At the college level, this can occur if enrollment space at local colleges and universities is not expanded to accommodate temporary population booms. At the secondary level, larger student populations may increase class sizes and strain school resources, thereby lowering school quality and reducing the benefits of staying in school. For these reasons I introduced an adjustment for the size of students’ birth cohorts. I also included a measure of the real costs of postsecondary tuition based on the in-state rate at “lower level” state colleges and universities when the respondents were 17 years old. Finally, as a control for within-state changes in socioeconomic conditions, I matched respondents to the poverty rates in their states when they were 17 years old.
One further note: I performed several tests to check whether my results were indeed more reliable than results using comparisons of students across states rather than within the same state. One method is to estimate the effect of a state-level policy that, in theory, should be totally irrelevant to educational outcomes. In this case I chose whether a state had any executions when the student was 18 years of age. In the across-state comparison, capital punishment generated large and statistically significant reductions in the probability of high-school completion and college entrance, clearly an implausible result. By contrast, the effects of capital punishment as estimated in the within-state comparisons were negligible, as they should be. This strongly suggests the increased reliability associated with using within-state comparisons.
Higher course requirements significantly reduced the probability of graduating from high school for blacks and for white males.
Education and Employability
For all students, the minimum-competency exams showed no statistically significant effect on the probability of graduating from high school. By contrast, higher curricular standards reduced students’ probability of graduating from high school by 0.5 percentage points (see Figure 1). An effect of this size represents a 3 percent increase in the probability of dropping out. Another way to frame the size of this estimate is to note that high-school completion rates among 18- to 24-year-olds increased from 82.8 percent in 1972 to 86.5 percent in 2000. My results suggest that in states that adopted high curricular standards these average gains were 14 percent less than they otherwise would have been. (Note that these percentages—as well as the data set I used—considered students who passed the General Educational Development, or GED, test to be high-school graduates. Thus my findings may actually understate the true reform-induced reduction in high-school completion.)
For all students and for subgroups broken down by race and gender, the first-wave reforms had statistically insignificant effects on the probability of entering college. This makes sense since these reforms were not aimed at the relatively high-achieving students who are considering college.
However, in looking at the results for students separated by race and gender, both types of reforms had fairly large and statistically significant effects on the probability of completing high school for some groups. For instance, higher course requirements significantly reduced the probability of graduating from high school for blacks and for white males, but not for white females. Among black students, higher curricular requirements reduced the probability of graduating from high school by roughly 2 percentage points—four times the impact that these reforms had on white males. Similarly, the only statistically significant effect of introducing a minimum-competency exam was among black males, who experienced an estimated 1.3 percentage point drop in the probability of completing high school. These estimated effects are fairly large relative to the recent growth in educational attainment among blacks. Between 1972 and 2000, the high-school completion rate of 18- to 24-year-old black students increased from 72.1 percent to 83.7 percent. (Once again, these percentages include GED recipients.)
These findings are largely consistent with the concerns sometimes raised by critics of standards-based reforms. The introduction of high-school graduation standards led to reductions in educational attainment that were particularly concentrated among minority students. But a full evaluation should also consider the implications of these reforms for students’ performance in the labor market, especially since local business leaders concerned with the quality of their workforce were often instrumental in the adoption of first-wave reforms. As noted earlier, higher standards may benefit students (even those who drop out) by creating an incentive to work harder in school, an effort that is rewarded in the labor market. Higher standards may also increase the prestige of being a high-school graduate and correspondingly reduce the stigma associated with being a dropout.
My findings suggest that both first-wave education reforms had no statistically significant effects on adult wages for all groups. However, the effects on adult employment were more substantial (see Figure 2). Higher course requirements appeared to increase the probability of adult employment by roughly 1 percentage point for white males and by 3 percentage points for blacks, even after controlling for educational attainment. Furthermore, the introduction of minimum-competency exams increased the probability of adult employment among black males by 1.6 percentage points. One useful way to underscore the magnitude of these policy effects is to note that, in these data, white males were roughly 19 percentage points more likely to be employed than black males. Since the employment gains attributable to each first-wave reform were roughly 2 percentage points larger for black males than for white males, states that implemented them closed the black-white employment gap by roughly 11 percent.
Taken together, these results indicate that higher course requirements had educational and labor-market consequences for almost all students. The effects of minimum-competency exams, by contrast, were narrower, observable only among black males. These results are consistent with the anecdotal evidence suggesting that minimum-competency exams were often weakened in response to political realities. These results also suggest that, when binding, higher standards of either type had decidedly mixed consequences. They had negative effects on students’ educational attainment, particularly among black students. But they also generated some employment rewards, again mainly for black students.
How can we compare these gains and losses? For those students who dropped out only in response to the new requirements, the loss was large: according to these data, high-school graduates can expect to earn approximately 33 percent more than the average dropout. This was only partially offset by improved prospects for employment. However, those students who would have dropped out regardless of the graduation requirements experienced significant gains because they were more likely to be employed.
Another way to frame the question is to ask how either type of reform might change the expected wage for students who were uncertain about whether they would be high-school graduates. My back-of-the-envelope calculations indicate that the expected wage benefits of higher curriculum standards due to their positive effect on employment exceed the expected wage costs due to their adverse effect on graduation rates by a factor of six. This suggests that a risk-neutral person might prefer a regime with higher curricular standards. However, a full cost-benefit analysis would need to account for not only the labor-market consequences but also the disparate impacts on various ethnic and socioeconomic groups and other social losses that might accompany increased dropout rates.
Higher standards contributed substantively to the upgrading of high-school curricula, particularly in English and the sciences.
Standards on the Ground
How did the first-wave education reforms influence students’ course-taking in high school? During the 1980s, national data show that course-taking in the core academic subjects did increase significantly, particularly in mathematics and science. Yet the extent to which this increase was driven by states’ new graduation requirements remains unclear.
Also at issue is the impact of these reforms on academic engagement. Critics have suggested that minimum-competency exams and stricter coursework requirements will indicate to students that learning for its own sake is not worthwhile. In particular, these standards may encourage otherwise high-achieving students to avoid challenges and simply choose the path of least resistance to satisfy their requirements. Two decades ago, the authors of A Nation at Risk expressed a similar worry, suggesting that minimum-competency exams ran the risk of becoming “maximum” standards and thus lowering expectations for high-ability students.
To investigate the claim that course requirements and basic-skills tests may dampen the efforts of high-ability students, I analyzed whether higher graduation standards did indeed encourage students to take more courses in the core academic areas. I also investigated whether these requirements narrowed curricular effort by looking at students’ course-taking in the visual and performing arts. To explore how these reforms may have influenced the intellectual engagement of students, I examined changes in time spent reading for pleasure, watching television, and doing homework.
The data for these evaluations were created by pooling observations from two of the National Center for Education Statistics’s major longitudinal studies: the sophomore cohort from the High School and Beyond study and the National Education Longitudinal Study of 1988. These surveys provide student-level data for nationally representative samples of 10th graders in 1980 and 1990—before and after the period when most first-wave reforms were implemented. Combining these data sets allowed me to adopt an analytic approach similar to the one I used to examine the effects of more stringent graduation requirements on educational attainment and employment.
My results uniformly suggest that the introduction of minimum-competency exams reduced course-taking in the four main academic areas, though these findings were statistically significant only in the sciences and in mathematics. This evidence suggests a lowering of intellectual aspirations among students who clearly exceeded the testing standards. However, an alternative interpretation is that schools and teachers redirected their efforts to respond to the needs of lower-performing students. Or it could simply reflect a reduction in curricular effort among those eventually encouraged by these reforms to drop out. But other evidence suggests that this is not the case. For example, the introduction of minimum-competency exams only increased the probability of dropping out among black males. But the reductions in course-taking associated with minimum-competency exams were more uniform across demographic traits and were particularly large for female students. Furthermore, the new tests also led to large and statistically significant reductions in credits for taking calculus, a margin relevant only for high-achieving students.
In contrast to minimum-competency testing, higher course requirements appeared to generate significant increases in the number of credits students earned in most of the core areas. For example, a high course requirement increased the number of credits earned in science by almost 0.4 Carnegie units, or 16 percent. Only in social studies was the estimated effect of a high course requirement statistically insignificant. These results suggest that higher standards contributed substantively to the upgrading of high-school curricula over this period, particularly in English and the sciences. The effect associated with higher course requirements, for instance, is equivalent to roughly 60 percent of the average growth in science credits over this period.
Were these gains in the core areas achieved at the expense of other valuable subjects, such as the visual and performing arts? The evidence is not entirely clear since the effects of first-wave reforms on students’ participation in school arts and music activities were statistically indistinguishable from zero. However, I found that the first-wave reforms did adversely influence the amount of time students spent reading for pleasure, watching television, and doing homework. Higher course requirements were associated with large and statistically significant reductions in the amount of time spent reading for pleasure and doing homework and a corresponding increase in television watching. These results suggest that course-taking standards harmed the intellectual engagement of students. However, the apparent reduction in time spent on homework also raises the possibility that newly required courses were more demanding in prospect than in reality.
Lessons
In the end, these findings indicate that the first wave of education reforms elicited many of the positive and negative consequences predicted by supporters and opponents. For example, both minimum-competency testing and increased coursework requirements led to reductions in educational attainment that were most pronounced among black students. Furthermore, minimum-competency testing led to some apparent reductions in the amount of courses completed, while increased coursework requirements appear to have had modest pejorative effects on the amounts of time students spend watching television, doing homework, and reading for pleasure. However, both types of reforms also increased students’ subsequent likelihood of being employed. And higher coursework requirements were partially responsible for the apparent upgrading of high school curricula during this period. In light of these mixed results, what can the nation’s experience with earlier reforms contribute to the current discussion of standards, testing, and accountability?
One lesson from our experiences with first-wave reforms is that standards-based reforms can generate a variety of costs and benefits, which can be compared only on subjective grounds. Some proponents of standards argue that we should implement targeted initiatives that soften the harsh consequences of standards by helping those at increased risk. However, any enthusiasm for that approach should be tempered by the knowledge that many of the state-level first-wave reforms were actually bundled with such efforts.
Our experiences with first-wave reforms also provide some suggestive evidence on the design of standards. In particular, these findings suggest that so-called “process” standards, such as higher course requirements, may be more effective than test-based standards, at least when testers seek only to establish a minimum level of performance necessary to graduate. While improved curricular standards yielded benefits in employment and in course-taking, minimum-competency testing had relatively few of the desired effects on educational attainment and early labor-market experiences. These results are consistent with the widely held perception that these test-based standards were often quite weak because of political pressures and the relatively easy and veiled manner in which they could be subsequently lowered. In contrast, newly introduced course requirements for graduation created binding standards for students and were also largely immune to political redesign.
Of course, whether these comparative first-wave results can be applied to the current generation of accountability systems is clearly open to conjecture. Perhaps the strategy of testing and tracking students’ progress throughout their education will prove more effective than a single high-stakes exam at the end. But the early state-level experiences with minimum-competency testing provide a promising yet cautionary tale.
Thomas S. Dee is an assistant professor of economics at Swarthmore College and a faculty research fellow at the National Bureau of Economic Research.
