 |
Why don’t American high-school students perform as well as their peers in other industrialized nations? One reason is that they devote less time and intellectual energy to their schooling. Learning takes work, and that work is generally not going to be as much fun as hanging out with friends or playing Grand Theft Auto. The question is how to pull them away from such distractions in favor of studying longer and harder. The strategy followed by nations throughout East Asia and much of Europe has been to base admission to specific universities and academic programs (such as law and medicine) largely on students’ performance on a battery of subject-specific examinations devised by a nation’s ministry of education. These high-stakes exams are very different from the multiple-choice aptitude tests-the SAT-1 and the ACT-that serve a similar function here. Each subject exam is three hours or more, and students write essays or solve multistep problems, showing their work. My analysis of data from the Third International Mathematics and Science Study (see “A Steeper, Better Road to Graduation,” Feature, Winter 2001) demonstrates that students in countries with such high-stakes exit exams outperform students in other, equally developed countries by 1.3 grade levels in science and 1.0 grade level in math.
The idea of a government-sponsored exam limiting access to postsecondary education has never been popular in America, the land of second chances. In fact, the American higher education system’s open-door admissions policies are a major strength of the U.S. economy, enabling adults of any age, whatever their background or mistakes in life, to return to school and gain new skills. Nevertheless, the problem of how to motivate students remains. The strategy in 17 states, such as Florida, Texas, and New Jersey, is to require students to pass a minimum-competency exam in order to graduate from high school. These are tests of basic skills, and the passing scores are typically set so low that the possibility of failing the tests does not motivate most students to try harder. New York State and North Carolina, by contrast, give students an incentive to study through the use of rigorous end-of-course exams that signal medium and high achievement levels, not just meeting minimum standards. The results of New York’s famed Regents exams are reported on students’ high-school transcripts, and students earn special Regents diplomas if they pass enough of the tests. Colleges consider the Regents diploma a mark of significant achievement, making it worth students’ while to learn the tested subjects. The Regents exams give students a lofty goal to aspire to, rather than a low hurdle to jump over.
The state of Michigan rejected the use of minimum-competency exams, largely because it wanted the state’s high-school test to reflect more challenging learning goals. Michigan policymakers also did not want to take the risk that a high-stakes exam would lower the rates of high-school graduation and college attendance. Instead, similar to New York and North Carolina, the state took the modest step of reporting students’ scores on their high-school transcripts.
In 1999, Michigan increased the reward for good academic performance by offering the Michigan Merit Award, a one-year $2,500 scholarship for any student who scores at Level I or Level 2 on the Michigan Educational Achievement Program (MEAP) tests in reading, mathematics, science, and writing. Students take these tests for the first time in the spring of their junior year. They can also earn the scholarship by meeting the standards in two subjects and scoring in the 75th percentile or higher on the SAT or the ACT. Students who attend college in Michigan are eligible for the full $2,500 scholarship, while students going to college out of state can receive up to $1,000. Starting with the class of 2005, students can be awarded up to $500 more if they meet or exceed state standards on two of the four MEAP exams in 7th and 8th grade as well. College financial aid officers are prohibited from taking the merit award into account when they make need-based awards, meaning that students who earned the scholarship would not have their need-based grant aid reduced, as is common. A number of other states also have merit-based scholarship programs that are open to all students, but the awards are typically less generous and are based not on tests developed by the states, but on grades or college-admissions test scores.
Scholarship programs like Michigan’s carry a low cost-benefit ratio. The total expense for the Michigan Merit Award comes to less than 1 percent of the state’s K-12 education spending, yet the program has the potential to realign incentives within the school system in a way that serves the interests of students, parents, educators, and the community. It encourages students to study harder; enhances their opportunity and willingness to go on to college; improves the learning climate in most schools; and strengthens the energy for reform among parents and teachers. What makes the Michigan program so powerful is that the scholarships are based on students’ performance on an external exam that reflects the state’s recommended curriculum. If the awards were based instead on, say, high-school grades, many students would respond by choosing easy courses where an A is guaranteed. Teachers would face little incentive to set higher standards; in fact, the pressure to inflate grades would intensify. If the awards were based primarily on SAT or ACT scores, the main result would be an increase in Kaplan’s revenues from test-prep courses. The only way to win the award is to learn the curriculum well enough to pass the state’s high-school test. Unless the curriculum is misguided, this focuses everyone in the system on the main goal of K-12 education.
Unvarnished Benefits
Before the advent of the scholarship program, students and parents in Michigan saw few short-term, tangible benefits when school districts opted for higher standards, more qualified teachers, or a heavier workload for students. The immediate consequences of such decisions-higher taxes, more homework, lower grade point averages (GPAs), a greater risk of being denied a diploma-were negative. As a result, parents pressured teachers to be easy graders and taxpayers were reluctant to pay them well enough to recruit highly qualified personnel. The Michigan Merit Award makes everyone stronger advocates of higher standards and better teaching.
The fact that the scholarships go to every student who meets or exceeds the absolute standard also enhances the classroom environment. Competitive merit scholarships have the dysfunctional effect of pitting classmates against one another. The winners of these traditional merit scholarships are seen as nerds, suck ups, or “Oreos” by their classmates. That is why many schools stopped awarding these honors at school assemblies. There were too many incidents of catcalls mixed with unenthusiastic applause. The merit award helps to reduce peer pressure against students who perform well academically. Students who joke around in class will no longer be honored and rewarded by their peers because their disruptions make it harder for the rest of the class to get the $2,500 award. The fact that the cut score for the merit award was set at a level that is achievable by almost all students makes everyone believe that it is an attainable goal, one worth striving for.
The merit award program enhances the state’s ability to hold schools accountable for student achievement. When the MEAP high-school exam was a no-stakes test, students had no reason to try their best on the primary indicator of performance in the state’s high-school accountability system. Many students were boycotting the test. School ratings thus reflected, in part, a school’s success in getting students to put in some effort on the tests. This reduced the validity of high-school tests as measures of student achievement. If the state had turned the MEAP into a high-stakes exam by making high-school graduation contingent on passing it, this would have affected only those students at the bottom of the achievement curve. The merit award has given students across the board an incentive to do the best they can on the high-school test, thus improving the fairness, validity, and effectiveness of the state’s school accountability system.
Increasing the stakes attached to the MEAP assessment also improved the academic environment by focusing the efforts of teachers and students on a good test. In fact, the MEAP is a much better exam than the tests that most teachers develop for themselves and use to grade their students. It is the product of an extensive consultative process, for which input was obtained from hundreds of well-regarded teachers. All test questions are pretested and reviewed for ambiguity and bias by trained testing professionals. By contrast, studies of tests developed by teachers have found that the overwhelming majority of the questions were created to tap the lowest of Bloom’s taxonomic categories: knowledge of terms, facts, or principles. A 1987 study found that less than 20 percent of the questions developed by secondary-school history teachers required the integration of ideas. College instructors required such integration in 99 percent of their test items. Most secondary-school teachers test for low-level competencies because that is what they teach. The MEAP high-school test pushes things in the other direction.
One final benefit of the merit award program is that it tends to redirect students away from preparing for high-stakes multiple-choice tests like the SAT-1 and the ACT. This is a good thing because the ACT and the SAT-1 are not comprehensive measures of a well-rounded secondary education. Both of these admissions tests fail to assess most of the material-economics, civics, literature, foreign languages, and the ability to write an essay-that high-school students are expected to learn. The energy that students devote to cracking the SAT-1 and the ACT would be better spent reading widely and learning to write coherently, to think scientifically, to analyze and appreciate great literature, and to converse in a foreign language. The MEAP tests have been developed with great care and are far superior to the ACT, yet, despite the existence of the scholarship program, the ACT continues to carry much higher stakes for Michigan students because it is the primary admissions test used by Michigan’s colleges.
Evidence from Canada and Michigan
Michigan’s scholarship program is still young. Therefore, we must look elsewhere for evidence of its likely long-term effects on student performance. Analyses of high-stakes exam systems around the world have demonstrated that they lead to significant achievement gains, but these are not comparable to the moderate-stakes system in Michigan. Canada affords a more relevant example. In the early 1990s, the provinces of Alberta, British Columbia, Newfoundland, Quebec, and Francophone New Brunswick administered curriculum-based examinations in English during students’ junior year of high school and in French, mathematics, biology, chemistry, and physics during their senior year. Students’ performance on the exams accounted for 40 to 50 percent of their final grade. This was only a moderate stake because college-admissions decisions were based almost entirely on high-school grades and were generally made before the senior-year exams were graded. Nevertheless, studies have found that, after controlling for the size and structure of the school and the social background of its students, schools in provinces with external exams taught their students a statistically significant one-half of a U.S. grade-level equivalent more math and science by 8th grade than comparable schools in provinces that did not give curriculum-based external tests.
Schools in provinces that used external exams were also more likely to have teachers who specialized in teaching one subject in middle school and to hire teachers who majored in the subject they will teach. Schools in these provinces devoted more hours to math and science instruction and built and equipped better science labs; their students were more likely to do experiments in science class and to agree with the statement that science is useful in everyday life. Students also talked with their parents more often about schoolwork and reported that their parents had more positive attitudes about the subject. The success of these exams led other provinces-Manitoba and Anglophone New Brunswick-to introduce curriculum-based external exit exams in the late 1990s.
Can we expect similar effects in Michigan? The early signs are promising. Before the introduction of the merit award, just 75 percent of public high-school juniors were taking the MEAP tests. When Michigan’s scholarship program became effective for high-school juniors who took the state tests in the spring of 1999, participation immediately jumped by 10 percentage points. Participation rates continued to climb thereafter, reaching 99.7 percent of seniors in the class of 2002.
More important is the substantial increase in the number of students meeting or exceeding Michigan’s education standards. Test scores usually decrease as participation improves, yet in math the proportion of test-takers meeting the standard rose from 61 percent in the spring of 1998 to 67 percent for the class of 2002. In reading, the proportion increased from 59 percent in 1998 to 71 percent for the class of 2002 (see Figure 1). Similar improvements were seen in writing and science. The proportion of the graduating class passing all four MEAP tests and winning a merit award increased by 7 percent from 2000 to 2002. Forty-six percent of the public and private school students who took all four tests got a merit award in 2000.
Michigan students have also improved their standing on other tests. Average scores on the ACT test were constant among Michigan students, while the nation’s average score declined by 0.2 points. On the SAT-1, Michigan’s scores on the combined verbal and mathematics tests rose 5 points more rapidly than for the rest of the nation.
There is also good news about Michigan’s high-school graduation rate. Nationally, graduation rates fell slightly during the 1990s; the ratio of the number of regular diplomas awarded to the number of 10th graders enrolled three years earlier declined from 77 percent in 1991 to 76 percent in 2001. By contrast, Michigan’s regular diploma to 10th-grade enrollment ratio rose from 78 percent in 1991 to 83 percent in 2001. Unlike Ohio and Indiana, its neighbors to the south, Michigan did not implement a high-school graduation test. This decision has apparently paid off. Indiana’s diploma to 10th-grade enrollment ratio declined from 78 percent in the five-year period before implementation of the graduation test in 2000 to 77 percent in 2001.
Since Michigan’s scholarship program enhances a student’s ability to attend college, one might expect that it would also increase college-attendance rates. This is what happened in Georgia, particularly at colleges in the state, after the creation of the Hope scholarship program. However, the Hope scholarship is considerably more generous than Michigan’s award. The Hope scholarship lasts four years (if you maintain at least a 3.0 GPA), while the merit award is good for only one year. The Hope scholarship also pays full tuition at Georgia public universities and community colleges and a similar amount at private colleges in Georgia. Michigan’s $2,500 award covers only about half of average tuition and fees at Michigan’s public four-year colleges. Consequently, one would expect a smaller response to the Michigan Merit Award than to the Hope scholarship. In any case, the data necessary to determine conclusively whether the merit award influenced college attendance are not yet available.
Students who blow off high school pay a very high price, much larger than they imagine when they are in school. They believe they will be able to attend college despite low grades and low achievement. But in fact their chances of completing a degree program are almost zero. They are also unaware that applying themselves in high school helps them get jobs that offer training and promotion opportunities and eventually higher wage rates. States that desire to focus students’ attentions would be wise to consider merit-based scholarship programs like those in use in Georgia and Michigan. They offer little in the way of downside risk, and they have the potential for wide-ranging benefits.
-John H. Bishop is a professor of human resource studies at Cornell University.
