As the debate over the reauthorization of No Child Left Behind (NCLB) makes its murky way through the political swamp, one thing has become crystal clear: Though NCLB requires that virtually all children become proficient by the year 2014, states disagree on the level of accomplishment in math and reading a proficient child should possess. A few states have been setting world-class standards, but most are well off that mark—in some cases to a laughable degree.
In this report, we use 2007 test-score information to evaluate the rigor of each state’s proficiency standards against the National Assessment of Educational Progress (NAEP), an achievement measure that is recognized nationally and has international credibility as well. The analysis extends previous work (see “Johnny Can Read…in Some States,” features, Summer 2005, and “Keeping an Eye on State Standards,” features, Summer 2006) that used 2003 and 2005 test-score data and finds in the new data a noticeable decline, especially at the 8th-grade level. In Figure 1, we rank the rigor of state proficiency standards using the same A to F scale teachers use to grade students. Those that receive an A have the toughest definitions of student proficiency, while those with an F have the least rigorous.
Measuring Standards
That states vary widely in their definitions of student proficiency seems little short of bizarre. Agreement on what constitutes “proficiency” would seem the essential starting point: if students are to know what is expected of them, teachers are to know what to teach, and parents are to have a measuring stick for their schools. In the absence of such agreement, it is impossible to determine how student achievement stacks up across states and countries.
One national metric for performance does exist, the National Assessment of Educational Progress. The NAEP is a series of tests administered under the auspices of the U.S. Department of Education’s National Center for Education Statistics. Known as the Nation’s Report Card, the NAEP tests measure proficiency in reading and math among 4th and 8th graders nationwide as well as in every state. The NAEP sets its proficiency standard through a well-established, if complex, technical process. Basically, it asks informed experts to judge the difficulty of each of the items in its test bank. The experts’ handiwork received a pat on the back recently when the American Institutes for Research (AIR) showed that NAEP’s definition of “proficiency” was very similar to the standard used by designers of international tests of student achievement. Proficiency has acquired roughly the same meaning in Europe and Asia, and in the United States—as long as the NAEP standard is employed.
This is not to say students are proficient either in this country or elsewhere. According to NAEP standards, only 31 percent of 8th graders in the United States are proficient in mathematics. Using that same standard, just 73 percent of 8th graders are proficient in math in the highest-achieving country, Singapore, according to the AIR study. In other words, bringing virtually all 8th graders in the United States up to a NAEP-like level of proficiency in mathematics constitutes a challenge no country has ever mastered.
Comparing the States
Three states—Massachusetts, South Carolina, and Missouri—have established world-class standards in math and reading as the goal for all students. Every other state has established a lower proficiency standard, and some states (for example, Georgia and Tennessee) declare most students proficient even when their performance is miles short of the NAEP standard. By setting widely varying standards, states render the very notion of proficiency meaningless. If Billy and Sally cannot read in South Carolina, they should not be able to pass muster simply by crossing the state’s western border.
We gauge the differences among states by comparing how students do on state assessments with how they perform on NAEP tests. By comparing the percentage of students deemed proficient on each, it is possible to determine whether states are setting expectations higher, lower, or equal to the NAEP standard. If the percentages are identical (or roughly so), then state proficiency standards can be fairly labeled as “world-class.” If state assessments identify many more students as proficient than the NAEP, then state proficiency figures should be regarded as inflated. In short, comparing state assessment results to NAEP scores can help reveal whether states are giving parents and voters the real scoop about where the state’s children stack up when measured against world-class benchmarks.
In Figure 1, we give Massachusetts, Missouri, and South Carolina an A for establishing rigorous expectations regarding what proficient students must know and be able to do. Note that a grade of A does not indicate students are performing at the highest level. Rather, the high grade indicates that the three states have set a high bar for students to reach if they are to be deemed proficient. So, for example, only 25 percent of 8th graders in South Carolina were deemed proficient on both the state reading test and on the NAEP reading test—an honest, if embarrassing, reckoning of the education situation in the state.
The remaining 47 states (information is not yet available for the District of Columbia) had distinctly lower standards. Three states—Georgia, Oklahoma, and Tennessee—expected so little of students that they received the grade of F. The state of Georgia, for instance, declared 88 percent of 8th graders proficient in reading, even though just 26 percent scored at or above the proficiency level on the NAEP. According to our calculations, Georgia 8th-grade reading standards are 4.0 standard deviations below those in South Carolina, an extraordinarily large difference. Thus, while students in Georgia and South Carolina perform at similar levels on the NAEP, the casual observer would be misled by Georgia’s reporting that its students achieve proficiency at three times the rate that South Carolina’s students do.
Twelve states—Alabama, Alaska, Idaho, Illinois, Michigan, Mississippi, Nebraska, North Carolina, Texas, Utah, Virginia, and West Virginia—received Ds because they had pitched their expectations far below other states. Illinois set its proficiency bar for 8th-grade reading at a level that is 1.01 standard deviations below the national average. If you believe those who set the Illinois standards, 82 percent of its 8th graders are proficient in reading, even though the NAEP says only 30 percent are.
In general, the states of the Northeast have the highest standards, while the states of the South and Midwest have the lowest. Western states fall in between.
A Downward Trend
There is some evidence of slippage in standards since our original report card was published in 2005 (see Figure 2). In 8th-grade reading, for example, standards overall are down by 0.2 standard deviations. This means that, in 8th-grade reading, states are reporting a substantial improvement that is not evident on the NAEP. The smallest amount of slippage was in 4th-grade math, where standards fell by 0.06 standard deviations. Most of the slippage at the 4th-grade level is due to the lower standards adopted by those states that were initially slow in complying with the NCLB accountability system; those that have had standards since 2003 have not altered them significantly. But at the 8th-grade level, standards are falling across the board—in both reading and math, and among both the states that had standards in 2003 and the states that have only adopted them more recently.
We also see slight convergence among the states. For example, the variation in 4th-grade math standards narrowed 0.11 standard deviations between 2003 and 2007. The good news is that differences among state standards are shrinking; the bad news is that states are converging downward, not upward.
By and large, the changes that are taking place in individual states are fairly small, perhaps so they do not stir controversy. A few states, though, have made big adjustments since 2003. Colorado and Texas have raised their proficiency bars enough to warrant a grade one letter better than the one given initially. Five states—Arizona, Illinois, Maine, Michigan, and Wyoming—have lowered the bar enough that their grades have dropped by a full letter.
| Grading Procedure In 2003, 2005, and 2007, both state and NAEP tests were given in math and reading for 4th- and 8th-grade students. The grades reported here are based on the comparison of state and NAEP proficiency scores in 2007, and changes for each are calculated relative to 2003. For each available test, we computed the difference between the percentage of students who were proficient on the NAEP and the percentage reported to be proficient on the state’s own tests for the same year. We also computed the standard deviation for this difference. We then determined how many standard deviations each state’s difference was above or below the average difference on each test. The scale for the grades was set so that if grades had been randomly assigned, 10 percent of the states would earn As, 20 percent Bs, 40 percent Cs, 20 percent Ds, and 10 percent Fs. The grade given each state is based on how much easier it was to be labeled proficient on the state assessment compared with the NAEP. For example, on the 4th-grade math test in 2007, South Carolina reported that 41.4 percent of its students had achieved proficiency, but 35.9 percent were proficient on the NAEP. The difference (41.4 percent — 35.9 percent = 5.5 percent) is about 1.6 standard deviations better than the average difference between the state test and the NAEP, which is 32 percent. This was good enough for South Carolina to earn an A for its standards in 4th-grade math. The overall grade for each state was determined by taking the average for the standard deviations on the tests for which the state reported proficiency percentages. |
Two years ago, we could see small evidence for a decline in standards but detected no race to the bottom. That is still true for 4th graders. But 8th-grade standards, if not exactly racing downward, are moving steadily away from world-class standards. Those responsible for NCLB reauthorization, as they struggle forward, should first and foremost establish a clear and consistent definition of grade-level proficiency in reading and math, even if it means giving up the cherished but decidedly unrealistic goal of proficiency for all students by 2014.
Paul E. Peterson and Frederick M. Hess are editors of Education Next.
