A small storm has blown up around the fact that certain math items on the 2015 National Assessment of Educational Progress (NAEP) do not align with what fourth and eighth graders are actually being taught in a few states—mainly places attempting to implement the Common Core State Standards within their schools’ curricula.
NAEP is only administered in grades four, eight, and twelve. So the specific issue is whether the fourth graders who sat for NAEP this spring had a reasonable opportunity to learn the skills, algorithms, techniques—broadly speaking, “the content”—on that test. If their state standards had moved some portion of what used to be fourth-grade math to the fifth or sixth grade, or replaced it with something else entirely, their state’s NAEP scores would likely be lower.
This kind of misalignment is blamed for some of the math declines that NAEP recently reported. Department officials in Maryland, for example, examined the NAEP math sub-scores and determined that many Maryland fourth graders are no longer being taught some of those things before they take the test.
We are left to wonder: Should NAEP frameworks and assessments be updated to reflect what’s in the Common Core, and perhaps other recent curricular developments?
Such questions arise periodically in all the subjects that NAEP assesses. They’re particularly acute in math, though, because the field is more “cumulative” than most disciplines; because the Common Core did make some significant changes in content emphasis and sequencing; and because math is one of only two subjects that are both included in the Common Core and part of the elaborate federal and state accountability structures that sit atop K–12 academic standards.
Although the NAEP math assessment is now given every two years, the framework on which it is based hasn’t changed much for grades four and eight since the early 90s. Achievement since then can therefore be reported as part of a continuous trend line stretching back a quarter-century. (The twelfth-grade math framework was changed in 2005—which, to use NAEP jargon, “broke the trend line” and started a new one, although some special analyses make it possible to connect the two.)
Just how closely aligned is the present NAEP math framework to the Common Core? A panel of experts convened by the American Institutes of Research (AIR) has just analyzed that very question and found “reasonable agreement.” That works out to 79 percent overlap in fourth grade and 87 percent in eighth, though the degree of alignment varies widely depending on the specific math topic.
Changing a NAEP framework is a huge deal—expensive, contentious, and slow—as well as amajor policy decision because of the issue of trend lines. Which brings us to the essential issue: Is NAEP intended to mirror what’s being taught today, or is it intended to monitor achievement over time? Is it an assessment of the efficacy of the present curriculum? Or should we use it as a barometer of how well today’s curriculum and instruction prepare students, as measured by a preexisting, unchanging standard?
Alongside “main NAEP,” there’s something called the “long-term trend NAEP,” the frameworks and content of which haven’t changed since the early 70s. That perspective is surely valuable to have. But because NAEP didn’t start providing state-specific data until the early 90s, we depend on main NAEP to know whether states and cities are doing any better—any better than before, any better in comparison with one another, and any better in relation to Singapore or Germany (once the statisticians work their magic and interconnect NAEP results with PISA or TIMSS).
This sort of sort of information is only knowable, obviously if the metric is stable. And that’s only possible when the tests don’t change. (The items change, of course, but not what’s being tested.)
As has no doubt become clear, I’m for stability here. It’s critical that NAEP’s math (and reading and writing) frameworks not flex with recent changes in standards, curriculum or pedagogical emphasis. That doesn’t mean they shouldn’t be reviewed from time to time, and it certainly doesn’t mean that NAEP is static; at the moment, for example, it’s in the middle of a complex and costly transition from paper-and-pencil to digital testing.
And stability is especially important today, because it’s never been more important that NAEP remain a trustworthy audit of American public education as it changes in all sorts of different ways. If the audit itself is revised to incorporate the Common Core, how will we know if Common Core states make greater gains than non-Common Core states? How will we know whether states and cities that faithfully implement their new standards are faring better than those that only pay lip service to them? How will we know whether places with lots of school choice are producing higher achievement than those without, or which of the state-specific accountability regimes likely to follow ESEA reauthorization are working best? How about correlations between spending and achievement? Between widespread reliance on technology and old-fashioned flesh-and-blood instruction? NAEP isn’t an experiment and therefore does not prove causation. But it’s the only reliable inventory of whether outcomes in American education are strengthening or weakening at the jurisdictional levels that matter most.
When we change the frameworks, we lose all that. It’s the same as replacing the yardstick, moving the goalposts, or sending in a different movie reviewer.
If every U.S. state and city were instituting the exact same procedures, then maybe we could constructively revise NAEP to align more precisely with the things they’re all doing. But when they’re reforming in so many different ways, NAEP mustn’t budge. The yardstick must remain thirty-six inches long.
(And that’s without even getting into the heinous politics that would surely follow if anyone made a serious move to remodel NAEP along the contours of the Common Core. Yikes—yet another effort by the federal government to ram the Common Core down everybody’s throats? There would be hell to pay before, during, and after the 2016 elections.)
So leave it alone. Listen to the president of the National Council of Teachers of Mathematics when she says to leave it alone. Accept the fact that the present NAEP math framework is still pretty much aligned with what cutting-edge math educators deemed most important twenty-odd years ago. Trust me, the math framework developed at that time was itself deeply controversial, the target of angry allegations that NAEP was getting “fuzzy” and rejecting traditional arithmetic. Understand that we broke the trend line for twelfth-grade a decade ago, which was controversial and probably unwise. Conduct as many studies as you like, but please do not stir this pot again now.
– Chester E. Finn, Jr.
This first appeared on Flypaper.
