Come out upon my seas
Cursed missed opportunities
Am I a part of the cure?
Or am I part of the disease?”
-Coldplay, “Clocks,” A Rush of Blood to the Head, 2002

Dear Deborah,

I am haunted by the title of your post:
“The Testing Obsession Widens the Gap” Could this possibly be true? Is test-based school reform reducing opportunity for America’s neediest children? Is everything for which we school reformers fight actually making things worse? Am I a part of the cure, or am I part of the disease?

###

“It’s OK to ask: ‘What if I’m wrong?’” you wrote last week. So let me ask it. It wouldn’t be the first time. A year ago, for example, I explored the “test score hypothesis“—a line of reasoning, undergirding much of the reform movement, that says that if we can significantly improve low-income students’ math and reading skills, as measured by standardized tests, we can significantly increase their chances of escaping poverty.

Let’s unpack this hypothesis a bit.

As it stands now, children born into poverty come into kindergarten with massive deficits—in terms of vocabulary, content knowledge, and non-cognitive skills. And if they make it to high school graduation 13 years later (and many will not), they will leave, on average, reading and doing math at an 8th-grade level. Of the low-income teens that give higher education a shot, the vast majority of will end up in remedial education and then wash out. More than half of poor children will become poor adults, with poor children of their own. The cycle will repeat. Our hope is that by improving our schools (and, yes, other things too), we can change this narrative.

Let’s imagine that our schools can help the average child born into poverty do somewhat better. Let’s say that with a combination of talented and well-trained teachers, a rich and rigorous curriculum, lots of supports, and strong leadership, we’re able to get poor students, on average, to a 10th-grade level by the time they graduate high school. Suddenly they can attend a community college, or even a four-year university, without starting in remedial education. They are much more likely to graduate, at least with an associate’s degree or a technical credential. Rather than making minimum wage, they will make a living wage.

They are less likely to get pregnant as teens, or end up in prison, or drop out of the workforce. Their children wouldn’t be born poor—they would be born middle class. This would be transformative.

Notice the key assumption built into this “theory of action”: reading and math matter a lot. Getting to the 10th-grade level instead of the 8th-grade level (even as measured by rinky-dinky standardized tests) would make a meaningful difference in real lives. With that assumption in place, it’s not crazy—in fact, it’s perfectly rational—to hold schools accountable for helping their students make progress every year with their reading and math skills. It’s smart to put in place clear, high standards—let’s call them common-core standards—that will delineate the path from poverty to prosperity, that will help schools and teachers focus on the knowledge and skills that matter most, and will get students to true readiness for college and career by the age of 18.

So Deborah, are you ready for the big question, the kicker, the heart of the matter?

How sure are we that it’s literacy and numeracy, and related academic knowledge and skills, that are the most important precursors to success in college, career, and life? What if something else is just as important, or even more important, like “non-cognitive skills” or personal relationships? (Or perhaps the habit of “serious intellectual inquiry,” as you put it?)

And what if our “testing obsession” is crowding these other things out?

These are critical questions, but here’s what gives me solace.

First, the evidence is quite strong that reading and math achievement are critical tickets to the middle class. Look, for example, at the blockbuster study from Raj Chetty, John Friedman, and Jonah Rockoff that examined the impact of teachers on students’ long-term outcomes. AsKevin Carey explained at the time,

If you believe standardized tests are worthless or highly flawed or deeply inadequate or even troublingly limited in accuracy and scope-and many reasonable people believe these things-then you could dismiss or downplay value-added measures of teacher effectiveness, by definition. … But now the CFR study says that teachers who are unusually good at helping students score high on standardized tests today aren’t just unusually good at helping students score high on standardized tests tomorrow. They also have an unusual effect on the likelihood of students going to college, going to a good college, earning a good living, living in a nice place, and saving for retirement. In other words, whatever the limitations of standardized tests may be, test-based value-added scores do, in fact, provide valuable information about the things most people care most about.

Or look at the evidence that E.D. Hirsch cites about the impact of teenagers’ vocabulary on their long-term prospects, such as a 1999 study that shows that “a gain of one standard deviation on the Armed Forces Qualification Test raises one’s annual income by nearly $10,000 (in 2012 dollars).”

Or a brand-new study from the United Kingdom (flagged by Joanne Jacobs ) that finds that “math skills at 7 predict earnings at 42.”

Surely reading and math aren’t all that matters. Paul Tough makes a good case for non-cognitive skills. Others, yourself included, point to the importance of strong personal relationships with mentors. We could name more. But reading and math skills are at least necessary, if not sufficient.

On the other hand, there’s little evidence that the “testing obsession” is systematically getting in the way of good teaching and learning in high-poverty schools. That’s not because an obsession with testing isn’t a problem. It surely is, with its temptations of cheating, narrowing of the curriculum, and the culture of fear that it often perpetuates.

But here’s the rub, Deborah: Studies of high-poverty schools in America have demonstrated for decades that great teaching and learning have always been the exception, not the norm. To believe that testing is making these schools worse, you have to believe that they were once pretty good, or at least better than they are now. I just don’t see it. Do you? Where’s the evidence of that?

Furthermore, think back to Kevin Carey’s comments on the Chetty study. If an obsession with reading and math was crowding out more important tasks, why would students with stronger reading and math gains do better long-term than their peers?

Here’s what your readers need to remember: The choice today is not between 100,000 Central Park Easts or Mission Hills and 100,000 test-prep factories. If it were, I’d pick the Deborah Meier schools in a heartbeat. But let’s face it: There aren’t more than a handful of Deborah Meier schools out there. (The same goes with Don Hirsch schools or Mike Feinberg/Dave Levin schools, or any other brand you want to name.)

The typical high-poverty school is, and has always been, pretty mediocre. That’s not an indictment of the people who work in these schools; the problem is the system. And it’s not unique to education. Any big, bureaucratic government agency is going to struggle to achieve effectiveness, much less excellence. (Think the DMV.) Heck, even most large, private-sector companies are pretty lame, especially ones that don’t face much competition. (Think the electric company.) Layer on top of that all of the distracting demands placed upon schools, the fragmented nature of education governance, and, in some places at least, too few resources, and it would be a miracle if the typical high-poverty public school were good, much less great.

###

So do I think testing and accountability make matters worse? No. In fact, based on the studies cited above, I think they will make matters marginally better. I also think stronger standards and tests (a la common core) will make things better still.

What about you, Deborah? Are you willing to ask “What if I’m wrong?” What if it’s true that reading and math skills are hugely related to opportunities in life, and indeed are malleable? What if “direct instruction,” which you say isn’t needed, really is the most effective method for helping children in poverty develop those skills? What if it’s patently untrue that children learn “vocabulary, grammar, syntax, and spelling … the same way we learn everything else that matters,” as you stated last week, but instead have to be taught systematically? What if the perfect for which you have spent decades championing really is the enemy of the good—and the greater good, for millions of boys and girls throughout America?

Deborah, with all due respect, I ask you to ask yourself: Am I a part of the cure, or am I part of the disease?

-Michael Petrilli

This article originally appeared on the Bridging Differences blog, where Mike Petrilli will be debating Deborah Meier for the next month.

What, you ask, is this all about?

Thirty years after a blue-ribbon panel declared the United States to be “a nation at risk” due to the weak performance and shoddy results of our public education system, one of the two great reforms to have enveloped that system is the setting of explicit academic standards in core subjects, standards that make clear what math youngsters should know by the end of fifth grade, what reading-and-writing skills they must acquire by tenth grade, and so on. (The other great reform: widespread acceptance of school choice.)

Up to now, individual states have set their own academic standards. Some did this well, but according to reviews undertaken by Fordham and others, most stumbled badly, putting forth vague expectations that lack content and rigor and often promote left-wing dogma. And even the good ones differ so much from state to state that school and student performance cannot be compared around the country, much less with other lands.

Public education is indisputably the responsibility of states—embedded deeply in their constitutions—but preparing young Americans to succeed in a mobile society on a shrinking and more competitive planet calls for some commonality of education expectations across the land, expectations that, if met, truly prepare young people for college and good jobs.

Many state leaders understand this and, beginning five years ago, the National Governors Association and the Council of Chief State School Officers (to which most state superintendents belong) launched a foundation-funded project called the Common Core State Standards Initiative, which gave birth to a set of commendably strong standards for English language arts and math from Kindergarten through high school. Our reviewers found them superior to the academic expectations set by three-quarters of states—and essentially on par with the rest.

But would states actually embrace them in place of their own? This was—and remains—totally voluntary, but decisions grew more complicated when the Obama administration started pushing states toward such adoptions by jawboning, hectoring, and luring them with dollars and regulatory waivers.

Whether it was the standards’ intrinsic merit, administration pressure, or the potential advantages of commonality—not just comparability but also cheaper textbooks and tests that need not be tailored to each state’s specifications—forty-five states plus D.C., several territories, and the Pentagon’s school network signed on. (Texas and Virginia are the big exceptions.) The top-priority education initiative in most of those places today is preparing teachers, parents, and others for these demanding standards—and for the likelihood that scores will plummet on the tougher tests now under development.

Then came the backlash. Some arose on the left from foes of testing and teacher groups wary of being evaluated against sterner criteria. Some arose from parents and educators fretful that heavier emphasis on English language arts and math will eclipse music, art, and the rest of a balanced curriculum.

The heavy artillery, however, came from the right. In true tea-party style, the Common Core was presented as a federal plot—worse, an Obama plot, in cahoots with the Gates Foundation, maybe even the United Nations—to take over American schools, end local control, undermine state sovereignty, and abolish school choice. Some decried the Common Core as a lowering of standards because, for example, it doesn’t mandate algebra in eighth grade. (Never mind that few eighth graders study real algebra today.) Others prophesied that Jane Austen and Mark Twain would be replaced by close study of auto-repair manuals. (The list of recommended readings that accompanies the Common Core is excellent—but bad choices by teachers or curriculum directors can subvert any standards.)

Many respected conservatives back the Common Core, including such scarred veterans of the education-reform wars as Jeb Bush, Bill Bennett, Chris Christie, Rod Paige, and Mitch Daniels. They understand that academic standards are just the beginning, describing a destination but not how to get there. They understand, too, that a destination worth reaching beats aimless wandering—and that a big modern country is better off if it knows how all its kids and schools are doing against a rigorous set of common expectations. As good conservatives, they realize that the Common Core in the long run should save dollars, enhance accountability, hasten development of powerful instructional technologies, strengthen American competitiveness, give a boost to the country’s shared civic culture, and (by supplying parents with better information about school performance) advance school choice.

They also recognize, however, that the Common Core is voluntary and that states unserious about implementing it are better off not pretending to embrace it.

Some day, we’ll know whether schools and students in the Common Core states do better than those in places that opt to go it alone. It’s hard to imagine that they’ll do worse.

Education reform is hard. Admiral Rickover once compared it to “moving a graveyard.” Standards-setting is just part of it—and common standards aren’t inherently better. (Newly released standards for science appear to have serious shortcomings.) But when a group of state leaders, many of them Republicans, can come together to set expectations for the curricular core that surpass what most of them set on their own, conservatives ought to applaud, not lash out.

-Chester E. Finn, Jr.

This blog entry first appeared in the Fordham Institute’s Education Gadfly Weekly.

I expect that PARCC and Smarter Balanced (the two federally subsidized consortia of states that are developing new assessments meant to be aligned with Common Core standards) will fade away, eclipsed and supplanted by long-established yet fleet-footed testing firms that already possess the infrastructure, relationships, and durability that give them huge advantages in the competition for state and district business.

In particular, I predict (as does Andy Smarick) that the new ACT-Aspire assessment system, which is supposed to be ready for use in 2014 (a full year earlier than either of the consortium products) and which some states are considering as their new assessment vehicle, will be joined by kindred products to be developed and marketed by the College Board. And the two of them will dominate the market for new Common Core assessments.

One straw in the wind: Alabama’s announcement last week that it is foreswearing both consortia and will use the ACT assessment system. And, of course, both Kentucky and New York have already concocted and deployed their own versions of Common Core assessments—possibly but not necessarily interim models.

Although the College Board and ACT have traditionally focused on the high-school-to-college transition, both also have experience earlier in the K–12 sequence. ACT Explore is aimed at eighth and ninth graders, ACT Engage goes down to sixth grade, and ACT “WorkKeys” is a significant player in determining career-readiness. The College Board’s Pre-SAT test is typically taken in tenth grade. Its “Readiness Pathway” assessment program reaches down to eighth grade, and its “Springboard” program to sixth—with “alignment” guides already prepared for Common Core standards in both English language arts and math for grades six through twelve.

So it’s not too big a stretch for either organization to dip deeper into the K–12 curriculum and assessment business, and it’s no stretch at all for their chief test-administration partners—Pearson in the case of ACT, ETS for the College Board. Each has ample experience in devising and administering tests from the early grades onward. (In fact, Pearson already has pre-K assessments.)

At least as importantly, these organizations know how to give tests to millions of people. They have the infrastructure and the test security. They have the systems for scoring and reporting. Perhaps above all, they have the relationships and the trust of thousands of school systems, dozens of states, and millions of parents.Plenty of states already use ACT products as part of their existing assessment systems. And both organizations are long established, well led, deep-pocketed, and pretty sure to be around a decade or two from now.

As yet, the new consortia have none of those things. They’re struggling with organizational structures, governance, post-federal financing, test-development agonies, uncertain costs, conflicting views of “cut scores,” and all manner of other puzzles.

Those would be significant challenges were there no competition, but ACT has made no secret of its intention to seek states’ Common Core assessment contracts—and Alabama may turn out to be the first of many to sign up. The College Board hasn’t (to my knowledge) announced itself yet, but testing insiders know that it’s lately been on a hiring binge—even luring key assessment developers from ACT—that surely points in this direction.

Will the ACT and College Board versions of Common Core assessments be true “next-generation” tests that probe deeper understanding and more sophisticated (“higher-order”) skills in more revealing ways? Will they be “adaptive” (via computer or otherwise) to kids at different levels of achievement or will they, like most of today’s tests (see discussion here at the seventeen-minute point), do a weak job of differentiating performance at the top and at the bottom of their range of difficulty? I do not know. But I do know that all of these accoutrements carry dollar costs that state assessment budgets may not be able to bear—and veteran testing firms are accustomed to cutting their cloth to fit the wearer’s dimensions.

I assume that scores and scales on the new assessments will be comparable across states (as are current ACT and SAT scores), but individual states will likely set their own “cut points” for purposes of grade-to-grade promotion and high school graduation. That’s tricky, however, if you’re serious about bona fide “career and college readiness,” which is a meaningless concept if it differs by state; what’s more, the new standards aren’t really worth the bother unless “proficiency” levels for every grade cumulate to a desired end-point by senior year. (I predict that, as with consortium-developed assessments, the ACT and College Board folks will recommend grade-specific proficiency scores that do cumulate in the intended way, but individual states will decide for themselves what signifies readiness for promotion and graduation.)

If I’m right that ACT and College Board scarf up much state business, there won’t be a lot left for the consortia—and they may founder. That would, of course, represent a considerable waste of federal dollars. On the other hand, it would remove from the Common Core debate (at least until NCLB-reauthorization time, if that day ever comes) the specter of Arne Duncan and Barack Obama clutching those standards to the federal bosom.

Besides, the consortia could remain useful, even if they don’t do assessments themselves. Neither ACT nor the College Board will want to alienate the many state leaders who have been earnestly advancing the consortium work, and these groups could readily convert into advisory and coordinating bodies that help member states implement and make sense out of the results on the new tests—and advise test developers and standard-setters alike on how their products work in the real world.

Time will tell. I might be jumping to premature prediction—and you may interpret these entrails differently than I do. Letters to the editor are cordially invited.

-Chester E. Finn, Jr.

This blog entry first appeared in the Fordham Institute’s Education Gadfly Weekly.

And while one can hope that Atlanta was an outlier in terms of the scope and longevity of its cheating conspiracy, it’s hardly an isolated case, as examples from El Paso, Philadelphia, Washington, D.C., and other locales demonstrate.

As expected, test critics are having a field day, using Atlanta as evidence of why all this must go. They yearn to throw the accountability baby out with the testing bathwater. But they’re wrong. The better approach is to “mend it, not end it.”

Try this thought experiment: What would happen if U.S. schools ceased all standardized testing—and related consequences? No more annual assessments, no more grading schools based on the results, no more interventions in low-performing schools, no more teacher evaluations tied to test scores, no more “merit pay” for high performing teachers or job jeopardy for low performers.

The result: In our most affluent communities, little would change. Schools would continue to drive toward the real-world standard of college acceptance at elite universities, via Advanced Placement exams and high SAT scores.

At schools serving both rich and poor kids, we would probably see a return to the 1990s, when achievement gaps were overlooked, wealthy students were guided toward rigorous coursework and “college readiness,” while poorer pupils were shepherded into easier classes with less challenge and weaker teachers.

And in high-poverty schools—the main target of twenty years of reform and the primary drivers of America’s improved student achievement since the 1990s—a few might keep pushing students toward college and good jobs, but many would return to the “soft bigotry of low expectations” and be satisfied with getting their students to graduation day, whether or not they learned much along the way.

I can’t prove that my forecast would come true, but the burden rests on those who want to eliminate testing and accountability to provide assurance that the system won’t revert back to its bad old ways.

If ending testing and accountability carries huge risks, what might mending it look like?

First, we should embrace testing as a diagnostic tool, not just an accountability weapon. We should get feedback into teachers’ hands much faster and make sure the tests themselves are of higher quality. All of this is the aim of the Common Core assessments currently under development, to be ready for prime time in 2015.

Second—and this is obvious—we need to invest in better test security. The Common Core assessments will be online, closing off current cheating strategies (like erasing and replacing answers on bubble sheets), but surely opening some new avenues. States need to spend the money to make sure test results can be trusted and cheaters can’t succeed.

Third, targets (for schools, students, and teachers) should be challenging but attainable. One source of the cheating scandals was educators feeling that fraud was literally the only way to produce the scores demanded by the system. They might have been right. The focus should transition to achievement growth over time, rather than hitting a particular “cut score.” (Many state accountability systems have moved in this direction recently, thanks to Education Secretary Arne Duncan’s waiver process.)

And fourth, official rankings or grades (of schools or teachers) should be informed by test scores but also leavened by human judgment. School grades might be conferred by British-style inspectors who look at pupil achievement along with much else.

And teacher evaluations should be the province of school principals, who should consider test scores as one set of data among many. The use of human judgment is particularly important if consequences are attached—closing schools, for example, or giving extra pay to great teachers, or terminating poor ones. (In this case, Duncan’s mandate for states to develop formula-driven teacher evaluations is a step in the wrong direction.)

Testing and accountability, properly conceived and implemented, can still be important tools in improving achievement and opportunity in America. Let’s keep the good, throw out the bad, and extinguish the anti-testing fire that started in Atlanta.

-Mike Petrilli

UPDATE: Joanne Weiss, Arne Duncan’s Chief of Staff, wrote to make this clarification: “Federal policy doesn’t require ‘formula-driven teacher evaluations,’ only that student growth be a significant consideration in the evaluation, and that evaluations should consist of multiple measures. We don’t stipulate any weights or formulas, nor do we require their use. Further, human judgment is critical to any good evaluation system. While it’s true that many states have implemented formula-driven evaluation systems (often including human judgment factors, like teacher observations and school/community contribution as part of the ‘formula’), we have no such requirements in our policies or regulations.”

This article first appeared in the New York Daily News.

The new Common Core math and reading standards adopted by 45 states have come under a firestorm of criticism from tea-party activists and commentators such as Glenn Beck and Michelle Malkin. Beck calls the standards a stealth “leftist indoctrination” plot by the Obama administration. Malkin warns that they will “eliminate American children’s core knowledge base in English, language arts and history.” As education scholars at two right-of-center think tanks, we feel compelled to set the record straight.

Here’s what the Common Core State Standards do: They simply delineate what children should know at each grade level and describe the skills that they must acquire to stay on course toward college or career readiness. They are not a curriculum; it’s up to school districts to choose curricula that comply with the standards. The Fordham Institute has carefully examined Common Core and compared it with existing state standards: It found that for most states, Common Core is a great improvement with regard to rigor and cohesiveness.

For decades, students in different states have been taught different material at different rates and held to radically different standards. Several years ago, a small group of governors joined together in an effort to align their states’ standards and assessments. This group expanded through the National Governors Association and the Council of Chief State School Officers. In 2007, curriculum experts began to devise the new Common Core standards. Drafts were circulated among the states, comments received, and the standards adjusted. So far, 45 states and the District of Columbia have signed up to implement these new expectations.Now let’s address the false claims circulated by the most vocal critics of Common Core.

Common Core is not “ObamaCore,” as some suggest. While President Obama often tries to claim credit, the truth is that the development of Common Core was well underway before he took office in January 2009. Some argue that states were coerced into adopting Common Core by the Obama administration as a requirement for applying for its Race to the Top grant competition (and No Child Left Behind waiver program). But the administration has stated that adoption of “college and career readiness standards” doesn’t necessarily mean adoption of Common Core. At least a handful of states had K–12 content standards that were equally good, and the administration would have been hard-pressed to argue otherwise.

Education policymaking — and 90 percent of funding — is still handled at the state and local levels. And tying strings to federal education dollars is nothing new. No Child Left Behind — George W. Bush’s signature education law — linked federal Title I dollars directly to state education policy, and states not complying risked losing millions in compensatory-education funding (that is, funding for programs for children at risk of dropping out of school).

Perhaps the clearest evidence that states can still set their own standards is the fact that five states have not adopted Common Core. Some that have adopted it might opt out, and they shouldn’t lose a dime if they do.

The most prominent criticism of Common Core is that it abandons classical literature and instead forces students to read dry government manuals. This claim reflects a profound and perhaps deliberate misunderstanding of Common Core literacy standards, which do encourage increased exposure to informational texts and literary nonfiction. The goal is to have children read challenging texts that will build their vocabulary and background knowledge, a strategy grounded in what education scholar E. D. Hirsch has shown: A broad, content-rich curriculum reduces the achievement gap between the middle class and the poor.

Common Core suggests that, as a student progresses through the grades, the nonfiction proportion of materials should increase until, by the end of high school, it represents 70 percent of total reading in all classes. The standards explicitly warn that English teachers “are not required to devote 70 percent of reading to informational texts.”

These “informational texts” include foundational documents of American history — the Gettysburg Address, Common Sense, and works of thought leaders like Emerson and Thoreau. Given the evidence that most American students cannot identify the decade in which the Civil War occurred, one would think that enhancing student knowledge of our nation’s rich history would be welcome.

But facts be damned when there are standards to undermine! Headlines blare: “Common Core Nonfiction Reading Standards Mark the End of Literature.” Reporters lament that To Kill a Mockingbird is being stripped from the “U.S. school curriculum.” Never mind that there is no “U.S. school curriculum” from which beloved literary classics are to be dropped — or that To Kill a Mockingbird actually appears on the list of “exemplar” texts supported by the standards.

Perhaps the most curious Common Core criticism comes on the math side, with opponents arguing that the standards are squishy, progressive, and lacking in rigorous content. While Common Core math standards do articulate ten math “practices,” mathematical content dominates the K–12 expectations. Unlike many of the replaced state standards, Common Core demands “automaticity” (memorization-based familiarity) with basic math facts, mastery of standard algorithms, and understanding of critical arithmetic. These essential math skills are not only required but given high priority, particularly in the early grades. The math standards focus in depth on fewer topics, and ones that coherently build on one another over time.

The Common Core standards are not a panacea; much depends on the curricula that states and districts select to implement them. Some critics suggest that we are enshrining mediocre standards for eternity. But the Common Core standards are a floor, not a ceiling. Students can still be accelerated and offered supplemental learning, the standards can be improved over time, and states are free to devise something better.

Common Core offers American students the opportunity for a far more rigorous, content-rich, cohesive K–12 education than most of them have had. Conservatives used to be in favor of holding students to high standards and an academic curriculum based on great works of Western civilization and the American republic. Aren’t they still?

-Kathleen Porter-Magee and Sol Stern

Kathleen Porter-Magee is the Bernard Lee Schwartz Policy Fellow at the Thomas B. Fordham Institute. Sol Stern is a senior fellow at the Manhattan Institute and a contributing editor of City Journal.

This blog entry first appeared in the National Review Online.

In this video, Eric Smith, Tom Luna, Ulrich Boser and Rick Hess discuss the report card at a forum hosted by the Fordham Institute and moderated by Mike Petrilli.

— Education Next

I won’t forget the book’s emotional grip, but equally influential to my intellectual development was the policy and political back story that explained how the boys’ toxic surroundings came to be.

Nearly two decades later, I’m still chastened by the book’s central lesson: A government policy developed by mostly benevolent leaders hoping to improve the lives of the disadvantaged—in this case, by razing old, low-income, ostensibly decaying neighborhoods in favor of gigantic public-housing skyscrapers—did incalculable harm to those it was designed to help.

This has been on my mind in recent weeks, as the national school-closure conversation has flared. Much of that conversation is familiar, but one assertion made by critics, namely that school closures destabilize entire neighborhoods, raises a question that hasn’t been discussed nearly enough. And though some might wave it away as irrelevant or worse, the lessons of the Kotlowitz book force me to take it seriously:

Can a bad school be good for a neighborhood?

Might there be compelling civic or social reasons for keeping open persistently failing or unsafe inner-city schools?

We know there are reasons to close them. First, school turnarounds seldom work (contra the recent statement from AFT president Randi Weingarten), and closures stop us from continuing to send kids to chronically underperforming schools. (The stickiness of failure is even true in the charter sector.)

Second, closures are a part of a continuous-improvement process known as “portfolio management.” Couple closures with new school launches and replications or expansions and you can move the quality curve to the right. This is the heart of my book, The Urban School System of the Future.

(There was a sophisticated discussion of this matter at the recent Bellwether book event. Kaya Henderson, John White, and Mike Casserly did a nice job of explaining the various forces involved. Emma Brown from the Post captured some the back and forth here.)

Third, as explained by Checker Finn, financial considerations make some closures unavoidable—and the likeliest targets are the district’s lowest performers, both because they’re not serving kids well and because they’ve typically seen their enrollments reduced by, among other things, students leaving in search of better educational options.

The anti-closure camp generally counters with a trio of time-worn and ultimately unpersuasive assertions.

• Their “fix-it-don’t-close-it” case is overwhelmed by empirical evidence.

• Their “closures-are-an-anti-union-strategy” argument can’t explain why reformers are also willing to close non-unionized, low-performing charter schools.

• Their “closures-are-a-civil-rights-violation” argument causes most to reply, “It’s a far greater violation to force low-income African American and Latino children to remain in failing, unsafe schools.”

Hence the camp that favors judicious but firm use of school closures increasingly carries the day.

Yet another argument has been waiting to be made, however—one that stokes the communitarian tendencies of my brand of conservatism. It starts with awareness that good schools are a powerful asset for troubled neighborhoods. Indeed, I’ve argued in many places that the closure of excellent inner-city Catholic schools is terrible for such communities. I’ve made the case that high-performing, high-poverty schools can provide safety, stability, and hope to at-risk kids and their families and that these features have immeasurable value in distressed neighborhoods beset by myriad challenges.

But perhaps my view has been too narrow; maybe I’ve put too much weight on the “high-performing” part. Maybe all urban public schools—perhaps even all schools—deserve a greater degree of deference because of characteristics associated with their “local-ness.”

For example, the school could be the major employer of adults in the area.

Even if educationally dysfunctional, the school likely has its share of caring, educated adults who serve as role models and mentors for needy children.

The school may serve as the community hub for social services or civic activities.

Maybe its athletic teams still serve as a source of community pride.

It could be the neighborhood’s last connection to a happier past. Perhaps this school was once the gem of the school system. It might have been the city’s first desegregated school. Maybe, pre-Brown, though segregated, it succeeded academically in breathtaking fashion. Perhaps it’s named after a revered civil-rights leader.

It might have been among the first employers to practice nondiscrimination, paving a pathway to the middle class for countless minority families.

Maybe the neighborhood sees that school as the last thing that is actually theirs. Other families moved away. Businesses shut down. Churches closed their doors. But their school remains.

It might be the case, then, that in these and other ways, the school—notwithstanding its persistent low academic achievement—acts as an important strand in the invisible web of social connectivity that helps to hold a community together despite all the malign forces trying to pull it apart.

Those who cleared Chicago’s “slums” to make way for new high-rise public-housing towers didn’t realize that they were severing intricate, generations-old social bonds. This loss of connection, when combined with the ills of concentrated poverty and the inherent flaws of the new building complexes, turned these behemoths into modern-day Trojan Horses: marvelous to behold from the outside (at least when they were new) but with danger lurking inside.

Environmental parallels are numerous: misbegotten projects that cleared eyesore swamps and walls of mangroves to make way for highways, waterfront condos, and more. We found out too late that these “messy” wetlands actually served as massive water filters, flood preventers, wildlife protectors, fish incubators, and much more. Profound environmental degradation was the consequence of well-intentioned, if naïve, attempts at progress.

To be sure, there are enormous differences between these examples and thoughtful, surgical school-closure strategies. Unlike quiet wetlands, failing, dangerous schools do real harm and, if their students move into higher-performing and safer schools, their replacements bear no resemblance to Cabrini-Green.

My point is merely that those pursuing school-closure strategies should be mindful that every school, even the lowest-performing, is woven into the fabric of its neighborhood—and tugging on that thread affects the entire cloth.

That doesn’t mean the scale tips to the “no closures” side. I remain a committed proponent of smart school-replacement strategies and portfolio management.

But it does mean that reformers should acknowledge both sides of the scale and recognize that, when the case for closure prevails, the arguments on the other side haven’t been eliminated; they’ve just been outweighed in a particular case.

That approach, I believe, is the responsible way to go about policymaking generally. And in this case, its effect on policy implementation in cities across the nation would matter: we’d see greater prudence in the process of deliberation, improved communication to stakeholders, and more thoughtful and respectful execution of an undesirable but ultimately necessary decision.

—Andy Smarick

This blog entry first appeared on the Fordham Institute’s Flypaper blog

That’s the miles of daylight between the platitudes about the new standards’ “dramatic,” “transformational” nature and the distressing reality of implementation.

The report’s upside is that we now know more about state-level planning. The downside is that we know nothing more about the quality of that planning—and this is the whole ball of wax.

This might sound like the classic unfair criticism of a research project—point out what you wanted a study to answer and then shame the authors for looking into something else.

I’m succumbing to this temptation because I’m troubled by all of the Common Core cheerleading going on. Apart from a still relatively small band criticizing the standards for stealing fiction and states’ rights, most reformers contend that Common Core is just shy of avert-your-eyes miraculous.

Tom Loveless had the temerity to wonder if the standards would improve achievement, and the response from their incredulous supporters was, said Loveless, “like putting my hand in a hornet’s nest.”

We’ve made the necessary oblations to Common Core, and now it’s time to get serious about the seriousness of implementation. That means no longer marveling at the shiny hubcaps and supple leather interior or, worse, just taking the salesman’s word for it, but opening up the hood and poking around.

It’s this mindset that I bring to the Education First–Editorial Projects in Education study on the results of a Common Core survey. It asked state leaders about implementation in areas like professional development and aligned instructional materials.

The high-level headline is that states are better off than they were last year and things are generally looking pretty good.

For example, 42 states report either having plans or building plans to revise their teacher-evaluation systems to comport with the expectations of Common Core. Thirty states claim to have “fully developed” plans to change their instructional materials to align with the new standards.

Moreover, if you just glance at the report’s Exhibit 1, you’ll see most boxes filled with “Completed.”

A reasonable person would walk away thinking that implementation is going swimmingly.

But things aren’t so rosy.

States do have “plans” in lots of areas; the issue, though, is what these plans amount to.

For example, even the best state departments of education were fretting about the massive challenges associated with overhauling educator evaluation systems before Common Core implementation was front and center. Student achievement data for untested grades and subjects and inter-rater reliability of observations were keeping smart folks up at night when state content standards, teacher professional standards, and assessments were static.

With changes afoot in all of these areas, teacher-evaluation reform has gotten exponentially more difficult. Sure, any SEA can put some ideas on paper and call it a “plan.”

But what if the survey had asked something like, “Are you confident that, when Common Core is fully implemented in 2014–15, your educator evaluation systems will accurately differentiate among educators based on their levels of effectiveness, provide meaningful feedback for teachers, and enable administrators and policymakers to make decisions related to preparation, certification, hiring, tenure, and compensation?”

There is no way that 42 states could, with clear consciences, answer, “Yes.” (Take a look at this recent article: Florida’s state board is raising red flags, and the department of education is creating a “Plan B.”)

The same applies to the optimistic self-reporting about professional development. You should dig into your state’s Common Core PD plan. Be prepared for it to look a lot like the SEA’s current PD plans: same state office, same providers, same higher-ed institutions, same quality monitoring, same number of hours required, etc.

Had the survey, instead, asked about states’ confidence that their plans would enable teachers to prepare students for successful acquisition of the skills and information required by the new standards, then the results would’ve been far less sanguine.

I’ll have much more to say about instructional materials in the weeks to come, but for the time being, I’ll leave it at this: Comparing the navigability of the CC-aligned resources marketplace to the Wild West would be an insult to the Wild West. You might want to check your state’s “fully developed” align resources plan to see how it intends to pilot these rocks.

Finally, the survey didn’t ask states about the area that concerns me most: whether there are any activities underway to improve teacher preparation programs so their graduates are ready for the demands of Common Core. As far as I can tell, most states haven’t even begun working in this area.

So about those vaunted “plans”…

The Prussian General Helmuth von Moltke famously wrote, “No plan survives contact with the enemy.” Heavyweight boxing champion Joe Louis said, “Everyone’s got a plan until he gets hit.” In other words, there’s a world of difference between what you put on paper and what actually carries you to victory.

When it comes to Common Core implementation, too much of the ed reform world is still acting like it’s at the ceremonial, bragging-as-currency, pre-fight weigh-in.

But the bell has rung. It’s go time.

Unless we stop hyping the crowd and flexing for the cameras and get down to the real business at hand, a whole lot of states, come 2014, are going to find themselves on the mat.

—Andy Smarick

This blog entry first appeared on the Fordham Institution’s Common Core Watch blog

Born to Rise: A Story of Children and Teachers Reaching Their Highest Potential
by Deborah Kenny
Harper, 2012, $25.99; 256 pages.

Mission Possible: How the Secrets of the Success Academies Can Work in Any School
by Eva Moskowitz and Arin Lavinia
Jossey-Bass, 2012, $27.95; 176 pages.

As reviewed by David Steiner

On page 87 of Mission Possible, the account by Eva Moskowitz and Arin Lavinia of the work of their charter schools, the reader is invited to watch a video of a book discussion in 1st grade. A detour to the included DVD is instructive: in this Harlem-based, lottery-selected public charter school, we see a 1st-grade classroom that challenges any in the country for the intellectual engagement of its students without any reliance on the regimented, direct instruction that the clichéd objections imagine dominate all successful charter schools. Faced with such examples, and the academic record of Moskowitz’s Success Academies, one’s first reaction should simply be applause. Having served (briefly) as a board member for one of the Success Academies, I know that such video clips are not cherry-picked: teachers in every classroom in every Academy school are expected to create such spaces of intense and demanding thinking and learning.

A different but equally positive reaction is evoked by Deborah Kenny’s intensely personal account of the grit, resolve, and courage that led her to take the opportunity offered by the charter school movement and create a model school that her students were immensely lucky to attend. Her passion for bringing out the best in all who worked with her, her disarming throwaways (“it was the first five minutes of our first day, and already I’d made a mistake”), and her candid directness about necessary conditions for her work (“it is impossible to nurture a positive culture without the right to hire and fire at will”) make this an inspiring story.

Sharing boundless drive and self-discipline, and an equal commitment to proving that a child’s past is not her or his predetermined destination, Moskowitz and Kenny take justified pride in being mission-driven realists who, along with their handpicked colleagues, have radically recast the life chances of their students. But they have produced quite different books. Kenny’s more personal memoir does not reach her charter school until late in the narrative, and even then, the focus is squarely on forging a human culture, building a team whose members will go to the wall for their students, and for each other. For her, the disaster of American education is summed up by the line “we got here by disrespecting teachers.”

What Kenny means by respect is creating a culture of accountability for children’s learning that “enables freedom” and a freedom that “unleashes teacher passion.” Kenny explicitly eschews the cookbook vision of school reform: “Schools,” she writes, “are not products to be designed and replicated.” She goes on: “Every school in America has access to the same pedagogical ideas and methods we use. The problem is not lack of information but a lack of motivation engendered by the low accountability/low empowerment culture of our public schools.” Many of her sources of inspiration—Maxime Greene, Peter Drucker, Dennis Littky—are not your typical charter-school heroes. What connects them is not a method but a conviction that successful outcomes are all about people learning together, creating spaces of continual mutual feedback, encouragement, and empowerment. Kenny’s book is not and does not purport to be a how-to manual; it is a moving work about the power of well-placed determination matched to a political opportunity: the freedom through accountability that is the charter school.

Moskowitz and Lavinia set themselves a different task. Theirs is a self-proclaimed “how-to book.” As a result, the bar is in some sense higher: we expect to learn what the secrets are and how they can work. The answers in the book will enlighten readers who have heard about charter schools on the news but want to learn more. Moskowitz and Lavinia write at a relatively high level about effective techniques like the push to challenge students with the most rigorous and demanding material and instruction; the immediacy and constancy of granular feedback; the sharing of best practices; the focus on literacy instruction; the long school day and school year, including summer professional development for teachers; the constant coaching; and the agonizing care taken to choose each teacher from a vast pool of applicants. Each of these practices is standard at high-performing charters including KIPP, Uncommon Schools, and Achievement First, to take just three networks of such schools with which I am somewhat familiar.

What well-read educators and policymakers will want to know, and will not find in this book, is how it is that in New York City, in multiple instances, Success Academies, despite its astonishingly rapid growth as a school network, gets stronger academic results than even the top next-rung charter schools. Is it just that Moskowitz pushes even harder with even more rigor with even better-selected teachers on the good practices that the other charters are engaged in? If there are true secrets to the results from Success Academies, they are not on show here. A useful contrast in this respect is Paul Bambrik-Santoyo’s recently published Leverage Leadership: A Practical Guide to Building Exceptional Schools. The tools he lays out may or may not produce results at the level of the Success Academies, but those tools are in plain sight, in such unsecret forms as his “Observation Tracker.”

At the same time, these books will do nothing to silence the critics. There are, for example, no statistics on the percentage of ELL students in the schools, no numbers on the privately raised funds the schools put to use, and only cursory gestures, in Kenny’s book, to the controversy over students counseled out of or removed from these charter-school classrooms and to their teacher turnover rate.

In the end, the real contribution of these books lies neither in their appeals to the heart nor in the practices they catalogue, but in the moral condemnation each makes of our current education system. When Moskowitz argues that her practices can be adopted in any school, we are immediately tempted to say, Come on! Nine-hour days? Weeks of summer professional development? Responsiveness 24/7 to breakdowns in the classrooms, be it from noisy pipes or a single underwhelming lesson? Show me a public school system that could get there anytime soon.

But then we have to ask, why not? We have seen urban public schools successfully adopt many charter school “secrets,” including the nine-hour school day (e.g., United for Success Academies in Oakland); a rigorous, standard curriculum (e.g., the more than a dozen Chicago public schools that offer the International Baccalaureate); merit pay (e.g., the Washington, D.C., system); and the regular use of teacher video in professional development and evaluation (e.g., the Houston system, which was using video in this way as early as the 1980s).

Why have the results of the best-performing charter schools consistently eluded public school systems? The answer, unsurprisingly, has to do with the structures underlying public K–12 education. To bring to regular public schools the full panoply of successful charter-school practices we would need to rethink our labor practices, funding structures, reluctance to embrace a rigorous and specific curriculum, and all the other bêtes noires the national education conversation avoids or reduces to partisan caricatures.

If, rightly, we want to reject a zero-sum trade-off between our values, if what we need are a highly attractive long-term profession for successful teachers, accountability for student results, and a far more rigorous curriculum driving far higher learning outcomes for our students, are we willing to rethink the system from scratch and put everything on the table? If we cannot build a public school system on heroic individuals (and we surely cannot), how do we remake our school systems to make the standards Kenny and Moskowitz demand, and have largely achieved, the baseline of our public education? These books do not tackle these hard questions.

But no matter. One is left with the indictment and an urgent call to action, captured on page 136 of Kenny’s book: “‘This school’s a blessing!,’ exclaimed Jasmine’s grandmother when she found out they’d won the lottery. We hadn’t even opened our doors yet.”

David Steiner is dean of the School of Education at Hunter College and former commissioner of education for the State of New York.

Now ‘til March is not much time to repair this important, ambitious, but still seriously troubled document. The drafters might be wise to take more.

We at the Fordham Institute have a long history of reviewing state science standards, and last week, we submitted our review, feedback, and comments on NGSS 2.0. A team of nine eminent scientists, mathematicians, and educators, prepared our analysis. You can find the full review here, including team members’ bios on page 8. (We previously reviewed Draft 1.0, and Dr. Paul R. Gross, the distinguished biologist who heads the team, also reviewed the National Research Council “framework” on which NGSS is based.)

If states are going to make rational decisions to replace their own science standards with NGSS, it’s only right to insist that NGSS be stronger—clearer, with better content, more rigorous, and more easily applied by teachers—than the standards that states have come up with on their own.

Fortunately for the NGSS team, that’s a low bar. In our most recent review of state science standards, published just a year ago, the Fordham team determined that the clarity, content, and rigor of most state K–12 science standards were mediocre to awful. The review assigned grades of C or worse to three quarters of the states. (Ten flunked altogether.)

Still and all, science education in America is no wasteland. Our reviewers also awarded “honors” grades (B or better) to a quarter of the states for their K–12 science standards. Tens of thousands of our ablest high school students every year earn high marks on Advanced Placement exams in physics, chemistry, and biology. On the 2011 TIMSS science assessment, among fifty-six jurisdictions participating at the eighth-grade level, just twelve produced stronger results than the United States. Remarkably, three of those were U.S. states! (Massachusetts surpassed Taiwan, Minnesota rivaled Finland, and North Carolina was strong, too.) And, of course, at the post-secondary level, the U.S. continues to house many of the world’s premier institutions of scientific research, and their scholars continue to win an impressive share of Nobel prizes and other key awards in scientific fields.

So while nobody should be satisfied with America’s overall performance in science education, it’s also important to keep in mind that, when one sets out to overhaul that system, it’s possible to make it even worse—particularly if, in our effort to raise standards for all students, we wind up lowering them for our best and brightest.

NGSS 2.0 falls into that trap. But that’s not all that’s wrong with it. If the drafters really want their final product to deserve widespread adoption, they still need to solve eight critical problems:

• In an effort to draft “fewer and clearer” standards to guide curriculum and instruction, NGSS 2.0 (like NGSS 1.0) omits quite a lot of essential content. Among the most egregious omissions are most of chemistry; thermodynamics; electrical circuits; physiology; minerals and rocks; the layered Earth; the essentials of biological chemistry and biochemical genetics; and at least the descriptive elements of developmental biology.

• As in version 1.0, some content that is never explicitly stated for the earlier grades seems to be taken for granted in the standards for later grades—where it won’t likely be found in students’ heads if the early-grade teachers aren’t prompted by the standards to teach it.

• Real science invariably blends content knowledge with core ideas, “crosscutting” concepts, and various practices, activities, or applications. Well and good. But NGSS 2.0 imposes so rigid a format on its standards that the recommended “practices” dominate them. The authors have forced practices on every expectation, even when they confuse more than clarify. For example, high school students are asked to “critically read scientific literature and produce scientific writing and/or oral presentations that communicate how DNA sequences determine the structure and function of proteins, which carry out most of the work of the cell.” Here as elsewhere, the understanding of critical content—which should be the ultimate goal of science education—becomes secondary to arbitrary and peripheral activities such as “critical reading” and “oral presentation.”

• Although the drafters made a commendable effort to integrate “engineering practices” into the science rather than treat engineering as a separate discipline, their insistence on finding such practices in connection with so many standards sometimes leads to inappropriate or banal exercises—and blurs the real meaning of “engineering.”

• The effort to insist on “assessment boundaries”—which narrow the focus of a standard by setting a ceiling on the content that can be assessed—in connection with every standard often leads to a “dumbing down” of what might actually be learned about a topic, seemingly in the interest of “one-size-fits-all” science that won’t be too challenging for students. Given that what gets tested is generally what gets taught, this will invariably limit how far and how deep advanced students (and their teachers) might go. (The vague assertion that the problem can be dealt with via “advanced” high school courses helps almost not at all.)

• A number of key scientific terms (e.g., “model” and “design”) are ill defined and/or inconsistently used.

• Even as the amplitude of new appendices attached to NGSS 2.0 adds welcome explanation of what is and isn’t present and why, it also produces a structure that most users, especially classroom instructors, will find complex and unwieldy. Will a fifth-grade teacher actually make her way to Appendix K to obtain additional (and valuable) information about science-math alignment and some pedagogically useful examples? Will the final version of NGSS omit some of the intervening appendices that have more to do with the philosophical, political, and epistemological leanings of project leaders than with anything of immediate value to real schools?

• Although the “alignment” of NGSS math with Common Core math is improved, the drafters seem to have consciously limited the amount of math-dependent science that students need to learn. This weakens the science and leads, once again, to a worrisome dumbing down, particularly in high school physics—which, as the reviewers remind us, “is inherently mathematical.” It must also be noted that Appendix K, valuable as it is in making the science-math alignment clear for grades K–5, is essentially AWOL from the middle and high school grades, where it is most needed.

Hope springs eternal. The NGSS team made some worthy improvements between drafts one and two (though they ignored most of our advice), and they have an opportunity—a final opportunity, it appears—to make further repairs.

We surely hope that they do so. While we did not review NGSS 2.0 with an eye toward grading it, we intend to evaluate the final version much as we did state standards—and provide states with a side-by-side that they may use in connection with adoption decisions. We sincerely hope that NGSS 3.0 fares well in such a comparison—but to get to that point, some major modifications will need to be made. And we urge the drafters to take as much time as necessary to accomplish that, for the present draft is problematic in more ways than it is strong.

-Chester E. Finn, Jr. and Kathleen Porter-Magee

This blog entry first appeared in the Fordham Institute’s Education Gadfly Weekly.

Half those gains are probably an illusion, however. The latest results from the math and science tests administered by the Trends in International Mathematics and Science Study (TIMSS), the respected international testing agency, show gains of only 0.8 percent of a standard deviation yearly between 1995 and 2011. Further, another respected international assessment of student performance, the Program for International Student Assessment (PISA), found gains of only 0.5 percent of a standard deviation annually for U.S. students over roughly the same time period. (For specifics, see page 19 of our full report, Achievement Growth: International and U.S. State Trends in Student Performance [PEPG, 2012].)

In other words, NAEP has been identifying gains that are somewhere between two and three times as large as those recorded by two respected international testing agencies that do not have a political stake in showing rising levels of student achievement in any particular country.

For some time, analysts have been wondering whether NAEP tests have become easier. Those who construct the main tests that NAEP administers frankly admit that they have adapted questions over time to meet the changing curricula offered by contemporary schools. NAEP has also introduced special accommodations for those who say they are in some way disabled and need additional time or other modifications of the standard testing protocol. Have testing changes and administrative innovations softened tests so that they now indicate higher levels of student achievement than would be the case if older practices had been retained?

It is well known that when measuring economic change it is critical to adjust for inflation so that real growth is not confused with nominal growth in prices. An entire bureau within the U.S. Department of Labor is devoted to measuring the extent to which prices for the same commodities are rising or falling. With that information ready at hand, economists can ascertain whether the economy is actually moving forward or whether nominal growth in the GDP is simply the result of inflation.

Nothing similar exists in education. The U.S. Department of Education does not have an agency that inspects NAEP tests or state tests to ascertain whether questions on the tests have been eased with the passage of time.

It is remotely possible that TIMSS and PISA have revised their tests so that they have become more difficult over time, thereby underestimating U.S. student gains. But few believe that any testing organization in the late 20th and early 21st centuries has actually made its tests more challenging over time. All the social and political pressures operate in the opposite direction.

We do know one thing for certain: U.S. students are not closing the international achievement gap. Our study shows that even when measured by NAEP criteria, the United States stands at the 25th rank among 49 countries in achievement growth. Similarly, the recent TIMSS data show the United States to be the middle-ranked country among the 11 for which the organization could fully track student performance since 1995. U.S. students are making middling gains that are keeping them on par with students in other countries. In comparative terms, the United States is not making any progress at all.

— Paul E. Peterson

As you probably know, the teachers of Seattle’s Garfield High School are “boycotting” the Measures of Academic Progress (MAP) assessment, which is required by the district. Ostensibly, their protest is about the overuse of tests, the instructional time that those tests devour, and the culture of soulless data-driven instruction that animates today’s brand of school reform.

Yet it’s hard to square their complaints with the actual test they decry, for the MAP is precisely the type of “good” assessment that many educators claim to favor. It’s instructionally useful; it provides instantaneous feedback to teachers and students alike; and it’s not used for high-stakes decisions on issues pertaining to students and schools.

The real reason the Garfield teachers attack the MAP, one must presume, is because it’s a small part of Seattle’s new teacher-evaluation system. (If students show low growth on the MAP for two years in a row, it triggers a “closer look” at their teacher by the principal—pretty benign by national standards.) That’s a smart move on behalf of district officials; because the test is “computer adaptive,” it can pinpoint precisely where students are on the achievement spectrum and can give teachers full credit for any progress they help their charges achieve over the course of the school year. (If a ninth grader moves from the sixth-grade level to the eighth-grade level, the MAP can detect it, while most state assessments cannot.)

What the teachers are really protesting, it seems to me, is the use of student test scores in educator evaluations. And to be sure, there’s a legitimate case to be made that we are rushing too rapidly into such evaluations. But of course, that’ s not what the teachers say they are worried about.

It’s hard not to hear the echoes of this fall’s teacher strike in Chicago, in which educators insisted that the walk-out was about “air conditioning” and “working conditions” when everyone knew it was really about jobs—namely, what would happen to the thousands of tenured teachers whose schools are likely to close in coming years.

I don’t doubt that some Garfield teachers have personal reservations about the overuse of tests in today’s education system. I can also believe that the MAP, on top of the state tests, creates a heavy testing burden on teachers and students alike. And I would never blame teachers for crying foul about evaluation systems designed on the fly.

But how about a little honesty and perspective, people? To compare this episode to Martin Luther King Jr.’s efforts, as the Seattle teachers union president did the other day, is to cheapen the historic battle for true civil rights. This is a skirmish about teacher work protections as our system lurches toward greater accountability. It’s no heroic effort to overcome the forces of evil. And it’s certainly not just a flap about the MAP.

-Mike Petrilli

This blog entry first appeared on the Fordham Institute’s Flypaper blog.

Martin Carnoy and Richard Rothstein, in a recent report for the Economic Policy Institute, now tell us that performance is not as bad as you think and that Secretary Duncan should stop making “exaggerated and misleading statements” about the performance of U.S. students.

To arrive at this conclusion and the accompanying one-liner for media consumption, Carnoy and Rothstein begin with the fact that U.S. students are disproportionately disadvantaged when compared to students in a sample of high performing countries (Canada, Finland, and Korea) and in a sample of post-industrial countries (France, Germany, and the United Kingdom).  They then re-weight average PISA test scores in the U.S. and these six countries by the distribution of socio-economic status (measured in their analysis by numbers of books in the home) in an attempt to equalize statistically the distribution of family backgrounds across countries.

Adjusted for books in the home, U.S. students in math still lag behind Canadian, Finnish, Korean, and German students but pull even with those in the U.K. and come close to those in France.  They then repeatedly work to convince us that everything is not as bad as thought and may even be pretty good.

But what is the question that these calculations answer?  The reason that Secretary Duncan and others, including me, are concerned about the performance of U.S. students is that the international achievement scores in math say a great deal about the skills that our students will take to the labor force.  These human capital differences, according to historical data, bear a direct relationship to growth of the national economy.  And the economic implications of mediocre performance are enormous.

The Carnoy and Rothstein calculations simply do not deal with the relevant differences in skills across countries.  We have the population that we have – not the population of Finland.  So their adjustments cannot address the question of how well prepared our future labor force might be.  How well prepared we are depends on the skills of all of our population, not just those that statistically look like Finns.   In fact, as prior analysis by Paul Peterson, Ludger Woessmann, and me shows, looking at just our most advantaged socio-economic subgroup (children of college educated parents) does little to erase the deficits with other countries in advanced math and science performance, which is the preparation for science and engineering careers.

There is a hint – although largely unstated in the report’s 100 pages – that these calculations can be used to answer the question ‘how well are our schools doing?’  Specifically, they might absolve our schools of guilt, because it is the parents of these disadvantaged children that lead to much of the difference in scores, perhaps even all of the difference between scores with the U.K. students.

Even if their adjustment removed all of the measured international skill gap, why should we chide Secretary Duncan for pointing out that our students are less prepared than those of the other six sampled countries?

It might mean that our schools face more difficult educational challenges than found in the other countries, but surely this does not imply that we should “rest on our laurels” (such as they are).  First, most of the international gaps remain after standardization.  Even after adjustment, the U.S. remains at best in a three-way tie for fifth out of seven.  Second, we are past saying that there is nothing we can do to educate our racial and ethnic minorities and our economically disadvantaged; hundreds of schools have proven this wrong.  Third, this is the population that we have.

The U.S. does look somewhat better in comparison if we use international reading assessments.  I personally am skeptical about our ability to obtain valid and reliable reading comparisons across different languages.  It also appears that math and science assessments say more about skills valued in the labor market than reading assessments.  Yet, again, even after adjustment, we know from the international reading assessments that we have a long way to go to compete with the top countries.

Historically, when it comes to economic performance, the U.S. has remained the strongest economy in the world.  A variety of advantages have allowed us to overcome the shortcomings of skills that are exposed by these tests.  This historic economic strength reflects our earlier historical commitment to universal secondary school attainment; our strong and well-developed economic system; our secure property rights and free movement of labor and capital; our world’s best universities that can overcome some of the low entering skills;  and our use of skilled immigrants to bolster our innovative capacity.

Unfortunately, all of these advantages over our economic competitors are going away as many have made great strides in emulating and even surpassing these strengths of the U.S.  In the future, we will simply have to rely just on our skills if we are to sustain our current economic standing – and skills are what are measured by the international assessments.

In sum, we cannot paper over the fact that a large number of other countries have shown that it is possible to develop considerably higher skills in their youth than we are doing.  Our future depends on the skills of our population, and it is time to recognize that we are lagging.  Secretary Duncan is neither exaggerating nor misleading about this.

Eric Hanushek is a senior fellow at the Hoover Institution of Stanford University and a member of its Koret Task Force on K-12 Education.

In the following essays, we identify some of the key findings from the sixth annual Education Next-PEPG Survey, a nationally representative sample of U.S. citizens interviewed during April and May of 2012 (for survey methodology, see sidebar). Highlights include

• the Republican tilt of the education views of independents

• the especially high marks that Hispanics give their public schools

• strong support among the general public for using test-score information to hold teachers accountable

• lower confidence in teachers than has previously been reported

• the public’s (and teachers’) growing uneasiness with teachers unions

• the shaky foundations of public support for increased spending

• majority support for a broad range of school choice initiatives.

In addition to the views of the public as a whole, in this year’s survey special attention is paid to Hispanics, African Americans, parents, and teachers, all of whom were oversampled in order to obtain a sufficient number of observations. And in an effort to assess the sensitivity of respondents’ opinions to information and question wording, we embedded in this survey, as we have done in previous ones, various experiments. Responses to all questions are posted on our website, educationnext.org.

Independents lean Republican in their views of teachers unions and school spending—and support private school choice.

With Barack Obama and Mitt Romney running neck and neck, the nation’s eyes are trained on independent voters, who will likely decide the presidential election. And in the days leading up to the national conventions, education policy, though hardly at the top of the public agenda, did assume a more prominent role in both campaigns. Which candidate is best positioned to use education to bring undecided voters into the fold? The answer may be surprising.

Just one-third of independents report that President Obama has done an “excellent” or “good” job of handling education issues, while the rest assign him a “fair” or “poor” rating. And on the education policy issues that most clearly divide the parties—the role of teachers unions and support for school spending—the views of independents hew closer to those of Republicans than of Democrats. Moreover, independents are more supportive than members of either party of expanding private school choice for disadvantaged students, the centerpiece of Governor Romney’s proposals for K–12 education reform.

Whereas 25 percent of respondents to the EdNext-PEPG survey report that they are Republicans and 34 percent say that they are Democrats, fully 41 percent claim no affiliation with either major party. Of this group, 52 percent claim that they lean Democratic, while just 40 percent lean Republican. On key education issues, however, these independents express views that better align with Republicans.

No single education issue divides Republicans and Democrats more sharply than the role of teachers unions (see Figure 1). Seventy-one percent of Republicans report that the teachers unions have a generally negative effect on schools, as compared to just 29 percent of Democrats. Though independents come down in between, a majority of them (56 percent) agree with Republicans that unions have a negative effect.

Republican and Democratic voters also diverge in their preferences on school spending and teacher salaries. Figure 2 shows that when not provided with information about current spending levels, 79 percent of Democrats say that spending on public schools in their local district should increase, as compared with 50 percent of Republicans. Among independents, 57 percent support increased spending, again placing them closer to Republicans in their view of the issue. And when respondents are informed about current spending levels, the gap between Republicans and independents vanishes: 39 percent of both groups support spending increases, compared to just 51 percent of Democrats.

The same pattern holds for teacher salaries: when respondents are not provided with information about current salary levels, 60 percent of independents support increasing teacher salaries, placing them closer to Republicans (54 percent of whom support increases) than to Democrats (75 percent). Providing information on current teacher salaries in their state reduces support for salary increases among independents to 34 percent—exactly the same as among Republicans. Information also shrinks the share of Democrats supporting salary increases to 41 percent.

Governor Romney has made the expansion of school choice for disadvantaged students central to his campaign, calling for the expansion of the Washington, D.C., voucher program and for allowing low-income and special education students to use federal funds to enroll in private schools. It is perhaps surprising, then, to find that Republicans are less supportive of this concept than are Democrats (see Figure 3). Just 42 percent of Republicans express support for the idea, compared to 52 percent of Democrats. Voucher support among independents appears to be as high as (or greater than) it is among Democrats, at 54 percent.

Hispanics like public schools but not all union demands in contract negotiations.

Increasingly, both the Republican and Democratic parties have sought ways to court Hispanics. Though they lean Democratic—63 percent of Hispanic adults approve of the way President Barack Obama is handling his job as president—they are not as blue as is the African American community, 92 percent of whom give Obama a thumbs-up.

Those seeking the Hispanic vote in 2012 should know that education is an issue that resonates with the Latino community. Almost 60 percent of those we surveyed say they are “very” or “quite a bit” interested in education issues, as compared to less than 40 percent of African American and white voters.

On many topics—including school vouchers, charter schools, digital learning, student and school accountability, common core standards, and teacher recruitment and retention policies—the views of Hispanic adults do not differ noticeably from those of either whites or African Americans.

But in certain domains—estimates of school costs and school quality, support for teachers unions, teacher tenure, and teacher pensions—the views of Hispanics differ rather substantially. Their judgment of the American school is generous, perhaps because they compare public schools in the United States to much less effective institutions in Mexico, Cuba, and other parts of Latin America. They also underestimate the costs of running public schools, though they revise their thinking rather substantially about the merits of spending increases once they learn the facts. They are less supportive of unions and union demands than are African Americans.

Nearly 40 percent of Hispanic adults give the nation’s public schools a grade of an “A” or a “B” on the traditional scale used to evaluate schools (see Figure 4). When asked about the public schools in their community, no less than 55 percent give such favorable assessments. By comparison, whites and African Americans express significantly less enthusiasm about the nation’s schools. Less than 20 percent of whites and African Americans accord the nation’s schools an “A” or a “B,” and only around 40 percent give the schools in their community one of these two top grades.

Hispanic respondents think American students perform better academically than is actually the case. American 15-year-olds ranked no better than 25th among the 34 developed democracies participating in the latest round of international tests. Yet nearly 45 percent of Hispanics say student math performance in the United States ranks among the top 15 countries in the world in math. Only 30 percent of white and African American respondents place the United States that high.

Hispanic respondents also think the public schools cost a lot less than they actually do. While annual per-pupil expenditures run around $12,500, Hispanics, on average, estimate their cost at less than $5,000. Whites and African Americans estimate the costs to be more than $7,000.

The same goes for teacher salaries, which average about $56,000 a year. On average, Hispanics think teachers are paid little more than $25,000 a year; blacks, on average, think they are paid around $30,000 a year; and whites estimate salaries at $35,000.

When told just how much schools cost, however, Hispanic respondents adjust their thinking quite dramatically. When informed about actual per-pupil expenditures, Hispanics’ support for higher taxes to fund spending increases drops from 46 percent to 25 percent. When given the actual amount teachers receive, their support for higher salaries plummets nearly in half—from over 60 percent to little more than 30 percent.

Although learning the truth about costs and salaries has a similar impact on white opinion, African Americans remain more committed to higher spending. Thirty-seven percent of African Americans favor higher taxes, even when told how much is currently being spent, only a slight dip from the 42 percent favorable when that information is withheld. When given the facts about teacher salaries, African American support for higher salaries drops 20 percentage points—from 74 percent to 54 percent.

Like other ethnic groups, Hispanics do not appear especially sympathetic to teachers union demands in collective bargaining negotiations. Sixty-two percent of Hispanic adults think teachers should pay 20 percent of their pension and health care costs, as do 56 percent of African Americans.

By an overwhelming margin (87 percent), Hispanic respondents favor proposals to condition teacher tenure on their students’ making adequate progress on state tests. Whites and African Americans also favor such proposals but not to the same degree (75 percent and 80 percent, respectively). When it comes to whether teachers unions are playing a more positive or a more negative role in their local community, Hispanic adults come out in the middle—at 59 percent in support, they are more supportive than whites (45 percent) but less supportive than African Americans (75 percent).

Use test scores for evaluations, says the public (but not the teachers).

Teachers have long been paid primarily on the basis of their academic credentials and years of experience, creating in most parts of the country a lockstep pay scale that does not account for a teacher’s classroom performance. This approach is often justified on the grounds that it precludes favoritism on the part of principals, school board members, and other administrative officials.

As teacher effectiveness has become an increasingly visible policy issue, standard approaches to salary and tenure decisions are undergoing substantial change. More than 20 states now require that student test-score gains be used in key personnel decisions, often including tenure and salary determinations. Four states go so far as to prohibit a teacher from receiving a top rating if students do not exceed a certain level of accomplishment, while another 10 require that achievement gains constitute at least 50 percent of each teacher’s evaluation.

Is the public onboard with these changes? And what do teachers think about them? To find out, we randomly divided those interviewed into two groups (see Figure 5). The first group was given a stark choice: How much weight should be given to test scores and how much should be given to principal recommendations?

Given this simple dichotomy, the public says test-score gains should be given more than half the weight (62 percent) in making salary and tenure decisions. Teachers, by contrast, are prepared to place only a quarter of the weight (24 percent) on this information, with the other three-fourths of the weight being given to principal recommendations.

The second half of the sample was asked a more complex question, which required giving weights to test scores and evaluations from four different sources: principals, parents, students, and fellow teachers.

When the question was posed this way, the public and the teachers once again disagree. The public would place about one-third of the weight (32 percent) on test scores, but teachers would assign them less than one-fifth (19 percent). Conversely, teachers would give principal recommendations nearly half the weight (44 percent), while the public would give their recommendations less than one-quarter (23 percent).

Perhaps surprisingly, teachers are unenthusiastic about being evaluated by their fellow teachers. Like the rest of the public, they divide up the remaining weight more or less equally among the three remaining sources of evidence (students, parents, and fellow teachers).

An even bigger gap between teachers and the public emerges on the desirability of releasing information about teacher performance to the public at large. In both New York City and Los Angeles, newspapers have published such information, provoking an outcry among teachers, who felt their privacy had been invaded. When we asked respondents about this as a general practice, 78 percent of the public expresses support, compared to just 33 percent of teachers (see Figure 6).

When given the option of expressing neutrality on the issue (as another randomly chosen half of the sample was), 60 percent of the public still says it supports the publication of information about teacher performance, while only 13 percent is opposed, the remaining 27 percent taking the neutral position. Teacher opinion is almost the mirror image. Fifty-four percent oppose making information on test-score impacts publicly available, 30 percent express support, with the remaining 16 percent not taking a clear position either way.

Are teachers unions undermining teacher popularity?

Teachers have long held a cherished place in American popular culture. In such films as Blackboard Jungle, Stand and Deliver, and Dead Poets Society, Hollywood has highlighted the power of teachers to utterly transform the lives of their students.

But is this now changing? Are Waiting for Superman, Bad Teacher, and Won’t Back Down (forthcoming “A Takeover Tale,” cultured, Winter 2013) harbingers of a new, more skeptical depiction of teachers? At first, it would seem that public trust in teachers is widespread. When we asked half of the respondents in our survey whether they “have trust and confidence in the men and women who are teaching children in the public schools,” no less than 72 percent say “yes” (see Figure 7). This is almost exactly what Phi Delta Kappan (PDK), a publication sympathetic to teachers unions, found in its 2012 poll and about the same as in PDK polls in previous years.

When we expand the possible response categories, however, a somewhat different picture emerges. Only 4 percent of the American public has “complete” trust and confidence in teachers, and just 38 percent has “a lot” of trust and confidence in them. Meanwhile, 49 percent has “some” trust and confidence, and 9 percent has “little” trust and confidence. In other words, 58 percent of those surveyed express less than “a lot of trust and confidence” in the teaching force.

Since this is the first time the public has been asked to break its assessment of teachers into four categories, we cannot document any trends over time. But we do know that public opinion toward teachers unions—and teachers’ opinions of them, too—has turned in a negative direction. The portion who thinks that teachers unions have had a positive effect on their local schools has dropped by 7 percentage points over the past year. Among teachers, the downward shift is no less than 16 percentage points.

In this year’s survey, as we have done in the past, we asked the following question: “Some people say teachers unions are a stumbling block to school reform. Others say that unions fight for better schools and better teachers. What is your opinion? Do you think teachers unions have a generally positive view on your local schools, or do you think they have a generally negative effect?” Respondents could choose among five options: very positive, somewhat positive, neither positive nor negative, somewhat negative, and very negative.

In our polls from 2009 to 2011, we saw little change in public opinion. Around 40 percent of respondents took the neutral position, saying that unions had neither a positive nor a negative impact. The remainder were divided almost evenly, with the negative share just barely exceeding the positive.

This year, however, the teachers unions lost ground. While 41 percent of the public still takes the neutral position, the portion with a positive view of unions dropped 7 percentage points in the last year, from 29 percent to 22 percent.

The drop is even greater, in both magnitude and significance, among our nationally representative sample of teachers. At a time when, according to education journalist and union watchdog Mike Antonucci, the National Education Association has lost 150,000 members over the past two years, and projects to lose 200,000 more members by 2014, teacher discontent appears to be rising. Whereas 58 percent of teachers had a positive view of unions in 2011, only 43 percent do so in 2012. Meanwhile, the percentage of teachers holding negative views of unions nearly doubled during this period, from 17 percent to 32 percent.

But when that same question was posed in either/or terms to the public as a whole, respondents split down the middle: 51 percent say unions had a negative impact, while 49 percent say their effect was positive. Teachers, meanwhile, offered a more positive assessment. When forced to choose between just two options, 71 percent of teachers claim that unions are a force for good, whereas 29 percent see them as a stumbling block to reform.

Support for school spending is shaky.

With the U.S. economy trying to crawl back to recovery, an unemployment rate above 8 percent, and state and local governments facing the prospect of insolvency, many school districts have found it necessary to cut expenditures and personnel. In California, the cities of Stockton and San Bernardino have declared bankruptcy. In Michigan, the financially bankrupt Muskegon schools have been handed over to a for-profit charter organization. Cuts in arts programs and extracurricular activities are becoming commonplace. Nationwide, the number of school employees has drifted downward by as much as 5 percent in the past few years.

Still, the American public continues to support increasing spending on local public schools. Or at least it appears to do so (see Figure 8). Sixty-three percent of the general public says it prefers an increase in school expenditures in the local district, well up from levels in 2007 when only 51 percent of the public called for expenditure increases. Not surprisingly, teachers are even more enthusiastic about increasing expenditures, 68 percent of whom like the idea.

When one investigates the issue just a bit further, however, fractures can be detected in the public’s willingness to spend more on public schools. Though most Americans still offer their support for spending increases in the abstract, their enthusiasm ebbs rather substantially when the taxes needed to pay for the increased expenditures are broached and when information about actual expenditures and salaries is provided.

Part of the explanation for this is the widespread ignorance on the part of the general public about just how much already is spent on public schools. When asked to estimate per-pupil expenditure in their district, Americans guess that expenditures are about $6,500 annually, when in fact they are around $12,500. That is only a slightly better set of estimates than the ones given in 2009, when Americans thought $4,231 was being spent per pupil and the reality was closer to $10,000 (see “Educating the Public,” features, Summer 2009).

When respondents are told the correct figure, support for spending on public schools shifts sharply downward. Support for increased spending on our standard question drops by 20 percentage points, a much bigger drop than what was observed in 2009, when support for increased spending fell only 8 percentage points (from 46 percent to 38 percent).

In another sign of less-than-wholehearted support for an education spending spree, only 35 percent of the public says taxes should increase to fund the schools. Support drops by another 11 percentage points—to just 24 percent—when those interviewed were first told how much was currently being spent.

Teachers, who stand to benefit from increased expenditure, remain committed to more spending when told the realities of the expenditure situation in their district. Their support slips only 8 percentage points from the high of 68 percent when no information is supplied about current expenditures. But even teachers are 17 percentage points less likely to support higher taxes to fund increases in education spending.

When the subject turns from per-pupil expenditures to teacher salaries, the same pattern emerges (see Figure 9). When asked without any accompanying information, nearly two out of three Americans think that teacher salaries should go up. Among teachers, support for a salary boost registers at no less than 85 percent.

As they do on per-pupil expenditures, however, Americans hold markedly inaccurate views about actual teacher salaries. When asked to hazard a guess, Americans estimate that public school teachers in their states receive, on average, about $36,000 in salary annually. The true figure, even without accounting for benefits, pensions, and the like, sits at about $56,000 nationwide.

Support for higher salaries plummets, however, when Americans are told how much teachers actually make in their states. Of those given the facts, only 36 percent favor an increase, which amounts to a whopping 28-percentage-point decline from the 64 percent favoring an increase when no information is supplied.

When teachers were given accurate information about salary levels in their state, their support slips by only 10 percentage points, probably because they are thinking about their own paycheck. Also, they have a better sense of teacher salaries in their state than the public has, estimating them to be about $44,000 annually.

Is public support for charters really that much higher than for vouchers and tax credits?

As a policy reform, school choice shows no signs of slowing. The number of states with school-voucher and tax-credit programs has escalated since 2010, the number of students attending charter schools climbs steadily year by year, and new technologies for online learning are being promoted by a cascade of new entrepreneurs.

The contours of elite debate about school choice, however, are not replicated in the larger public. While charter schools and digital learning are thought to be the safest choice options for political elites to promote, tax credits are even more popular than charters, and vouchers, the most controversial proposal, also command the support of half the population when the idea is posed in an inviting way.

Vouchers and tax credits. When it comes to school vouchers, apparent levels of public support turn on the wording of the question. For the past two years, PDK has asked whether respondents “favor or oppose allowing students and parents to choose a private school to attend at public expense.” Even with the rather loaded “at public expense” phrasing, PDK reported that support shifted upward from 34 percent to 44 percent between 2011 and 2012.

If one asks the question in a more inviting manner, as we have, support jumps further still (see Figure 10). Told about a proposal “that would give low-income families with children in public schools a wider choice, by allowing them to enroll their children in private schools instead, with government helping to pay the tuition,” 50 percent of the American public comes out in support and 50 percent expresses opposition.

Still, support for vouchers does not match public willingness to back tax credits, even though most economists think the difference between vouchers and tax credits more a matter of style than substance. Nearly three-fourths (72 percent) of the public favors a “tax credit for individual and corporate donations that pay for scholarships to help low-income parents send their children to private schools.” We find little evidence that support for tax credits has changed significantly since 2011.

Charters. Figure 11 shows that when given a choice of supporting or opposing charter schools, 62 percent of the public says it favors ”the formation of charter schools,” nearly identical to what PDK finds (66 percent favoring ”the idea of charter schools”). Support for charters, however, is softer than it might seem. When respondents are given the opportunity to take a neutral position that neither supports nor opposes charters, no less than 41 percent choose that option. Among the remainder, the split is nearly three to one in favor of charters.

Meanwhile, public knowledge about charters remains as impoverished as ever. As our survey did two years ago, we asked respondents a variety of factual questions: whether charter schools can hold religious services, charge tuition, receive more or less per-pupil funding than traditional public schools, and are legally obligated to admit students randomly when oversubscribed. We found little change in the level of public information over the past two years. Large percentages of respondents still say they don’t know the answers to these questions. Among those who hazard a guess, they are as likely to give the wrong answer as the correct one. Although teachers do a better job of accurately identifying the characteristics of charter schools, even a majority of teachers get many of the answers wrong or say they don’t know.

Online education. As major universities—Harvard, Stanford, MIT, and others—are joining community colleges and state universities in a nationwide dash toward online learning in higher education, many states are exploring ways of incorporating new digital technologies into secondary schools.

A substantial share of both the public and the teaching force seems ready to consider the expansion of online learning. When asked if high school students should be allowed to take “approved classes either online or in school,” opinion splits down the middle, with a bare majority (53 percent to 47 percent) favoring the idea. Teachers are more enthusiastic, among whom no less than 61 percent feel students should be given an online option.

The public, however, is not equally enthusiastic about all uses of the online tool, nor is support for the idea gaining strength. The most popular uses are for rural education and advanced course taking. Fifty-eight percent of those surveyed think students in rural areas should have online opportunities, with only 14 percent opposing the idea. That is down modestly from the 64 percent who supported this use in 2008. Similar percentages of support and opposition are expressed for advanced courses taken online for college credit. But once again, levels of support have slipped since 2008 (from 68 percent to 57 percent in 2012).

Less popular are online courses for dropouts and home schoolers. Only 44 percent favor, and 30 percent oppose, using public funding to help dropouts take courses online. Many home schoolers find online courses to be a valuable tool, but the public remains dubious. Only 28 percent favors public funding for such uses, and 38 percent opposes it. Those percentages have not changed materially since 2008.

William G. Howell is professor of American politics at the University of Chicago. Martin R. West is assistant professor of education at the Harvard Graduate School of Education and deputy director of Harvard’s Program on Education Policy and Governance. Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution.

But what is comprehensive implementation? True, we’ve heard much palaver about what the Common Core portends for assessment, for teachers’ professional development, and for curricular/instructional materials. All true, all crucial, and all probably the most urgent. But these issues are also just the tip of the CCSS iceberg, most of which remains invisible under water. What I haven’t seen yet is clear recognition that the Common Core, taken seriously, eventually changes everything in American education and that implementation, done right, must be comprehensive.

Which means what? Start with a substantial analogy: World War II. A new book profiles General Albert C. Wedemeyer, who was assigned by General Marshall to the Army’s “War Plans Department” as the conflict loomed and (I quote the Wall Street Journal’s book review) “tasked…with reducing America’s mobilization requirements to a single document.” Then FDR asked Wedemeyer’s team to turn it “into a blueprint on how to defeat America’s likely enemies in a future war.” The book explains:

Completed in an astonishing 90 days, this plan laid down all the critical politico-military-industrial assumptions for the looming conflict, correctly identifying America’s adversaries and where the main fighting would take place and estimating the industrial capacity needed to feed the war machines of China, the Soviet Union, Great Britain and the United States and how much war materiel could be spared to allies. Wedemeyer proposed overrunning Germany and Japan with an army of nearly nine million draftees, a number that he concluded would leave sufficient factory workers and farmers back home to feed the troops and keep the tanks, bombers and artillery shells rolling off assembly lines. He also called for an invasion of Europe in 1943, before Germany could strengthen its defenses.

The Wedemeyer plan was carried out only in part. (Churchill was convinced that an early invasion of the European continent would end disastrously.) But, dramatic though this may seem, full-on Common Core implementation will demand a plan a plan of similar comprehensiveness and vision. I don’t know who will play the role of Albert Wedemeyer, but maybe we can accelerate the process by offering a provisional table of contents.

Here’s my list of  topics that the plan should include, submitted  with some humility, as I’ve surely overlooked important items (bring ‘em on!), and with some trepidation, as understanding these implications may cause Common Core skeptics to stiffen their resistance.

Curriculum

1. Curriculum guides for teachers

The Common Core sets forth what students needs to have learned by the end of each year. It doesn’t help teachers with “scope and sequence,” much less lesson planning. Not all teachers want such help—and there’s much shrieking about “national curriculum”—but some will welcome guidance. (Keep it voluntary!)

2. Textbooks that are truly aligned
Every big ELA- and math-textbook publisher has already declared that its products are “aligned” with the Common Core, but mostly that’s not true. Who is going to apply the excellent publishers’ guidelines produced by Student Achievement Partners and actually rate the textbooks (and fast-proliferating digital resources) on how well they’re aligned?

3. Additional instructional materials

Traditional U.S. textbooks and “reading programs”—bulky, lumbering, and linear—aren’t going to work very well with the instructional demands of the Common Core. Teachers will need to be able to muster instructional resources from many sources, including electronic ones. (Some excellent nonprofit groups are already at work on materials that will end up being freely available, which also portends a radical transformation of the textbook market!)

4. Curriculum narrowing

How do we keep K-12 education from being whittled down to the two subjects in the Common Core? Yes, “next generation” science standards are in the works. But what about history, geography, civics, languages, and the arts? Health and phys ed? Where do they fit? What standards will apply? How will they be taught and assessed?

Staff

5. Professional development

Hundreds of thousands of current teachers need to update, alter, and amplify their own knowledge base and pedagogical arsenal if they’re to succeed in imparting the Common Core to their pupils.

6. Teacher (and principal) preparation

Pretty nearly every teacher-preparation program in the land, whether university-based or “alternative,” will need to revamp its own standards and curriculum if it’s to prepare tomorrow’s instructors to impart the Common Core to their students. Ditto for those who purport to train school leaders. A lot of professors will need to change their ways, too!

7. Evaluations

These will change, too, with implications for everything that is attached to them (tenure decisions, merit pay, layoffs, and more). Will it get harder or easier to make “value-added” calculations at the classroom level once Common Core assessments kick in? What about rubrics for teacher observations? (And how well trained will those observers be in what to look for in a Common Core classroom?)

Schools

8. The day and year

I wager that today’s standard school day and year will prove insufficient for many kids to master the Common Core plus everything else that they need to learn. Can instructional time be individualized, too, in school or online? What are the budget implications?

9. Promotion and graduation requirements

Some states have third-grade “reading guarantees,” but what about the rest of the K-12 sequence? Will going from sixth grade to seventh hinge on a student having mastered the Common Core standards for sixth? What about entering high school? Earning a diploma? Will this continue to be based on Carnegie units and course credits or on actual mastery? (And what about subjects outside ELA and math?) What does Common Core portend for the two dozen or so states with high-school-graduation tests that are pegged to yesterday’s ninth- or tenth-grade expectations?

10. Internal organization of schools

Though the Common Core is built around grade levels, kids don’t learn at the same speed—and individualization of instruction grows ever more important. What about moving kids forward as they master stuff rather than through lock-step progressions? Why can’t one be in third grade for ELA, say, and fourth or fifth for math? How about those who will need five years rather than four to master the challenges of high school? (Today they’re counted as “drop outs” in most states’ statistics!) And since we can no longer afford to individualize by shrinking class size further, we’ll need to rely more on technology—which the new assessments also need—and on more flexible ways of organizing school itself.

11. Preschooling

Remind yourself what the Common Core expects Kindergartners to learn (review page ten of the ELA standards or page eleven of math standards). Then ask yourself what must a child know and be able to do upon entry into Kindergarten to maximize the odds that she will be ready to succeed there. Then ponder how few of today’s preschool programs (Head Start included) have standards, curricula, and staff that are up to this challenge? And how many of today’s needy pre-Kindergartners don’t even have access to those programs?

12. Technology

This stuff is evolving at warp speed, with profound implications for schooling and for kids’ lives. Much of it’s about communication and entertainment, but as those realms overlap more with formal education—for good and ill—and as more kids gain 24/7 access to all of them, what will this mean for K-12 schooling? How much of it will actually take place in school? How much will require flesh-and-blood instructors—and of what sorts? And what’s to become of libraries, book rooms, backpacks, and the rest? Picture every school kid with her own iPad in hand…

Assessment and accountability

13. Tests

New Common Core assessments are under development for deployment in 2014-15—and let’s hope they turn out well—but for states and districts to make good use of them means rethinking their entire approach to student assessment, right down to the classroom level. How will the end of a “six-week unit,” for example, be assessed? What about those end-of-week vocabulary reviews? Weekly reports to parents on what was and wasn’t learned?

14. Accountability systems

Most state accountability systems incorporate multiple factors, including but not limited to student test scores, which are geared to current state standards and tests. Every state does it differently—and those differences are apt to widen as federal NCLB prescriptions ease with recent waivers and (maybe someday) ESEA reauthorization. States that embrace the Common Core will need to reconstruct their accountability systems, as will districts that have their own.

15. Alternate assessments

Then there’s the GED and other ways of gauging “equivalency” for those who don’t earn a conventional on-schedule diploma. Big changes are afoot there, but will the new tests equate to the Common Core—and redress the long-standing problem of the GED: namely that people possessing it don’t fare much better in life than dropouts?

The reception

16. Graduation rates

What happens, politically, when graduation rates plummet and dropout rates soar, at least for a few years? Are states and communities ready for this? Nobody ever is. But does that mean we’ll “phase in” the more rigorous graduation expectations? How long will that take?

17. Higher education

Once Common Core rigor takes hold (if ever) of high-school-exit expectations, will our universities actually accept that diploma as proof of college readiness? Will it yield automatic admission and placement into credit-bearing college courses? If not, why should K-12 students (and parents and taxpayers) take it seriously? If it does, what happens to faculty members who have been teaching remedial courses? What happens to collegiate English and math classes if entering students are truly prepared? Will that compulsory first-year writing course still be needed?

18. Career education

The Common Core claims to be geared to college and career readiness. We know that not everyone is headed to (or belongs in) college, at least not the four-year kind. But what exactly are the implications for employer expectations, hiring practices, and on-the-job training? (How about the armed forces as a major employer?) How about secondary-level technical-vocational education? Will Common Core expectations make it into those institutions, too? That will likely mean major-league curricular and instructional alteration.

19. NCLB and other federal policy

Everybody knows this but it needs underscoring: When Congress gets around to reauthorizing the Elementary and Secondary Education Act—and other programs such as IDEA, Head Start and TRIO—it must contend with the changed expectations that most states will have for their students and the implications of those changes for the special populations, additional services, and so forth that Uncle Sam focuses on. And it must do so without turning the Common Core itself into a federal mandate. (Remember, four states want no part of it—and at least a few more are apt to back out along the way.)

20. NAEP

The role of the Nation’s Report Card will evolve, too. If most states end up using new English language arts and math assessments, calibrated to Common Core standards, at the individual, building, district, and state levels, there will be less cause to press for NAEP (and PISA, TIMSS, etc.) to be administered to everybody. But NAEP will remain the crucial external auditor for Common Core states and those that do their own thing. At the same time, the curricular frameworks that determine what NAEP assesses may need to be re-examined.

Yikes. It’s sort of scary. Daunting. Politically and organizationally challenging. Expensive in a time of tight budgets. Disruptive to myriad entrenched institutions and practices. But if we don’t wrap our minds around the totality of it, we may not win this war. Are you listening, General Wedemeyer?

-Chester E. Finn, Jr.

This blog entry first appeared on the Fordham Institute’s Flypaper blog.

“The United States’ failure to educate its students leaves them unprepared to compete and threatens the country’s ability to thrive in a global economy.” Such was the dire warning issued recently by an education task force sponsored by the Council on Foreign Relations. Chaired by former New York City schools chancellor Joel I. Klein and former U.S. secretary of state Condoleezza Rice, the task force said the country “will not be able to keep pace—much less lead—globally unless it moves to fix the problems it has allowed to fester for too long.” Along much the same lines, President Barack Obama, in his 2011 State of the Union address, declared, “We need to out-innovate, out-educate, and out-build the rest of the world.”

Although these proclamations are only the latest in a long series of exhortations to restore America’s school system to a leading position in the world, the U.S. position remains problematic. In a report issued in 2010, we found only 6 percent of U.S. students performing at the advanced level in mathematics, a percentage lower than those attained by 30 other countries. And the problem isn’t limited to top-performing students. In 2011, we showed that just 32 percent of 8th graders in the United States were proficient in mathematics, placing the U.S. 32nd when ranked among the participating international jurisdictions (see “Are U.S. Students Ready to Compete?” features, Fall 2011).

Admittedly, American governments at every level have taken actions that would seem to be highly promising. Federal, state, and local governments spent 35 percent more per pupil—in real-dollar terms—in 2009 than they had in 1990. States began holding schools accountable for student performance in the 1990s, and the federal government developed its own nationwide school-accountability program in 2002.

And, in fact, U.S. students in elementary school do seem to be performing considerably better than they were a couple of decades ago. Most notably, the performance of 4th-grade students on math tests rose steeply between the mid-1990s and 2011. Perhaps, then, after a half century of concern and efforts, the United States may finally be taking the steps needed to catch up.

To find out whether the United States is narrowing the international education gap, we provide in this report estimates of learning gains over the period between 1995 and 2009 for 49 countries from most of the developed and some of the newly developing parts of the world. We also examine changes in student performance in 41 states within the United States, allowing us to compare these states with each other as well as with the 48 other countries.

Data and Analytic Approach

Data availability varies from one international jurisdiction to another, but for many countries enough information is available to provide estimates of change for the 14-year period between 1995 and 2009. For 41 U.S. states, one can estimate the improvement trend for a 19-year period—from 1992 to 2011. Those time frames are extensive enough to provide a reasonable estimate of the pace at which student test-score performance is improving in countries across the globe and within the United States. To facilitate a comparison between the United States as a whole and other nations, the aggregate U.S. trend is estimated for that 14-year period and each U.S. test is weighted to take into account the specific years that international tests were administered. (Because of the difference in length and because international tests are not administered in exactly the same years as the NAEP tests, the results for each state are not perfectly calibrated to the international tests, and each state appears to be doing slightly better internationally than would be the case if the calibration were exact. The differences are marginal, however, and the comparative ranking of states is not affected by this discrepancy.)

Our findings come from assessments of performance in math, science, and reading of representative samples in particular political jurisdictions of students who at the time of testing were in 4th or 8th grade or were roughly ages 9‒10 or 14‒15. The political jurisdictions may be nations or states. The data come from one series of U.S. tests and three series of tests administered by international organizations. Using the equating method described in the methodology sidebar, it is possible to link states’ performance on the U.S. tests to countries’ performance on the international tests, because representative samples of U.S. students have taken all four series of tests.

Comparisons across Countries

In absolute terms, the performance of U.S. students in 4th and 8th grade on the NAEP in math, reading, and science improved noticeably between 1995 and 2009. Using information from all administrations of NAEP tests to students in all three subjects over this time period, we observe that student achievement in the United States is estimated to have increased by 1.6 percent of a standard deviation per year, on average. Over the 14 years, these gains equate to 22 percent of a standard deviation. When interpreted in years of schooling, these gains are notable. On most measures of student performance, student growth is typically about 1 full standard deviation on standardized tests between 4th and 8th grade, or about 25 percent of a standard deviation from one grade to the next. Taking that as the benchmark, we can say that the rate of gain over the 14 years has been just short of the equivalent of one additional year’s worth of learning among students in their middle years of schooling.

Yet when compared to gains made by students in other countries, progress within the United States is middling, not stellar (see Figure 1). While 24 countries trail the U.S. rate of improvement, another 24 countries appear to be improving at a faster rate. Nor is U.S. progress sufficiently rapid to allow it to catch up with the leaders of the industrialized world.

Students in three countries—Latvia, Chile, and Brazil—improved at an annual rate of 4 percent of a standard deviation, and students in another eight countries—Portugal, Hong Kong, Germany, Poland, Liechtenstein, Slovenia, Colombia, and Lithuania—were making gains at twice the rate of students in the United States. By the previous rule of thumb, gains made by students in these 11 countries are estimated to be at least two years’ worth of learning. Another 13 countries also appeared to be doing better than the U.S., although the differences between the average improvements of their students and those of U.S. students are marginal.

Student performance in nine countries declined over the same 14-year time period. Test-score declines were registered in Sweden, Bulgaria, Thailand, the Slovak and Czech Republics, Romania, Norway, Ireland, and France. The remaining 15 countries were showing rates of improvement that were somewhat slower than those of the United States.

In sum, the gains posted by the United States in recent years are hardly remarkable by world standards. Although the U.S. is not among the 9 countries that were losing ground over this period of time, 11 other countries were moving forward at better than twice the pace of the United States, and all the other participating countries were changing at a rate similar enough to the United States to be within a range too close to be identified as clearly different.

Which States Are the Big Gainers?

Progress was far from uniform across the United States. Indeed, the variation across states was about as large as the variation among the countries of the world. Maryland won the gold medal by having the steepest overall growth trend. Coming close behind, Florida won the silver medal and Delaware the bronze. The other seven states that rank among the top-10 improvers, all of which outpaced the United States as a whole, are Massachusetts, Louisiana, South Carolina, New Jersey, Kentucky, Arkansas, and Virginia. See Figure 2 for an ordering of the 41 states by rate of improvement.

Iowa shows the slowest rate of improvement. The other four states whose gains were clearly less than those of the United States as a whole are Maine, Oklahoma, Wisconsin, and Nebraska. Note, however, that because of nonparticipation in the early NAEP assessments, we cannot estimate an improvement trend for the 1992‒2011 time period for nine states—Alaska, Illinois, Kansas, Montana, Nevada, Oregon, South Dakota, Vermont, and Washington.

Cumulative growth rates vary widely. Average student gains over the 19-year period in Maryland, Florida, Delaware, and Massachusetts, with annual growth rates of 3.1 to 3.3 percent of a standard deviation, were some 59 percent to 63 percent of a standard deviation over the time period, or better than two years of learning. Meanwhile, annual gains in the states with the weakest growth rates—Iowa, Maine, Oklahoma, and Wisconsin—varied between 0.7 percent and 1.0 percent of a standard deviation, which translate over the 19-year period into learning gains of one-half to three-quarters of a year. In other words, the states making the largest gains are improving at a rate two to three times the rate in states with the smallest gains.

Had all students throughout the United States made the same average gains as did those in the four leading states, the U.S. would have been making progress roughly comparable to the rate of improvement in Germany and the United Kingdom, bringing the United States reasonably close to the top-performing countries in the world.

Is the South Rising Again?

Some regional concentration is evident within the United States. Five of the top-10 states were in the South, while no southern states were among the 18 with the slowest growth. The strong showing of the South may be related to energetic political efforts to enhance school quality in that region. During the 1990s, governors of several southern states—Tennessee, North Carolina, Florida, Texas, and Arkansas—provided much of the national leadership for the school accountability effort, as there was a widespread sentiment in the wake of the civil rights movement that steps had to be taken to equalize educational opportunity across racial groups. The results of our study suggest those efforts were at least partially successful.

Meanwhile, students in Wisconsin, Michigan, Minnesota, and Indiana were among those making the fewest average gains between 1992 and 2011. Once again, the larger political climate may have affected the progress on the ground. Unlike in the South, the reform movement has made little headway within midwestern states, at least until very recently. Many of the midwestern states had proud education histories symbolized by internationally acclaimed land-grant universities, which have become the pride of East Lansing, Michigan; Madison, Wisconsin; St. Paul, Minnesota; and Lafayette, Indiana. Satisfaction with past accomplishments may have dampened interest in the school reform agenda sweeping through southern, border, and some western states.

Are Gains Simply Catch-ups?

According to a perspective we shall label “catch-up theory,” growth in student performance is easier for those political jurisdictions originally performing at a low level than for those originally performing at higher levels. Lower-performing systems may be able to copy existing approaches at lower cost than higher-performing systems can innovate. This would lead to a convergence in performance over time. An opposing perspective—which we shall label “building-on-strength theory”—posits that high-performing school systems find it relatively easy to build on their past achievements, while low-performing systems may struggle to acquire the human capital needed to improve. If that is generally the case, then the education gap among nations and among states should steadily widen over time.

Neither theory seems able to predict the international test-score changes that we have observed, as nations with rapid gains can be identified among countries that had high initial scores and countries that had low ones. Latvia, Chile, and Brazil, for example—were relatively low-ranking countries in 1995 that made rapid gains, a pattern that supports catch-up theory. But consistent with building-on-strength theory, a number of countries that have advanced relatively rapidly were already high-performing in 1995—Hong Kong and the United Kingdom, for example. Overall, there is no significant pattern between original performance and changes in performance across countries.

But if neither theory accounts for differences across countries, catch-up theory may help to explain variation among the U.S. states. The correlation between initial performance and rate of growth is a negative 0.58, which indicates that states with lower initial scores had larger gains. For example, students in Mississippi and Louisiana, originally among the lowest scoring, showed some of the most striking improvement.  Meanwhile, Iowa and Maine, two of the highest-performing entities in 1992, were among the laggards in subsequent years (see Figure 3). In other words, catch-up theory partially explains the pattern of change within the United States, probably because the barriers to the adoption of existing technologies are much lower within a single country than across national boundaries.

Catch-up theory nonetheless explains only about one-quarter of the total state variation in achievement growth. Notice in Figure 3 that some states are well below the line (e.g., Iowa and Maine) while others are well above  (e.g., Maryland and Massachusetts). Note also that Iowa, Maine, Wisconsin, and Nebraska rank well below that line. Closing the interstate gap does not happen automatically.

What about Spending Increases?

According to another popular theory, additional spending on education will yield gains in test scores. To see whether expenditure theory can account for the interstate variation, we plotted test-score gains against increments in spending between 1990 and 2009. As can be seen from the scattering of states into all parts of Figure 4, the data offer precious little support for the theory. Just about as many high-spending states showed relatively small gains as showed large ones. Maryland, Massachusetts, and New Jersey enjoyed substantial gains in student performance after committing substantial new fiscal resources. But other states with large spending increments—New York, Wyoming, and West Virginia, for example—had only marginal test-score gains to show for all that additional expenditure. And many states defied the theory by showing gains even when they did not commit much in the way of additional resources. It is true that on average, an additional $1000 in per-pupil spending is associated with an annual gain in achievement of one-tenth of 1 percent of a standard deviation. But that trivial amount is of no statistical or substantive significance. Overall, the 0.12 correlation between new expenditure and test-score gain is just barely positive.

Who Spends Incremental Funds Wisely?

Some states received more educational bang for their additional expenditure buck than others. To ascertain which states were receiving the most from their incremental dollars, we ranked states on a “points per added dollar” basis. Michigan, Indiana, Idaho, North Carolina, Colorado, and Florida made the most achievement gains for every incremental dollar spent over the past two decades. At the other end of the spectrum are the states that received little back in terms of improved test-score performance from increments in per-pupil expenditure—Maine, Wyoming, Iowa, New York, and Nebraska.

We do not know, however, which kinds of expenditures prove to be the most productive or whether there are other factors that could explain variation in productivity among the states.

Causes of Change

There is some hint that those parts of the United States that took school reform the most seriously—Florida and North Carolina, for example—have shown stronger rates of improvement, while states that have steadfastly resisted many school reforms (Iowa and Wisconsin, for instance), are among the nation’s test-score laggards. But the connection between reforms and gains adduced thus far is only anecdotal, not definitive. Although changes among states within the United States appear to be explained in part by catch-up theory, we cannot pinpoint the specific factors that underlie this. We are also unable to find significant evidence that increased school expenditure, by itself, makes much of a difference. Changes in test-score performance could be due to broader patterns of economic growth or varying rates of in-migration among states and countries. Of course, none of these propositions has been tested rigorously, so any conclusions regarding the sources of educational gains must remain speculative.

Have We Painted Too Rosy a Portrait?

Even the extent of the gains that have been made are uncertain. We have estimated gains of 1.6 percent of a standard deviation each year for the United States as a whole, or a total gain of 22 percent of a standard deviation over 14 years, a forward movement that has lifted performance by nearly a full year’s worth of learning over the entire time period. A similar rate of gain is estimated for students in the industrialized world as a whole (as measured by students residing in the 49 participating countries). Such a rate of improvement is plausible, given the increased wealth in the industrialized world and the higher percentages of educated parents than in prior generations.

However, it is possible to construct a gloomier picture of the rate of the actual progress that both the United States and the industrialized world as a whole have made. All estimations are normed against student performances on the National Assessment of Educational Progress in 4th and 8th grades in 2000.  Had we estimated gains from student performance in 8th grade only on the grounds that 4th-grade gains are meaningless unless they are observed for the same cohort four years later, our results would have shown annual gains in the United States of only 1 percent of a standard deviation. The relative ranking of the United States remains essentially unchanged, however, as the estimated growth rates for 8th graders in other countries is also lower than for estimates that include students in 4th grade (see the unabridged report, Appendix B, Figure B1).

A much reduced rate of progress for the United States emerges when we norm the trends on the PISA 2003 test rather than the 2000 NAEP test. In this case, we would have estimated annual growth rate for the United States of only one-half of 1 percent of a standard deviation. A lower annual growth rate for other countries would also have been estimated, and again the relative ranking of the United States would remain unchanged (see the unabridged report, Appendix B, Figure B2).

An even darker picture emerges if one turns to the results for U.S. students at age 17, for whom only minimal gains can be detected over the past two decades. We have not reported the results for 17-year-old students, because the test administered to them does not provide information on the performance of students within individual states, and no international comparisons are possible for this age group.

Students themselves and the United States as a whole benefit from improved performance in the early grades only if that translates into measurably higher skills at the end of school. The fact that none of the gains observed in earlier years translate into improved high-school performance leaves one to wonder whether high schools are effectively building on the gains achieved in earlier years. And while some scholars dismiss the results for 17-year-old students on the grounds that high-school students do not take the test seriously, others believe that the data indicate that the American high school has become a highly problematic educational institution. Amidst any uncertainties one fact remains clear, however: the measurable gains in achievement accomplished by more recent cohorts of students within the United States are being outstripped by gains made by students in about half of the other 48 participating countries.

Politics and Results

The failure of the United States to close the international test-score gap, despite assiduous public assertions that every effort would be undertaken to produce that objective, raises questions about the nation’s overall reform strategy. Education goal setting in the United States has often been  utopian rather than realistic. In 1990, the president and the nation’s governors announced the goal that all American students should graduate from high school, but two decades later only 75 percent of 9th graders received their diploma within four years after entering high school. In 2002, Congress passed a law that declared that all students in all grades shall be proficient in math, reading, and science by 2014, but in 2012 most observers found that goal utterly beyond reach. Currently, the U.S. Department of Education has committed itself to ensuring that all students shall be college- or career-ready as they cross the stage on their high-school graduation day, another overly ambitious goal. Perhaps the least realistic goal was that of the governors in 1990 when they called for the U.S. to be first in the world in math and science by 2000. As this study shows, the United States is neither first nor catching up.

Consider a more realistic set of objectives for education policymakers, one that is based on experiences from within the United States itself. If all U.S. states could increase their performance at the same rate as the highest-growth states—Maryland, Florida, Delaware, and Massachusetts—the U.S. improvement rate would be lifted by 1.5 percentage points of a standard deviation annually above the current trend line. Since student performance can improve at that rate in some countries and in some states, then, in principle, such gains can be made more generally. Those gains might seem small but when viewed over two decades they accumulate to 30 percent of a standard deviation, enough to bring the United States within the range of, or to at least keep pace with, the world’s leaders.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Paul E. Peterson is director of the Harvard Program on Education Policy and Governance. Ludger Woessmann is head of the Department of Human Capital and Innovation at the Ifo Institute at the University of Munich. An unabridged version of this report is available at hks.harvard.edu/pepg/

Florida’s gains in reading and math achievement, as measured by the National Assessment of Educational Progress

Among the 50 states, Florida’s gains on the National Assessment of Educational Progress (NAEP) between 1992 and 2011 ranked second only to Maryland’s (see “Is the U.S. Catching Up?” features, Fall 2012). Florida’s progress has been particularly impressive in the early grades. In 1998, Florida scored about one grade level below the national average on the 4th-grade NAEP reading test, but it was scoring above that average by 2003, and made further gains in subsequent years (see Figure 1). Scores on Florida’s own state examinations revealed an equally dramatic upward trend.

Many have cited the series of accountability and choice reforms that Florida adopted between 1998 and 2006, under the leadership of Governor Jeb Bush, as the driving force behind the large and rapid improvement in student achievement (see “Advice for Education Reformers: Be Bold!” features, Fall 2012). Others have insisted that Florida’s NAEP scores do not represent true improvements in student reading achievement. Boston College professor Walter Haney, for example, argues that the scores are “dubious” and “highly misleading.” He contends that it is “abundantly clear” that Florida’s aggregate test-score improvements are a mirage caused by changes in the students enrolled in the 4th grade after the state began holding back a large number of 3rd-grade students in 2004 (all school years are reported by the year in which they ended). His argument has been touted by other researchers, most notably by some at the National Education Policy Center, and it has been cited in testimony presented before state legislatures considering the adoption of Florida-style reforms.

It is certainly true, as Haney has said, that one of the Florida reforms was to curtail social promotion of underachieving students from 3rd to 4th grade. In most school districts, students who do not warrant promotion on academic grounds move on to the next grade regardless, because many educators believe that keeping students with their peer group is desirable. But in Florida, those students who completed 3rd grade in the spring of 2003 and since have had to meet a minimum threshold on the Florida Comprehensive Assessment Test (FCAT) reading examination in order to be promoted to the 4th grade, unless they receive a special waiver. As a result, the percentage of students retained in 3rd grade increased substantially. In the two years prior to the policy change, only 2.9 percent of 3rd-grade students were retained, while in the two years following the policy’s implementation, 11.7 percent of Florida’s 3rd-grade students were told they had to remain in the same grade for the coming year.

Haney and others have concluded that this policy change artificially drove up 4th-grade test scores, because it removed from the cohort of students tested those who were retained in 3rd grade, the very students most likely to score the lowest on standardized tests. Although the point would seem to be well worth considering, it has not been subjected to serious empirical analysis. Does the holding back of the lowest-performing students in 3rd grade explain all the 4th-grade gains in Florida, as Haney contends? Does it explain some of the gains? Or none at all? The best way to answer the question is to look at changes in student test-score performance among those in 3rd grade for the first time, as their test scores are unaffected by the retention policy. If the gains observed for 4th graders were a function of differences in the type of students entering that grade due to the retention policy, then the performance of those entering 3rd grade should look essentially the same after 2002 as it did before the retention policy was put into place.

Drawing on information on student performance available from the Florida Department of Education, I was able to analyze test-score trends of students enrolled in the 3rd grade for the first time. I find that the gains among initial 3rd graders were not as dramatic as those shown on the 4th-grade NAEP, thereby suggesting that the 4th-grade scores did create the appearance of steeper achievement growth than actually took place. Nonetheless, the gains among initial 3rd graders were very substantial, about 0.36 standard deviations between 1998 and 2009, and more than enough to justify Florida’s claims that its gains have outpaced those in most other states.

Reading Test Scores for 3rd Graders

I first analyze changes over time in the FCAT test scores of students in their initial 3rd-grade year in order to discern the extent to which Florida’s elementary-school students made true achievement gains during the period in question. Because the state has not yet identified students for retention, the test scores of students the first time they are in the 3rd grade are not affected by any change in the student cohort resulting from the retention policy.

The administrative data set for the State of Florida contains individual test scores and demographic information for the universe of test-taking students in grades 3 through 10 in Florida from 2001 through 2009. The data set includes a unique student identifier, which allows me to follow the progress of each student over time and to determine which students have been retained.

Figure 2 shows the changes since 2001 in the performance of students at the 25th, 50th, and 75th percentiles in their initial 3rd-grade year. The figure documents clear positive movement across the test-score distribution for the first cohort of students that needed to reach a minimal score on the FCAT exam in order to be promoted from the 3rd to the 4th grade (2003). The achievement distribution makes another leap forward the following year (2004), which was the first year that began with a sizable number of retained students due to implementation of the policy. Student achievement continued to grow in subsequent years.

The test-score improvements shown on the figure are substantial. By 2009, the median reading test score of students in their initial 3rd-grade year had improved by more than one-third of a standard deviation since 2001, as had nearly all points on the distribution. A gain of this magnitude amounts to roughly a full year of academic progress for students in the early elementary grades. The test-score gains among the state’s lowest-performing students were even more impressive; for instance, students at the 10th percentile improved by more than half a standard deviation. The gains made by initial 3rd-grade students on the math exam are even larger than the gains in reading at all points on the distribution.

The results do suggest, however, that the aggregate test scores on the 4th-grade NAEP could well be inflated by the retention policy. The improvement in the median reading score for those students entering 3rd grade is smaller than the NAEP increase for 4th graders over the same time period. Even so, the 3rd-grade gains remain noteworthy enough to substantiate the basic claims of those who praise the Florida track record.

Rescaling NAEP Reading Scores

To assess how well Florida performed relative to the rest of the nation, one can use the results for initial 3rd-grade students on the FCAT to rescale the state’s 4th-grade scores on the NAEP reading exam. The rescaling assumes that test-score improvements on the FCAT for cohorts in their initial year as 3rd graders are a good proxy for gains in reading achievement made by Florida’s students in the next elementary grade. Though imperfect, this assumption is justified to the extent that most consider 4th-grade NAEP scores to be an assessment of overall elementary-school performance.

Because Florida did not participate in the NAEP in 2000, I use as the state’s baseline score its median score on the 4th-grade NAEP reading exam in 1998. Thus, I also assume that the state made no meaningful gains in 4th-grade reading between 1998 and 2000 that would have shown up on NAEP, which squares with the scores on the state’s own reading assessment. I then use the improvements of the median reading test score for initial 3rd-grade students on the FCAT since 2001 in order to rescale the state’s mean NAEP test score in the spring of the same year.

In addition to providing the originally reported NAEP score trend in median scores between 1998 and 2009 for Florida and the United States as a whole, Figure 1 shows the rescaled trend in Florida after making the adjustment described above. The first class affected by the retention policy entered the 4th grade during the 2004 school year, and thus the first NAEP score that could have been influenced by the exclusion of low-performing students from the 4th-grade NAEP sample was the spring 2005 administration.

The figure shows that Florida’s reading gains prior to the introduction of the policy were actually larger on the NAEP than on the FCAT. Such a difference cannot be explained by the retention policy, because students had not yet been retained. After introduction of the policy, Florida’s achievement on the state exam after accounting for sample selection increased between 2003 and 2005 in a way that did not show up on the NAEP scores. But the state’s NAEP scores quickly caught up to the FCAT performance. Adjusting the state’s NAEP scores for sample selection in 2007 and 2009 leads to a decrease in the state’s performance of about 0.07 and 0.08 standard deviations, respectively. However, Florida’s adjusted median score remains above the median score for all U.S. public-school students, and it continues to show substantial improvements relative to the prior decade.

Even after the adjustment, Florida’s students still made larger gains in reading than did the rest of the nation. The national gain, at 7 points (or about 0.19 standard deviations), was only slightly larger than half Florida’s rate. Prior to the adjustment, only Washington, D.C., made larger gains on the 4th-grade NAEP reading exam during this period. After the adjustment, only D.C. and Delaware made a larger test-score improvement.

What Reforms Might Have Produced the Reading Gains?

Putting a finger on exactly which policy changes produced the test-score improvements is remarkably difficult, because the state adopted a wide array of policies that may have had a beneficial effect. It is possible, however, to rule out some potential candidates.

For example, some have noted the state’s participation in the federal Reading First program, in which public schools received grant money to implement instructional and assessment tools. Florida also supplemented the Reading First grants with its own financing of reading coaches for schools across the state. The data clearly show, however, that any additional test-score gains made by schools that participated in Reading First or had reading coaches were far too small to explain the substantial improvements observed on both the NAEP and the FCAT.

Others have found it tempting to argue that the state’s constitutional amendments to reduce class size and provide universal pre-kindergarten services—both of which could have a sustained positive effect on young kids—are the most likely driver of the gains. Perhaps those reforms will prove effective. The 3rd-grade class of 2003, for which the large gains begin, however, was subject to neither policy.

Current research findings for the accountability and choice reforms adopted by Florida during this time period also appear insufficient to explain such large test-score improvements. Florida assigned letter grades—A, B, C, D, and F—to schools based on their performance on the FCAT. It put into place a school voucher program for students who were attending schools that received the grade of F twice in a row. A tax credit provided scholarships for low-income students. Studies of all these programs have shown that each had a positive effect. And studies have also shown that the retention policy has a positive impact on the performance of students who were retained. Though each of these policies has been tied to student test-score improvements, either the effect size was too small or the policy affected too few students to alone account for the substantial test-score improvements seen on the NAEP and FCAT.

Conclusion

The evidence presented here shows that Florida’s elementary-school students did in fact make large improvements in reading proficiency in the 2000s. As critics contend, the state’s aggregate test-score improvements on the 4th-grade FCAT reading exam—and likely on the NAEP exam as well—are inflated by the change in the number of students who were retained in 3rd grade in accordance with the state’s new test-based promotion policy. Large test-score improvements are also observed, however, among students whose scores were not influenced by changes in the sample selected.

Though somewhat smaller than what is apparent on the NAEP test, the portion of Florida’s reading test-score improvements during this time period that cannot be attributed to changes in the sample of students tested due to the retention policy is nonetheless substantial. Identifying the causes of these improvements remains an important task for future research.

Marcus A. Winters is senior fellow at the Manhattan Institute’s Center for State and Local Leadership and assistant professor at the University of Colorado–Colorado Springs.

Two authors of that report, Paul Peterson and Eric Hanushek, discuss their findings in this video. “Doing well on these tests is not a matter we should be indifferent to,” explains Hanushek. If American students were making gains as large as those made by students in Germany, he notes, our country would experience much greater GDP growth over the next decades.

-Education Next

There is some evidence that this kind of race to the bottom has been occurring.  Rick Hess and Paul Peterson, for example, have compared state cut scores for proficiency on their state tests to results on the U.S. Department of Education’s National Assessment of Educational Progress (NAEP) to show that the level of achievement required to be declared proficient in many states has been dropping over the last decade. In his recent review of the Maranto and McShane book on Obama’s education policies, Nathan Glazer described how advocates for national standards see them as a fix for this race to the bottom:

in Race to the Top, “the Obama administration tacitly gave its approval to a set of ‘Common Core Standards’ developed by a consortium of state school officers and tied Race to the Top dollars to participation in the program.” This may be a path to finally getting a set of national standards and overriding the standards the states set, which have in many states been pushed lower. This “race to the bottom” has made it easier to show adequate yearly progress (AYP) and avoid triggering measures required for schools that do not show AYP.

So does competition among states and localities really produce a race to the bottom or does competition motivate improvement and spark continual improvements?  The answer depends on what states and localities are competing for.  If states and localities are competing to receive federal funds and/or avoid federal sanctions, as Glazer describes states seeking to make AYP, then competition will produce a race to the bottom.  In competing for bureaucratic approval from the feds, states only have to appear good (satisfy the bureaucratic requirements), but they don’t have to actually be good.  Competing for the bureaucratic approval of the federal government turns education into a redistributive policy where the goal is to get a larger share of the federal largess.

But if states and localities are competing for residents and businesses to increase their tax base, then the incentive from competition is to increase standards and quality.  Millions of individuals are not so easily fooled and can distinguish between phony claims of progress created by lowering the bar and real progress.  Clever bureaucrats can also tell the difference but they are bound by the rules for dispersing rewards and sanctions and so are forced into encouraging a race to the bottom.  Individual face no similar constraints.  They want to move to the areas with the best schools to help their kids, enhance their property values, and have access to a quality labor force.  Individuals may make mistakes or have bad taste, but in aggregate they reward real educational progress not fake, race to the bottom, manipulation.

The history of U.S. education is filled with evidence of how this competition for residents and tax base has spurred improvements in quality and increases in rigor.  The economic historian, William Fischel, carefully documents how the development and spread of high school education in the United States was driven by localities seeking to compete for residents demanding a more rigorous education.  And the standards required for graduating high school have steadily increased over time.  Graduation requires more college-prep coursework.  In almost half of the states students now have to pass a state test to receive a standard diploma.  And 37 states instituted their own testing and accountability systems before NCLB was adopted.  The result of these state and local efforts was not always a rigorous education, but they clearly show a trend toward higher standards and quality in response to consumer demand.  Competition produces a race to the top as long as it is competition for individual taxpayers and business instead of competition for federal government handouts.

So, if a race to the bottom is fueled by the desire to satisfy federal bureaucratic rules, why would we think the solution is in the adoption of more federal bureaucratic rules?  National standards will just create a new regime of gaming, manipulation, and the appearance of progress without the actuality of it.  Expanding choice and competition for individuals is the solution to a race to the bottom, not more centralized control that stifles that competition.

-Jay Greene

The U.S. is in the throes of developing new standards and new tests of student performance, actions that reflect a general dissatisfaction with the level of student achievement.  Much of this movement is hooked to a focus on better preparing students for college and work – a focus partly emanating from the consensus opinion that we must improve our human capital if we are to be internationally competitive.

But the available (and prospective) information on student performance is extraordinarily hard to interpret.  The states can get some information about student performance through the NAEP tests, and local districts can find out how they are doing relative to other districts in the state through state accountability measures.  And, while states or localities that perform relatively low on their specific tests know that they could improve, higher performing districts or higher performing states often develop a complacency that has serious side effects.

This general complacency about performance is quite unwarranted when information from international competitors is added.  Students from Massachusetts, our best performing state, place 17^th in international league tables for advanced math skills.  And, by Jay Greene’s calculations, even the best districts tend to lag the international competition.  Yet, none of this has made it far into the public consciousness or into the policy makers focus.

Consider now giving a sample of students in a district or in a state the PISA examination – the international examination given throughout the developed world.  The primary advantage of this exercise is comparing student performance to “what is possible.”  As such, it personalizes and makes more understandable the educational challenges we face.  And, it likely helps in developing a broader constituency for improvement.

Most educational standard setting, performance assessment, and judgments about appropriate levels of achievement today are based on history and custom with a little bit of “professional dreaming.”  The process generally lacks any context of what our international competitors are doing.

Why do local benchmarking?

The overall performance challenges to U.S. students can of course be seen from the national statistics for PISA or TIMSS, the regular international assessments.  But, states and localities, with their own tests and outcomes, can simultaneously retain their own views of performance that is not anchored in the reality of other locales, particularly the international competition.   Ultimately, it is easy to discount or ignore any deficits.

The local testing makes it clear what can be done and what needs to be done in order to compete with other nations.  It is an unambiguous statement of the level of performance.  The state and local assessments can also provide localities with some idea of how improving their labor force might affect future economic growth.  The national implications of improved schooling are both clear and large.  While local areas must worry more about the migration of skilled labor to other areas, the long run impact of improved schools cannot be ignored.  And, as such better information can foster and reinforce pressure for improvement from local and state groups that see the need to reform the schools.

From a national perspective it is likewise very important that states and districts understand this challenge of international competition – because of the primacy of states and districts in education policy and implementation.  Improvement will come from the actions of the states, and these actions are not readily dictated from the national level.

What benchmarking does not do?

It would be wrong, however, to use the international tests to design a local curriculum.  Test design is often linked to a starting point of clear standards, and curricula define how learning and instruction can achieve the goals imbedded in the standards.  The opposite is not the case.  One would not want to take the international assessments – which are attempts to develop tests that are not dependent on specific national curricula – and then try to reverse engineer standards or curricula.

The presumption of the international assessments is that achieving high performance on the local and national standards will also produce high scores on PISA or TIMSS.   The international tests give a direct measure of what high performance means.

What’s at stake for the nation?

While approaching the level of a cliché, it is simply the case that competition is not just with others in the same city or same state with the internationalization of the world economy.  International trade moves the location of economic activity to the place that has a comparative advantage in production.  Historically, the U.S. has dominated the world in high-skill, high-value-added production, in large part because its workers have had more human capital when compared to workers in other countries.

This dominance in worker skills appears to be ending, as evidenced by performance on PISA exams that is below the average for developed countries.  This skill gap forces us to rely on other national advantages if we are to continue as the world economic leader.  We do have the best economic institutions – free and open labor and capital markets, secure property rights, and limited governmental intrusion.  We also have strong colleges and universities, and we have been open to accepting immigrants with high skills.  But, each of these advantages is fading and is unlikely to carry us in the future, as other countries are moving to address these issues themselves.

A future in which the U.S. is no longer the high-skill country and in which other countries increasingly are the innovators does not mean that U.S. workers will be unemployed or that they will do only menial jobs.  It does imply slower economic growth in the U.S. and a resultant stagnation in incomes.  It also implies a noticeable change in the locus of world economic activities and leadership.

-Eric Hanushek

Policymakers and reform advocates alike have rallied around the movement toward a national curriculum, suggesting that this will break the stagnation in achievement of U.S. students.  But there is little evidence that confusion about what we should teach has been a real inhibition to student achievement.  In fact, the existing evidence suggests just the opposite:  there is no relationship between the learning standards of the states and student performance.

To be sure, it is a real problem when students in one state learn very different things than those in other states, and in particular when students from some states lack the skills needed for our modern economy.  We really do have a national labor market, and significant numbers of our population end up living and working in a state different than that where they were born and went to school.  The presumption behind having national standards (whether voluntary or coerced) is that having a clearer and more consistent statement of learning objectives across states would tend to lessen the problem of heterogeneous skills that students bring to the labor market.  Again, however, the fundamental problem is lack of minimal skills and not the heterogeneity of skills per se.

Experience provides little support for the argument that just more clearly declaring what we want children to learn will have much impact.   In arguing for focusing on standards, proponents of national standards conventionally point to Massachusetts:  strong standards and top results.  But it is useful to expand thinking from just Massachusetts to include California, a second state noted for its high learning standards.  Indeed, some have argued that both states would have to lower their standards in order to fit into the structure of the Common Core.  But California balances Massachusetts:  strong standards and bottom results.

In order to see the issue more broadly, it is possible to compare state-by-state measures of learning standards to student outcomes.  There are different independent ratings of the quality of the learning standards currently existing for each state, and these can be combined with assessments of student performance from the National Assessment of Educational Progress (NAEP).  The most comprehensive rating of state standards is probably that of Education Week.   Education Week developed a comprehensive grading across grade-specific standards, testing, and the accountability that goes with them in each state.  This ranking provides aggregate grades for each state.  (Another widely acknowledged rating of state standards by subject is produced by the Fordham Institute.  These competing rankings are correlated with those of Education Week, though not perfectly, and it really makes no difference for the analysis which we use.)

The figure below shows how the ranking of standards compares to NAEP scores – here the 8^th grade math scores.  (The specific NAEP assessment for grade and subject has no influence on the overall conclusions).

As can be seen, the better the state standards the worse the students tend to do.  But, of course, this does not imply that we should move toward weaker standards.  The real conclusion is that state standards have little to do with student performance.

In other words, what really matters is what is actually taught in the classroom.  Simply setting a different goal – even if backed by intensive professional development, new textbooks, and the like – has not historically had much influence as we look across state outcomes.

There are a number of refinements that one can think about for this analysis, but they do not change the answer.  This conclusion holds even under more sophisticated analysis, as demonstrated quite conclusively by Tom Loveless of the Brookings Institution.  Indeed his analysis helps to frame the entire debate.

The continuing emphasis on Common Core standards, including the debates about the legality of them, is often interpreted as indicating that the Common Core is a really big deal in school reform.  The data suggest otherwise.

The one possible complementary gain from the move to national standards is that the assessments of performance might become better.  It is widely recognized that the current tests used to judge outcomes within individual states tend to be quite weak.  (This concern about tests is not leveled at NAEP, which was used in the comparisons above, but instead applies to the tests states use for accountability purposes).  If the new standards lead to better tests – something that might come out of the two testing consortia funded by the U.S. Education Department – we might have the basis for improved school policies.  But that is also not certain and cannot be used as a primary justification for the focus on Common Core standards.

One interpretation of the emphasis on developing the Common Core curriculum is that these debates provide a convenient distraction from potentially more intractable fights over bigger reform ideas like teacher evaluations, expanded school choice, or improved accountability systems.    While I am not against having better learning standards, I believe that we cannot be distracted from more fundamental reform of our schools.  The future economic well-being of the U.S. is dependent on improving the achievement and skills of today’s students.

-Eric Hanushek

Then came the meteoric shock of consequential accountability, and student test scores (on the National Assessment of Educational Progress and state exams, too) started to take off. For some subgroups of students, math and reading skills improved by two or three grade levels since just the mid 1990s.

Yet we all know the downsides of the narrow focus on reading and math scores in grades three through eight and once in high school. This regimen puts enormous pressure on schools to ignore or exclude other important subjects (art, music, history, even science). It penalizes schools with an educational strategy that succeeds in the long term but doesn’t produce sky-high scores now. (I’m thinking of Waldorf schools, for instance, such as the preschool my son attends.) And it undervalues other important contributions that schools make, such as to students’ character development and social skills.

When it comes to evaluating teachers, there’s wide agreement that we need to look at student achievement results—but not exclusively. Teaching is a very human act; evaluating good teaching takes human judgment—and the teacher’s role in the school’s life, and her students’ lives, goes beyond measurable academic gains. Thus the interest in regular observations by principals and/or master teachers. These folks can pick up on nuances missed by the value-added data—plus can provide actionable feedback to instructors so that they can improve their craft. (Harrison School District Two in Colorado has one of the best plans in this regard.)

So why do we assume, when it comes to evaluating schools, that we must look at numbers alone? Sure, there have been calls to build additional indicators, beyond test scores, into school grading systems. These might include graduation rates, student or teacher attendance rates, results from student surveys, AP course-taking or exam-passing rates, etc. Our own recent paper on model state accountability systems offers quite a few ideas along these lines. This is all well and good.

But it’s not enough. It still assumes that we can take discrete bits of data and spit out a credible assessment of organizations as complex as schools. That’s not the way it works in businesses, famous for their “bottom lines.” Fund managers don’t just look at the profit and loss statements for the companies in which they invest. They send analysts to go visit with the team, hear about their strategy, kick the tires, talk to insiders, find out what’s really going on. Their assessment starts with the numbers, but it doesn’t end there.

So it should be with school accountability systems. The best ones today take various data points and turn them into user-friendly letter grades, easily understandable by educators, parents, and taxpayers alike. So far so good. Why not add a human component to the process, via school inspectors like those in England? (See this excellent Education Sector paper, by my friend Craig Jerald, for background on how that works.)

Imagine: At least once a year (more would be better) a group of inspectors visits a school. (These would be professionals on contract with the state department of education—typically retired teachers and principals. In the case of charter schools, authorizers would be involved, too.) They would mostly look for two things:

• Evidence that the school is achieving important outcomes that may not be captured by the state accountability system. For example, the school’s administrators might show them test score data from a computer adaptive exam like NWEA’s that demonstrates progress for individual kids (especially those well above or below grade level) that isn’t picked up by the less-sensitive state test. Or perhaps a high school has compelling data about its graduates’ college matriculation and graduation rates that put its mediocre test scores in a different light.
• Indications that the school’s culture and instructional program are inculcating valuable attributes in their students. This is to guard against the “testing factory” phenomenon. Is the school offering a well-balanced curriculum (and extra-curriculars), or engaging in test-prep for weeks on end? Is it focused on teaching “non-cognitive” skills and attributes, such leadership, perseverance, and teamwork? Character traits like empathy, honesty, and courage?

The school visits should not be exercises in excuse-making. This isn’t about lowering expectations because of difficulties particular communities face, or delaying needed changes because the school’s educators appear to be “trying hard.” Rather, it’s a chance to round out the picture generated by the state’s (inevitably) incomplete accountability report.

So here’s how it would work: The state would develop school grades based on a variety of indicators, as it does now. Then those grades could be raised or lowered based on the findings of the school inspectors. (Generally just a letter-grade, but sometimes more.) Grades would go up because of evidence of strong outcomes not captured by the state accountability system; grades would go down because of evidence of unhealthy curricular narrowing.

Such a system would remain imperfect. Human judgment would introduce subjectivity and error into the process. Inspectors might face pressure (maybe even bribes) to raise schools’ grades. And it would be expensive—at least as compared to the testing-and-accountability systems we have now. These issues would need to be addressed.

Still, it’s worth it. To the extent that school grades (and consequences linked to them) drive policy and behavior, we ought to make sure that those grades are informed by more than just numbers. The correct response to the unintended consequences of accountability isn’t to end accountability, but to make it work better. That could have positive consequences for many years to come.

-Michael Petrilli

This post originally appeared on the Fordham Institute’s Flypaper blog.

• Its content-and-skill expectations for grades K-12 in English and math are, by almost everyone’s reckoning, about as rigorous as the best state-specific academic standards and superior to most.
• It was developed outside the federal government, voluntarily by states, using private dollars. (The related assessments are another matter.) And both standards and assessments remain voluntary for states.
• It opens the way, for the first time, to comparing student, school and district performance across the land on a credible, common metric—and gauging their achievement against that of youngsters in other countries on our shrinking and ever-more-competitive planet.
• Besides comparability, it brings the possibility that families moving around our highly mobile society will be able to enroll their kids seamlessly in schools that are teaching the same things at the same grade levels.

Ever since it landed, however, the Common Core has been the object of ceaseless attacks from multiple directions. The number of zealous assailants is small and, for a time, it all looked like a tempest in a highly visible teapot. That may yet turn out to be the case. But the attacks are growing fiercer; some recent recruits to the attack squad are people who tend to get taken seriously; and anything can happen in an election year. Remember the classic Peter Sellers movie, The Mouse That Roared? The Duchy of Grand Fenwick ended up triumphing over the United States of America. As you may recall, that happened in large part because the U.S. government contributed to its own defeat. In the present case, something similar could well transpire. Please read on.

Before examining the assaults, however, let’s remind ourselves what the Common Core is not. It is no guarantee of stronger student achievement or school performance. Huge challenges await any (serious) academic standards on the implementation, assessment and accountability fronts. To get traction in classrooms, states that adopt these standards (and all but four say they’re doing so) must take pains with curriculum, teacher preparation, assessment, accountability and more. To yield real rigor (and comparability), the currently-under-development assessments must avoid numerous pitfalls and incorporate hard-to-achieve consensus on genuinely challenging issues (such as where to set the “cut score”).

In and of themselves, academic standards merely describe the end point to be reached and the major stops en route. They don’t get you there. But it’s far better to have an education destination worth reaching, i.e. rigorous standards set forth with sufficient specificity, clarity, and rich content to provide real guidance to curriculum designers, classroom teachers, test developers and more. Few states have managed to do that on their own.

To be sure, other states could simply copy the best of those that already exist. But that’s more or less what the Common Core is: an amalgam of good standards put together by people who know a lot—and care a lot—about both content and skills.

So why the nonstop attacks against it? As best I can tell, they arise from six objections and fears.

First, a few earnest critics are convinced that the standards are substantively flawed, that the algebra sequence (or grade level) is wrong, the English standards don’t contain enough literature, the emphasis on “math facts” isn’t as strong as it should be, etc. This sort of thing has accompanied every past set of standards of every sort, and it’s perfectly legitimate. Insofar as such criticisms are warranted, the Common Core can be revised, states can add standards of their own, and jurisdictions that find the common version truly unsatisfactory can change their minds about using it at all.

Second, the Common Core will be difficult and expensive to implement. Many organizations are working hard to help states surmount these genuine challenges. Many philanthropists are kicking money into the effort. And some groups (Fordham included) are trying to cost it all out. Nobody denies that doing this right will be hard and costly (though some of those costs are already embedded in state and district budgets.) Of course, those who think the country is doing OK today have every reason to shirk that challenge and stick with what they’re used to.

Third, the Common Core won’t make any difference in student achievement—but may cause a politically-unacceptable level of student failure. As noted above, standards per se do not boost achievement. (Of course, standards per se don’t carry costs or failure rates, either. They don’t, by themselves, do much of anything!) And failure rates will worsen only if (a) the new assessments are truly rigorous and (b) schools neglect preparing their pupils to pass them.

Fourth, states have done as well, or better, on their own, and switching over to the Common Core will just mess them up. This criticism mostly emanates from Massachusetts, which has done a commendable job on its own and where the decision to adopt the Common Core was truly conflicted. Other states that prefer to go it alone, mostly notably Texas and Virginia, have simply declined to adopt the Common Core. Others are free to exit from it (though doing so would, for some, violate commitments they made in their Race to the Top proposals.)

Fifth, “national” is not the right way to do anything in American education. We retain a deep (if, in my view, unwarranted) affection for “local control” in this realm and constitutional responsibility for education is undeniably vested in the states. Some folks dread the prospect of a “national curriculum.” (Some simply mistrust the Gates Foundation, which has bankrolled much of this work.) Others are incapable (perhaps willfully so) of seeing any distinction between “national” and “federal”, though we seem to have no difficulty making that distinction elsewhere in education. (E.g. National Governors Association, S.A.T., A.P., ACT.)

Sixth, and closely related to the blurring of national with federal is the expectation that Uncle Sam won’t be able to keep his hands off the Common Core—which means the whole enterprise will be politicized, corrupted and turned from national/voluntary into federal/coercive. This is probably the strongest objection to the Common Core and, alas, it’s probably the most valid, thanks in large measure to our over-zealous Education Secretary and the President he serves.

Let’s face it. Three major actions by the Obama administration have tended to envelop the Common Core in a cozy federal embrace, as have some ill-advised (but probably intentional) remarks by Messrs. Duncan and Obama that imply greater coziness to follow.

There was the fiscal “incentive” in Race to the Top for states to adopt the Common Core as evidence of their seriousness about raising academic standards.

Then there’s today’s “incentive,” built into the NCLB waiver process, for states to adopt the Common Core as exactly the same sort of evidence.

(In both cases, strictly speaking, states could supply other evidence. But there’s a lot of winking going on.)

The third federal entanglement was the Education Department’s grants to two consortia of states to develop new Common Core-aligned assessments, which came with various requirements and strings set by Secretary Duncan’s team.

This trifecta of actual events is problematic in its own right, not because the federal government is evil but because Washington has become so partisan and politicized and because of angst and suspicion that linger from failed efforts during the 1990’s to generate national standards and tests via federal action.

What’s truly energized the Common Core’s enemies, however, has been a series of ex cathedra comments by President Obama and Secretary Duncan. Most recently, the Education Secretary excoriated South Carolina for even contemplating a withdrawal from the Common Core. Previously, the President indicated that state eligibility for Title I dollars, post-ESEA reauthorization, would hinge on adoption of the Common Core. Talking with the governors about NCLB waivers earlier this week, he stated that “if you’re willing to set, higher, more honest standards then we will give you more flexibility to meet those standards.” I don’t know whether he winked. But everybody knew what standards he was talking about.

It will, of course, be ironic as well as unfortunate if the Common Core ends up in the dustbin of history as a result of actions and comments by its supporters. But in March 2012 there can be little doubt that the strongest weapons in the arsenal of its enemies are those that they have supplied.

-Chester E. Finn, Jr.

This post originally appeared on the Fordham Institute’s Flypaper blog.

He has more to say about a larger strategy for boosting teacher quality in “An Effective Teacher in Every Classroom,” which appeared in the Summer 2010 issue of Ed Next.

He also authored “Valuing Teachers: How Much is a Good Teacher Worth?” which appeared in the Summer 2011 issue of Ed Next.

Rather, I think the reluctance to contribute [to a debate in support of Common Core] is due to hubris, impatience to focus on implementation, political naivete, and disdain for what they see as mean-spirited carping….

There are long rows of argument and persuasion still to be hoed. And, if you’re eager to overhaul what gets taught in forty-odd states serving forty million or more students, that’s probably as it should be. If Common Core-ites don’t have the patience or stomach for that task, they should let us know now–and save everyone a whole lot of grief.

The notion that Common Core proponents needn’t make their case is an affront to democratic values.

Well, Ed Next managed to find someone to argue for and against the quality of Common Core standards, producing a really excellent and illuminating exchange.  W. Stephen Wilson took the pro side and Ze’ev Wurman was on the con side.  I would encourage you to read the entire debate yourself, but here is my takeaway:  They were mostly in agreement about the quality of Common Core.  Both seemed to agree that Common Core was better than the standards previously in place in most states but worse than in a non-trivial number of other states.  They also agreed that Common Core standards are significantly weaker than the ones in most high-achieving countries.

So if they agree that Common Core is sort of mediocre, why does Wilson support them while Wurman oppose them?  Wilson sees the improvement on the standards of 30 or more states to be substantial progress.  He sees this as a first step toward developing stronger national standards that would be comparable to those of our overseas competitors and better than all previously existing state standards.

Wurman sees Common Core as significantly lowering the bar relative to several previously existing state standards, including very large states like California.  More importantly, he sees Common Core as the end of progress in improving standards rather than the beginning.  Once put in place, he sees no incentive for anyone to toughen national standards since no state will be competing to offer a more rigorous education in order to attract residents and businesses.  He also sees national standards as more easily captured and dummied-down by teachers unions and other entrenched interests who would prefer to have their members (and students) jump over a lower bar.

-Jay P. Greene

UPDATE — Stephen Wilson contacted me over at the Jay P. Greene blog to object to the description of his views as supporting the adoption of Common Core. He thinks Common Core math standards are much better than those that previously existed in 30 states but still lagging those in other states and high achieving countries. And he generally has no opinion on whether universal adoption of Common Core would represent progress or not or is desirable or not.

It appears that I was wrong. The Ed Next forum was more a discussion among critics than a debate between a supporter and opponent.

So we are back to Rick’s original complaint. We still don’t have anyone who was willing to debate in favor of the national adoption of Common Core based on the quality of the standards.

It’s pretty pathetic that supporters of Common Core couldn’t produce anyone to take the “pro” side of this debate. And it’s even more pathetic that supporters are determined to cram Common Core down our throats without feeling the need to intellectually defend it.

Call this the test score hypothesis. It explains reformers’ enthusiasm for test-based accountability; for “college and career-ready standards”; for teacher evaluations based, in significant part, on student outcomes; for “data-based instruction”; and for much of the rest of the modern-day reform agenda. After all, if reading, math, and science knowledge and skills are so directly linked to the life chances of individual kids, and of the livelihood of the country as a whole, why not get the education system focused like a laser on them?

But is this hypothesis correct? Is stronger academic performance related to better life outcomes for kids and better economic outcomes for nations?

In a word: yes. As Kevin Carey noted recently, the big Chetty et al study didn’t just demonstrate the importance of teacher effectiveness. It also offered strong support for the Test Score Hypothesis.

If you believe standardized tests are worthless or highly flawed or deeply inadequate or even troublingly limited in accuracy and scope–and many reasonable people believe these things–then you could dismiss or downplay value-added measures of teacher effectiveness, by definition….But now the CFR study says that teachers who are unusually good at helping students score high on standardized tests today aren’t just unusually good at helping students score high on standardized tests tomorrow. They also have an unusual effect on the likelihood of students going to college, going to a good college, earning a good living, living in a nice place, and saving for retirement. In other words, whatever the limitations of standardized tests may be, test-based value-added scores do, in fact, provide valuable information about the things most people care most about.

Then there’s the international evidence. As Eric Hanushek has been arguing vociferously for years, there’s a direct link between academic achievement (as measured by math and science tests) and a country’s economic growth.

The level of cognitive skills of a nation’s students has a large effect on its subsequent economic growth rate. Increasing the average number of years of schooling attained by the labor force boosts the economy only when increased levels of school attainment also boost cognitive skills. In other words, it is not enough simply to spend more time in school; something has to be learned there.

Hanushek further argues that the only way to solve our country’s long term fiscal challenge is to grow our way out of it. If we could indeed boost the cognitive skills of our students, even by a little, our structural deficit would go away.

So student achievement matters a lot. But does it matter the most? It’s hard to make the case anymore that test scores are irrelevant. But what remains unknown is whether reading, math, and science are the most important things that schools could be teaching. As Dana Goldstein noted back in December,

I’ve been struck again and again by the newness of the idea that schooling is primarily a matter of academic achievement…. It is only really since “A Nation at Risk” that we’ve had a national dialogue about academic excellence for every child. This is a much-needed development in American culture, but its discontents are numerous: A lack of attention paid to the civic, social, and artistic benefits of schooling, and the ways in which children are (ideally) shaped as moral, cultured, socially-responsible people by their teachers and school communities. 

We might all want schools to walk and chew gum at the same time—to boost “academic achievement” while also developing “moral, cultured, socially-responsible people.” But our policies—especially school-level accountability and test-based teacher evaluations—focus on academic achievement alone.

The nagging question then—the “known unknown”—is whether other stuff matters more—both to kids’ life chances and to the country’s economic success. What if, for instance, “social and emotional intelligence”—knowing how to relate to others—is more important than many reformers have been willing to acknowledge? What if these interpersonal skills are what help lift poor kids out of poverty and enable economies to succeed? Or other “soft skills” and attributes like grit, perseverance, industriousness, the ability to delay gratification, and so forth?

In that case, is it smart to push Head Start centers to focus overwhelmingly on pre-literacy and pre-numeracy skills (as many of us have)? Is it wise to cut time for recess, to trim extracurriculars, or to push for the maximum amount of homework, to be completed by solitary would-be scholars? Does it make sense to ask teachers to obsess about student achievement over everything else?

The private school sector, which many reformers admire, is not so conflicted. Every high-end school boasts about its commitment to the “whole child,” to kids’ intellectual, emotional, social, and physical development. These schools would never consider their graduates to be well-educated without an appreciation for the arts, participation in sports, a commitment to community service, and the development of strong character. And judging by the admissions policies of the nation’s great universities, our elite higher education institutions hold this holistic view, too. Are these non-academic attributes just “extras”—luxuries that schools serving poor or working class kids just can’t afford? Or are they as essential as academics, for everyone?

Reading, math, and science matter a lot, but they are almost certainly not enough. That is why we must tread carefully when designing next-generation school accountability and teacher evaluation systems. If we accidentally create incentives for schools and teachers to focus solely on academic achievement and ignore the rest, we could be making our children and our nation less competitive, not more so. Let us proceed with care.

-Mike Petrilli

This blog entry originally appeared on the Fordham Institute’s Flypaper blog.

• It worked!

As Mark Schneider shows in his recent paper for Fordham—and as Eric Hanushek and others demonstrated before him—poor, minority, and low-achieving students made huge progress in math, and sizable progress in reading, during the late 1990s and early 2000s. Their most recent scores on the National Assessment of Educational Progress indicate all-time highs for most grades and subjects. These students are typically performing two grade levels ahead of where their peers were fifteen years ago in math, and are reading at least one grade level higher. So how to explain these historic gains? While we can’t draw causal conclusions from NAEP, we can make educated guesses. What’s clear is that states that adopted “consequential accountability” in the nineties saw big test-score jumps, and the late-adopter states saw similar progress once No Child Left Behind kicked into action. So, while other factors could have been in play, too (such as efforts to reduce class size or the cessation of the crack-cocaine epidemic), there’s a pretty good case that testing and accountability succeeded in spurring higher student achievement, at least at the bottom of the performance spectrum.

• But it couldn’t work forever. As Schneider argues, the test-score gains sparked by NCLB-style accountability appear to have hit a plateau. We’re back to anemic progress in most grades and subjects, particularly in the states (like Texas) that embraced testing and accountability first. That shouldn’t be too surprising. While the initial pressure (and shame) provided by consequential accountability appears to have changed behavior at the district and school level, after a while being called a “failing school” loses its sting. Furthermore, holding “schools” accountable has rarely equaled holding individuals accountable—real-live teachers and principals who might lose their jobs. Once it became clear that NCLB was all bark and no bite, schools could return to the status quo ante.
• The trade-offs are real. The good news is that we’ve seen enormous progress for our lowest-achieving students. The bad news is that we’ve seen languid progress for our highest achievers. The good news is that math scores are way up and, to a lesser degree, reading scores are up, too (especially for poor and minority kids). The bad news is that history and science have been squeezed out of the elementary school curriculum, particularly in high-poverty schools. Whether these trade-offs were worth it depends on your point of view. Personally, I’d prefer a policy that aims for more balance: achievement gains across the performance spectrum, not just at the bottom; and a more holistic view of what it means for students to be well educated. Literacy and numeracy are (obviously) not enough.
• Pet ideas from both parties crashed and burned. The Democrats gave the country the “white elephant” gift of the “highly qualified teachers” mandate, a policy that succeeded in turning the nation’s teachers against NCLB from the very beginning; managed to tie up myriad schools (including charters) in all manner of red tape; and gravely threatened Teach For America, one of the most promising reforms of the NCLB era. From the Republicans we got “supplemental educational services,” a.k.a. free tutoring. This was more of an impulse than a fleshed-out idea. It was never clear whether SES was meant to be a sanction for failing districts (if you don’t improve your test scores, we’ll take some of your Title I money away from you); a serious effort at parental choice; or a way to “extend” learning time for needy kids. Regardless, its entire design was predicated on cooperation from school districts, which were responsible for facilitating the flow of funds away from their coffers and into the hands of nonprofit and for-profit providers. As my Italian grandmother would have said, “Fatta chance.
• It’s time for something new. On this point, virtually everybody agrees. But what should the next phase of education reform entail? The contours are now taking shape. First, there’s agreement that, for accountability to be real, it has to be placed upon real-live people, not just amorphous “schools.” That means, first and foremost, holding teachers accountable for their performance. Thus the interest in: more sophisticated teacher-evaluation systems, tenure reform, performance pay, and all the rest. Second, there’s broad consensus that we need to balance the “tough love” approach of accountability with the “helping hand” of capacity-building: Providing teachers with tools like a coherent curriculum—linked to the new Common Core standards—so they don’t have to make it all up on their own. And third, we can all glimpse the promise of digital learning, if technology can be harnessed effectively and if the political and governance roadblocks can be removed. But what’s the appropriate (and politically feasible) federal role in all of this? In all of these reforms, Uncle Sam’s involvement will be—and should be—minimal. The political thirst for aggressive federal involvement in education has been quenched, and the dollars to fund it spent. Plus these “next wave” reforms require nuance, care, and thoughtfulness to get them right—attributes not associated with Uncle Sam. In other words, reform will continue, but the federal government will lead from behind. As well it should.

Happy birthday, No Child Left Behind. And here’s hoping that you don’t make it to eleven.

-Michael Petrilli

This post also appeared on Flypaper.

Many educators and elected officials, including more than a few members of Congress, regard “No Child Left Behind,” the well-known moniker of George W. Bush’s 2001 education act, as a discredited “brand.” Indeed, the very acronym NCLB is about to be tossed into the dustbin of history in favor of its progenitor, ESEA (the Elementary and Secondary Education Act), or perhaps some new title yet to be devised on Capitol Hill. There are many reasons why NCLB has been discredited, including, to quote Kevin Carey, the “apocalyptic language out there, that standards and tests have ruined American public education, driven the best teachers out of the classroom, etc., etc.”

Yet, as the data presented below demonstrate, NCLB—and the accountability movement it embodied, codified, and symbolized—contributed to a major change in the performance level of American students in math. The data also suggest, however, that the accountability movement has likely reached a point of diminishing (or perhaps even no) returns. While moving on from NCLB is probably essential to produce further growth in student performance, “consequential accountability” was an important and meaningful education reform and ought not be dismissed as a failed initiative.

Debates over the effects and effectiveness of NCLB almost always revolve around national and state scores on the National Assessment of Educational Progress (NAEP). Not surprisingly, the release in November 2011 of the newest NAEP Mathematics and Reading Report Cards set off a new round of discussion about the impact of NCLB and accountability more generally. Given the ongoing fights surrounding the overdue reauthorization of ESEA/NCLB, the debate over the effects of accountability is more important now than ever.

Remember that NCLB’s system of consequential accountability (in which schools face cascading penalties for failure, e.g., replacement of the school’s principal, reconstitution, closure, etc.) was built upon the experience of many states that had already developed such systems before 2001. There is considerable agreement that states adopting consequential accountability before NCLB experienced more rapid growth in their test scores relative to non-adopting states. However, as Hanushek and Raymond note, as NCLB took hold, all states became “effectively consequential accountability states.” Perhaps not surprisingly, after NCLB, states that were new to the accountability regime experienced faster growth on NAEP assessments than states that had introduced their own accountability regimes before 2001.

The Case of Texas

Texas was one of the first states in the nation to adopt strict and consequential accountability. The Texas experience was fundamental to the framing of NCLB, as George W. Bush took the lessons and practices of Texas along with him when he moved from Austin to Washington. Thus, looking at the growth in NAEP scores in Texas relative to changes in the nation as a whole allows us to tease out some lessons about the effects of accountability on student performance and to speculate about the effectiveness of accountability past, present, and future.

As we look at these data, we should remember that, while NAEP is rightfully viewed as the “gold standard” of assessments, it is not the ideal instrument for detailed statements of cause and effect. We should further keep in mind one of the prime maxims of statistics: Correlation is not causation.

The Remarkable Growth in NAEP Math Scores

It is well known that, as measured by NAEP, American students have improved substantially in math (more in fourth grade than in eighth) and little in reading over the last two decades. Separate and apart from overall averages, there has been continuing concern for the level of skills among racial/ethnic minorities as well as concern for the effects of accountability on low- versus high-performing students (specifically, whether or not NCLB placed so much attention on low-performing students that high-performing students were neglected and suffered as a result). Looking at trends in Texas versus the nation presents some insights into these issues.

Fourth-Grade Mathematics

Consider Figure 1, which graphs the average scale scores on NAEP’s math assessment for fourth-grade students in Texas and in the United States as a whole. The growth in the performance of these students is nothing short of remarkable. Using the very rough rule of thumb that a 10-point change in NAEP scores equals about one year of learning, in 2011 our fourth graders are about two years ahead of where they were in 1992. But, as the figure shows, Texas and the nation marked their peaks of achievement at two distinct points in time.

In 1992, students in Texas were performing at the same level as the students in the nation. In the 1993-94 school year, Texas introduced its system of consequential accountability and, by the time of the next NAEP assessment in 1996, Texas fourth-graders had surpassed their peers nationwide. Between 1992 and 2000, math scores across the nation began to creep up; during the same period, a growing number of states began to adopt accountability systems.

By 2003, NCLB had turned every state into a consequential accountability state, and the rate of increase nationwide in math scores between 2000 and 2007 was remarkable. While Texas students continued to outperform the nation as a whole through 2007, the sharp uptick in national performance after 2000 narrowed the Texas lead substantially. Indeed, the last two assessments, in 2009 and 2011, show no significant difference between fourth graders in Texas and fourth graders nationwide.

We return to these overall patterns later, but first we turn to the performance of three groups of students who served as particular focal points of NCLB and the accountability movement more generally: blacks (Figure 2), Hispanics (Figure 3), and low-performing students (Figure 4), defined here by the cut score identifying those students performing at NAEP’s 10th percentile.

At the beginning of the series in 1992, black and Hispanic fourth-grade students in Texas scored slightly higher than their nationwide peers, while those low-performing students at the 10th percentile in Texas achieved at the same level as those at the 10th percentile nationally.

Between 1992 and 2000, the scores of Texas students in all three groups increased faster than those of their peers nationwide, with the size of the gap between student in Texas and the nation widening to well over 10 points for each group. Between 2000 and 2003, nationwide, the gains for students in each group increased dramatically but then slowed substantially. Gains among Texas fourth graders were sustained over a longer period of time, but also show evidence of little growth since 2005, with Hispanic and the lowest-performing students actually scoring lower in the latest assessments than in 2007.

The growth in fourth-grade math achievement represents one of the most significant success stories in contemporary American education. Again, the reader is reminded that, while correlation is not causation, the introduction of consequential accountability in Texas and then across the nation coincided with impressive spikes in the performance of students in fourth-grade math, and in particular among the students of most concern to NCLB and the accountability movement more generally.

Eighth-Grade Mathematics

NAEP test results for eighth-grade math represent a somewhat weaker reflection of this striking pattern (Figure 5). The first NAEP eighth-grade math assessment was in 1990, at which time Texas eighth graders lagged the nation by 5 points. That gap disappeared by 2000. By 2005, as the strong fourth-grade performers moved into the eighth grade and as the Texas system of consequential accountability continued to gain traction, Texas eighth graders moved past their national peers, producing a gap of 6 points. Whether eighth-grade test scores can continue to grow, given the flattening scores at the fourth grade, is something that remains to be seen.

Among black and Hispanic eighth graders, Texas students started at about the same place as their national peers in 1990. Over time, however, they experienced steady growth in performance, producing a widening gap with the nation. Indeed, the size of the gap for black students (in favor of Texas) has increased from 6 or 7 points before 2000 to 10 points in the last three assessments (Figure 6). The size of the gaps in favor of Hispanic students in Texas has been somewhat more variable, and was not statistically significant before 2000 (Figure 7). But this gap has grown to over 10 points in the last three assessments. Similarly, the cut score defining the lowest 10th percentile has risen more rapidly in Texas than in the nation as a whole (Figure 8), becoming statistically significant in 2000 and almost doubling in size from 2000 (7 points) to the latest assessment in 2011 (13 points).

High-Performing Students

A frequent criticism of the accountability movement and NCLB was that the focus on racial and ethnic minorities and on the lowest-performing students led to a neglect of the nation’s highest-performing youngsters.

Here we define high-performing students as those performing at NAEP’s 90th percentile. Fourth-grade math scores for these students both in Texas and in the nation display sharp increases since 1992 (Figure 9). The cut score for the top performers nationwide stood at 259 in 1992 and steadily rose to 276 in 2011, a gain of 17 points. The highest-performing fourth graders in Texas saw a correspondingly large jump in cut scores from 256 in 1992 to 273 in 2005. (Interestingly, half of that gain occurred between the assessments immediately preceding and following implementation of the state’s accountability system in 1993-94). Since 2005, however, there has been no statistically significant change in cut score for those Texas youngsters, although the national cut score for high performers has continued to rise—producing a statistically significant difference (to the disadvantage of Texas) in the two most recent administrations of NAEP.

Eighth-grade math scores among the highest performers also improved substantially over the period, gaining 14 points nationally and 17 points in Texas (Figure 10). The sharpest gains for these high-performing eighth graders in Texas were between 2000 and 2005, building on the improvement made in math by Texas fourth graders four years earlier. Gains continued thereafter at somewhat slower rates, likely reflecting the slower growth in fourth-grade math skills.

The growth in NAEP scores of the highest-performing students in Texas and the nation essentially mirrors the gains made by student groups that were focal to the policy goals of NCLB. Whatever changes more directly focused on specific target populations apparently spilled over to affect the performance of high performers as well. And just as we saw evidence of diminishing effectiveness in recent years for average, minority, and low-performing students, there is evidence that the spillover effects of accountability on high-performing students are also wearing thin. The recent absence of growth in Texas fourth-grade math skills among these high-performing students may portend the end of a remarkable period of growth among the highest performers in the second-largest state in the union.

The Disappointing Case of Reading Scores

The improvements in NAEP math scores were an unquestionable success for America’s fourth and eighth graders and even more so for students in Texas. However, neither the nation as a whole nor Texas has done nearly as well improving students’ reading skills. Figure 11 shows no significant difference between the reading scores of fourth-grade students in Texas and in the nation as a whole, except in 2003, and minimal improvement across the board. And Texas’s eighth graders have significantly lagged the nation since 2003: by 2 points in 2007 and by 4 to 5 points in every other assessment between 2003 and 2011 (Figure 12).

Accountability and NCLB Were a Success, But…

In 1972, Stephen Jay Gould and Niles Eldridge proposed a theory of evolutionary change that emphasized what they termed “punctuated equilibrium.” Their core insight was that complex systems will exist in long periods of stasis. Rather than coming in small incremental steps, change is often characterized by abrupt radical transformations caused by events external to the existing system. Perhaps the most dramatic example is the relatively sudden disappearance of dinosaurs associated with a meteor crashing into the Earth and changing the climate. As a result, the dinosaurs’ long reign was replaced by a new equilibrium dominated by mammals.

In 1993, political scientists Frank Baumgartner and Bryan Jones introduced this theory to the study of public policy, and it has since become a common lens through which to view change in social systems. Baumgartner and Jones argued that policy generally changes only incrementally, until some event, such as change in the party control of government or sizable shifts in public opinion, lead to large policy alterations. In their approach, large changes in external conditions (what Baumgartner and Jones term an “exogenous shock”) are often needed to produce change in complex social and political systems.

The pattern of test scores in Texas and the nation suggest that consequential accountability—adopted early by Texas, then by more states, and finally by the nation as a whole—was a shock to the U.S. school system that altered the ecosystem and led to a different outcome than had existed before. Over a relatively short period, math performance in fourth and eighth grade abruptly shifted to higher levels of performance. For example, between 2000 and 2005—the five years spanning the introduction of accountability via NCLB—the average math scale score nationwide at the fourth grade rose by 12 points, roughly a year of learning. In the same period, the average scale score for black fourth graders rose by 18 points, for Hispanic students by 17 points, and the cut score defining the 10th percentile of performance increased by 16 points. The corresponding changes among eighth-grade math scores are small only in comparison: 6 points nationwide, 11 points for black students, 10 points for Hispanic students, and 8 points for those students at the 10th percentile.

To be sure, an important lingering issue is the absence of growth in reading scores in Texas and in the nation as a whole. Many have argued that the foundation for reading, compared to math, is far more dependent on what happens early in children’s lives—before they enroll in school—and that improving reading skills is therefore much harder to accomplish. Whatever the explanation, clearly the absence of growth reflects a failure of the accountability “meteor” to affect reading levels in a fundamental way.

There is once final pattern to note: As would be expected when viewed through the punctuated- equilibrium lens, once the disruption of consequential accountability has wrung all changes out of the system, a new stasis should take hold. Indeed, Texas, an early adopter, led the nation to higher scores and seems to be ahead of the nation in reaching a new plateau where changes are minimal compared to what came in response to the introduction of an accountability system. The nation, which lagged Texas in adopting accountability, now seems to be entering a period of little change in test scores.

In the 1990s and early 2000s, accountability was an exogenous shock that produced radical gains in math if not in reading. But we now need a new shock to prevent a prolonged period of stasis and stagnation. Scanning the heavens for the next meteor, the most likely candidates to come crashing into the school ecosystem are the Common Core and the better measurement of teacher performance. If the United States is lucky, one or both of these shocks will produce yet another major uptick in math scores. If we are really lucky, these shocks will produce upticks in reading and other subject areas as well.

Mark Schneider, a former commissioner of the National Center for Education Statistics, is a vice president at American Institutes for Research and visiting scholar at the American Enterprise Institute. This article was commissioned and also published by the Fordham Institute.

Checked by Eric A. Hanushek

The No Child Left Behind Act of 2001 (NCLB) was scheduled for reauthorization in 2007, and its future has in recent months garnered renewed attention. Yet so far, Congress has found it impossible to reach sufficient consensus to update the legislation, as competing groups want to a) keep all the essential features of the current law as a way of maintaining the pressure on schools to teach all students, b) modify the federal law by moving to a value-added or some alternative testing and accountability system, or c) eliminate federal testing and accountability requirements altogether, reverting to the days when the compensatory education law was simply a framework for distributing federal funds to school districts. Critics of NCLB’s testing and accountability requirements have a litany of complaints: The tests are inaccurate, schools and teachers should not be responsible for the test performance of unprepared or unmotivated students, the measure of school inadequacy used under NCLB is misleading, the tests narrow the curriculum to what is being tested, and burdens imposed upon teachers and administrators are excessively onerous.

But in all the acrimonious discussion surrounding NCLB, surprisingly little attention has been given to the actual impact of that legislation and other accountability systems on student performance. Now a reputable body, a committee set up by the National Research Council (NRC), the research arm of the National Academy of Sciences, has reached a conclusion on this matter. In its report, Incentives and Test-Based Accountability in Education, the committee says that NCLB and state accountability systems have been so ineffective at lifting student achievement that accountability as we know it should probably be dropped by federal and state governments alike. Further, the committee objects to state laws that require students to pass an examination for a high school diploma. There is no evidence that such tests boost student achievement, the committee says, and some students, about 2 percent, are not getting their diplomas because they can’t—or think they can’t—pass the test. The headline of the May 2011 NRC press release is frank and bold in the way committee reports seldom are: “Current test-based incentive programs have not consistently raised student achievement in U.S.; Improved approaches should be developed and evaluated.”

Needless to say, the report can be expected to play an important role in the continuing debate over NCLB. Upon its initial release, the report captured top billing, appearing on Education Week’s front page. Certainly, the NRC intends for the report to influence the NCLB conversation, rushing a draft version to the media five months before the completed report was available to the public.

Unfortunately, the NRC’s strongly worded conclusions are only weakly supported by scientific evidence, despite the fact that NRC’s stated mission is “to improve government decision making and public policy, increase public understanding, and promote the acquisition and dissemination of knowledge.”

The Report

Reports from the NRC are generally treated as highly credible. The NRC convenes panels of outside experts who volunteer their time to provide consensus opinions on issues of policy significance. And this particular panel includes a number of especially qualified researchers (see sidebar). The committee chair, Michael Hout, is a member of the National Academy of Sciences; 7 of the 17 panel members have named professorships; 2 are deans (of law and education schools); and a majority have published articles about testing, accountability, or incentives.

When it comes to gathering together the general literature, both theoretical and empirical, on the use of incentives in various contexts, the committee’s work is solidly constructed. But this strong scientific discussion of theory and empirical analysis of incentives and accountability breaks down when it comes to the committee’s core purpose: evaluating accountability regimes in education that employ incentives and tests.

The report comes to two policy conclusions: NCLB and state accountability systems have proven ineffective and state-required high-school exams are counterproductive. The unequivocal presentation of the conclusions is clearly designed to leave little doubt in the minds of policymakers. When the underlying evidence is examined, however, it becomes apparent that neither conclusion is warranted. Instead of weighing the full evidence before it in the neutral manner expected of an NRC committee, the panel selectively uses available evidence and then twists it into bizarre, one might say biased, conclusions.

Selecting Evidence

To get a grasp of the bias that motivated the report’s authors, consider how its first conclusion is phrased:

Test-based incentive programs, as designed and implemented in the programs that have been carefully studied, have not increased student achievement enough to bring the United States close to the levels of the highest achieving countries.

Note especially that the conclusion does not say that there is no evidence that testing and accountability work. It says that testing and accountability, by themselves, cannot lift the United States to the level of accomplishment reached by the world’s highest-achieving countries, an extraordinary standard for evaluating a policy innovation. To catch up to the leading countries would require gains of at least half of a standard deviation, or roughly two years of learning (see “Are U.S. Students Ready to Compete?” features, Fall 2011). No individual reform on the public agenda—neither merit pay, class size reduction, salary jumps for teachers, nor Race to the Top—can claim or even hope for anything close to that level of impact. The appropriate question is not whether testing and accountability is a panacea, but whether it has proven worthwhile.

By that more appropriate standard of judgment, the committee’s own data indicate that testing and accountability have proven effective, if not quite the spectacular success promised by those who enacted NCLB into law. The committee report tells us that the average estimated impact of these interventions is 0.08 standard deviations of student achievement. In other words, the average student in a state without accountability would have performed at the 53rd percentile of achievement had that student been in a state with an accountability system, all other things being equal.

That estimate may well be too low. The report states that “our literature review is limited to studies that allow us to draw causal conclusions about the overall effects of incentive policies and programs,” and then it goes on to describe several types of studies that would be excluded by this criterion. Where does the 0.08 come from? The committee considers a review from 2008 of 14 studies, and 4 studies conducted after that review. The review presents an average impact of 0.08. The NRC committee apparently felt no need to look any further and ignored the fact that a majority of the 14 studies would not come close to meeting its standard of enabling a “causal conclusion.” The committee determines that one of the more recent studies also supports an estimate of 0.08, although that study’s authors prefer estimates that are much higher. The 14 earlier studies and the 4 later ones produce a wide distribution of estimated impacts, but the committee makes no attempt to investigate whether the unusual estimates suggest circumstances under which accountability seems particularly effective (or ineffective). The committee chooses to emphasize the studies with negative findings (10 percent) while downplaying a number of those that have positive findings (90 percent). Thus the NRC mantra, repeated with slightly different wording throughout the report: “Despite using them for several decades, policymakers and educators do not yet know how to use test-based incentives to consistently generate positive effects on achievement and to improve education.” Apparently, the inconsistent results heralded in the press release reflect the 10 percent of studies that differed from the overwhelming majority.

Small Gains Add Up

Let us put this concern aside and consider the increment in student performance of 0.08 standard deviations of individual achievement that the committee presents as its best estimate. Is that so small an effect that it cannot justify continuation of testing and accountability? Consider that this is the average effect of a program that has been implemented on a national scale, affecting students across the country. We are hard pressed to come up with any other education program working at scale that has produced such results. Moreover, these average gains are the result of accountability systems that many people believe have important flaws. Even larger gains might be expected if those flaws could be corrected, as many experts, though not the NRC panel, have suggested.

The estimated benefits from a 0.08 standard deviation gain in student performance vastly outweigh its estimated costs. The cost of designing, administering, grading, and reporting the results from statewide examinations have been estimated at between $20 and $50 per pupil, a trivial sum considering that per-pupil education expenditures in the United States run above $12,000 annually. Most reforms—including class size reduction, merit pay, across-the-board raises for teachers, in-service training programs, or the scaling up of charter schools—would cost many, many times as much. For these innovations to have the same kick for every dollar invested, results would have to be improbably large.

The NRC, instead of considering these actual costs, suggests that implicit costs in the form of narrowed curricula are the most important, but it provides no evidence for its view.

What might the economic impact of a 0.08 standard deviation improvement in average achievement nationwide be? Along with University of Munich professor Ludger Woessmann, I have estimated the impact on U.S. Gross Domestic Product (GDP) of higher levels of student achievement. These estimates project the historical pattern of growth to determine the result of gains in student achievement, calculate the additions to GDP over the next 80 years, and discount them back to today so that they are comparable to other current investments. A 0.08 improvement has a present value of some $14 trillion, very close to the current $15 trillion level of our entire GDP, and equivalent to $45,000 for every man, woman, and child in the U.S. today. In other words, an inexpensive program that affects every student nationwide can, over the long run, have a very large impact, even if its average effect seems at first glance to be quite small. Indeed, if we figured testing cost $100 per student each year for the next 80 years and we tested all students rather than the limited grades tested now, the rate of return on the investment would be 9,189 percent. Google investors would be envious.

Several omissions from the report are also noteworthy. The report gives only passing attention to the positive impact of NCLB on the education of the most disadvantaged students, a consequence of the requirement to report performance by specific subgroups (e.g., racial and ethnic groups and the economically disadvantaged). The NRC report’s main reference to this feature of current accountability systems is that consideration of subgroup performance has added analytical difficulties because of the smaller samples.

Perhaps more telling, this panel of experts on testing and incentives makes absolutely no effort to describe how accountability programs could be improved. Being good researchers themselves, they do favor continued research on testing, however, and provide recommendations on what research should be done, which not surprisingly matches their own interests and expertise.

Lower the Bar?

The report also addresses a second, widely used accountability policy: high-school exit exams that hold students responsible for meeting a set of content standards. The report’s second conclusion reads,

The evidence we have reviewed suggests that high school exit exam programs, as currently implemented in the United States, decrease the rate of high school graduation without increasing achievement.

The panel strongly suggests that states that impose an exit exam should repeal this requirement. To understand this conclusion, it is necessary to understand the exams themselves and to evaluate the evidence behind the committee’s conclusion.

Currently, more than half of the states require that students pass a test of some sort to obtain a normal diploma (see Figure 1), and virtually all of these current requirements have been put in place since 2000. The tests almost always cover English and math, but many states add science and history. Test difficulty varies by state, but the modal level is grade 10. Although that standard may seem low, it is considerably more stringent than the standards that existed prior to 1990, when no state had a test reaching even the 9th-grade level. The current tests are not as high a barrier to high school graduation as they are often alleged to be, as a student may generally take the exam multiple times in order to achieve a passing score. And in all but three states (South Carolina, Tennessee, and Texas), students can either appeal the test result, if they feel the score misrepresents their accomplishments, or obtain a diploma by some alternative path.

The motivations for administering exit exams are to create incentives for students to apply themselves to the task of learning and to set uniform (minimum) quality standards for the state’s schools. Such content standards provide guidelines to schools about what to teach. They also indicate to colleges and universities what knowledge and abilities a graduate can be expected to possess. And they give similar information to prospective employers.

According to the best available evidence (discussed below), perhaps 2 percent of students are induced to drop out of school either because of failure to pass the exam or because of fear of not being able to pass the exam. Implicitly, the committee assumes this consequence does considerable harm to the affected students, given the substantial economic rewards that accrue, on average, from receiving a high school diploma. But average effects do not necessarily apply to the 2 percent on the border line between graduating and failing to graduate from high school. The impact for this particular group of students is likely to be much less, unless you make the bizarre assumption that it is only the diploma—not what the student learns—that affects job prospects and future income. The people who are induced to drop out because they cannot pass a 10th-grade exam would most likely be near the bottom of the earnings distribution of graduates were they to be handed a diploma. The economic impact on these students will be much lower than the average difference between graduate and dropout.

Perhaps the best argument against exit exams is simple: If a student shows up for school for 12-plus years and cannot pass a 10th-grade exam, it must be the school’s fault, and it would be unfair to hold the student responsible. This argument, interestingly enough, is the precise opposite of one of the primary arguments against the testing and accountability provisions of NCLB: We should not hold schools responsible for low achievement, because achievement is affected by student motivation and family background characteristics beyond the school’s control. Taken together, the arguments embedded in the committee’s two conclusions imply that nobody—not schools, not teachers, not even students themselves—bears responsibility for low student achievement.

Interestingly, the committee’s conclusion with respect to exit exams does not pick up on the full report’s emphasis on the importance of the design features of incentive systems, which include warnings that tests aimed at ensuring minimum competency may lower expectations, and concerns about both the potential narrowing of the curriculum and the tendency for score inflation on a known test. Instead, the presumed problem is the inherent unfairness of denying a diploma to a student who has met the attendance and course distribution requirements for a diploma.

If the main objective is to maximize high school graduation, there are many ways to do that. We could eliminate all exams, even those administered by teachers. We could loosen up course requirements. We could offer the diploma after 10 or 11 years of schooling, instead of 12. Of course, nobody is willing to take such steps, even though class exams, course requirements, and the inclusion of the 12th grade of schooling all have negative impacts on graduation rates. So why then does the NRC promote the idea of eliminating a 10th-grade-level examination as a requirement for high school graduation on the narrow basis that a few students will, as a result, not earn the degree? Is the NRC also against the movement of many states toward increasing the required amount of math or moving to college and career-ready standards?

The Data Shuffle

Let’s examine the evidence the committee supplies for its exit exam conclusion. The report marshals three studies that explore the issue: two on dropouts and one on achievement. Evaluating the impact of exit exams on achievement is inherently difficult. Because the exams apply to everybody in a state at the same time, it is not possible to compare students of the same age within the same state to find out the impact of exams. It is possible, however, to look at different cohorts of students, for example, those who attended school before the exam was in place and those who attended after, and to compare these to similar cohorts in other states where no such change in policy took place. In conducting this type of study, one must rule out other differences, such as those in family background or those in state education policies that might also affect student performance over time. Even when these challenges are met, one cannot be entirely sure of the results, as exit exams may influence student and school performance even before they come into effect, if teachers and students know that they will soon be introduced, which is usually the case.

The committee tosses out every exit-exam study (save three) that has ever been conducted on the grounds that it is not possible “to draw causal conclusions about the overall effects of test-based incentives” (that is, the very same criteria the committee ignored in considering school-level accountability). Some of the excluded studies use the well-regarded quasi-experimental technique known as regression discontinuity analysis. In the committee’s view, “Such regression discontinuity studies provide interesting causal information about the effect of being above or below the threshold, but they do not provide information about the overall effect of implementing an incentives program.” That criticism is odd, since the impact of an exit examination is of special interest for exactly those students on the cusp of adequate levels of achievement. While these excluded studies are not really appropriate for studying achievement, they tend to show little impact of exit exams on dropout behavior or graduation outcomes.

The committee relies for its conclusion regarding exit examinations exclusively on a 2009 study by Eric Grodsky, John Robert Warren, and Demetra Kalogrides. Because of the significance of this piece of research for the committee project as a whole, it is worth considering in some depth. The Grodsky team identified trends in student achievement in each state that administers an exit examination by drawing on data provided by the long-term trend assessments of the National Assessment of Educational Progress (NAEP). The long-term NAEP, begun in the late-1960s and continued with testing every few years, was designed to provide consistent score information to judge achievement of the nation as a whole. It was not designed to be used to evaluate the schools of any particular state or district. As a result, NAEP never collected in its long-term trend assessment a representative sample of students for any specific state, and the median number of tested students in each state was very small.

Grodsky et al. pretend that the NAEP provides them with just that: a representative sample of students for each state. They assume that the average performance of students in each state on the long-term NAEP provides an accurate measure of the average performance of students in that state, thereby violating the first principle of statistical sampling.

They then merge the information with information on the timing of the adoption of an exit exam by a state between 1971 and 2004. The study includes observations of math and reading achievement at 9 and 10 different points in time, respectively. The researchers report results for achievement of 13-year-olds and 17-year-olds separately, acknowledging that there are limitations to using either cohort. Thirteen-year-olds may be too young to detect the impact of exit exams, while the sample of 17-year-olds suffers from the noninclusion of school dropouts.

The Grodsky analysis encounters a further difficulty. For the most part, the researchers consider only the very early years, when exit exams were first introduced, a time when the exams were set at a very low level of difficulty, below that of a 9th-grade student. Only 1 percent of the observations included in their analysis are for states that had an exit exam rated at the 9th-grade level or higher, as most current examinations are.

Not only does the Grodsky team rely on inadequate data, but the analysis itself is flawed. Any attempt to see the effects of state tests should compare the changes that occur in the states that introduce them with changes in the states that do not. But the Grodsky study effectively tosses out all the information available for the 27 states that do not have an exit examination before 2004. As important, the analysis does not consider any measures of state policies except for exit exams, implying that any other policy changes for the three decades between 1971 and 2004 are either irrelevant for student performance or are not correlated with the introduction and use of exit exams.

The central finding is that exit exams do not have a statistically significant effect on test scores. But this insignificance could arise because of any or all of the above-mentioned problems rather than the absence of an effect of exit exams, as the NRC committee wants us to presume.

The committee’s estimate of the effects of exit exams on school dropout rates is less controversial. It relies on two quite reliable studies, although they are not without limitations: they study the effects of specific exit exams, which may not generalize to other arrangements. The studies indicate that perhaps 2 percent of potential high-school graduates would have received the diploma had it not been for the exit exams.

The committee touts the possibility of alternative incentives to exit exams: “Several experiments with providing incentives for graduation in the form of rewards, while keeping graduation standards constant, suggest that such incentives might be used to increase high school completion.” The key of course is just what the phrase “while keeping graduation standards constant” means. The idea behind exit exams is to ensure a minimum level of quality, as distinct from meeting the course completion requirements. Moreover, the report never makes the case that exit exams and other potential incentive programs are mutually exclusive. In principle, nobody would argue against employing other incentive programs as long as they were worth the expense and, as the committee says elsewhere, do not introduce perverse incentives of one kind or another.

The Takeaway

The NRC clearly wants to enter into the current debate about the reauthorization of NCLB. And the NRC has an unmistakable opinion: its report concludes that current test-based incentive programs that hold schools and students accountable should be abandoned. The report committee then offers three recommendations: more research, more research, and more research. But if one looks at the evidence and science behind the NRC conclusions, it becomes clear that the nation would be ill advised to give credence to the implications for either NCLB or high-school exit exams that are highlighted in the press release issued along with this report.

The framing of policy in the NRC report is simple: “The small or nonexistent benefits that have been demonstrated to date suggest that incentives need to be carefully designed and combined with other elements of the educational system to be effective.” Nobody would oppose careful design of incentives. Nobody would oppose evaluating the intended and unintended outcomes of incentives. And nobody would oppose combining carefully designed incentives with “other elements of the educational system to [make them] effective.”

The NRC is careful to offer no guidance on how NCLB or state exit exams might be modified to make them more effective. And the NRC is very careful not to offer any guidance on “other elements of the educational system.” The message that comes through is clear: keep working on test development, but never use tests for any incentive or policy purposes.

A better takeaway message might be, “Never rely on the conclusions of this NRC report for any policy purpose.”

Eric Hanushek is senior fellow at the Hoover Institution of Stanford University and member of the Koret Task Force on K–12 Education.

American education has problems, almost everyone is willing to concede, but many think those problems are mostly concentrated in our large urban school districts. In the elite suburbs, where wealthy and politically influential people tend to live, the schools are assumed to be world-class.

Unfortunately, what everyone knows is wrong. Even the most elite suburban school districts often produce results that are mediocre when compared with those of our international peers. Our best school districts may look excellent alongside large urban districts, the comparison state accountability systems encourage, but that measure provides false comfort. America’s elite suburban students are increasingly competing with students outside the United States for economic opportunities, and a meaningful assessment of student achievement requires a global, not a local, comparison.

We developed the Global Report Card (GRC) to facilitate such a comparison. The GRC enables users to compare academic achievement in math and reading between 2004 and 2007 for virtually every public school district in the United States with the average achievement in a set of 25 other countries with developed economies that might be considered our economic peers and sometime competitors. The main results are reported as percentiles of a distribution, which indicates how the average student in a district performs relative to students throughout the advanced industrialized world. A percentile of 60 means that the average student in a district is achieving better than 59.9 percent of the students in our global comparison group. (Readers can find all of the results of the Global Report Card at http://globalreportcard.org. The web site contains a full description of the method by which we calculated the results. For a summary, see the methodology sidebar.)

For the purposes of this article, we focus on the 2007 math results, although the GRC contains information for both math and reading between 2004 and 2007. We focus on 2007 because it is the most recent data set, and we focus on math because it is the subject that provides the best comparison across countries and is most closely correlated with economic growth. Readers should feel free to consult the GRC web site to find reading results as well as results for other years.

The Example of Beverly Hills

It is critically important to compare exclusive suburban districts against the performance of students in other developed countries, as these districts are generally thought to be high-performing. The most wealthy and politically powerful families have often sought refuge from the ills of our education system by moving to suburban school districts. Problems exist in large urban districts and in low-income rural areas, elites often concede, but they have convinced themselves that at least their own children are receiving an excellent education in their affluent suburban districts.

Unfortunately, student achievement in many affluent suburban districts is worse than parents may think, especially when compared with student achievement in other developed countries. Take for example Beverly Hills, California. The city has a median family income of $102,611 as of 2000, which places it among the top 100 wealthiest places in the United States with at least 1,000 households. The Beverly Hills population is 85.1 percent white, 7.1 percent Asian, and only 1.8 percent black and 4.6 percent Hispanic. The city is virtually synonymous with luxury. A long-running television show featured the wealth and advantages of Beverly Hills high-school students (as well as their overly dramatic personal lives). If Beverly Hills is not the refuge from the ills of the education system that elite families are seeking, it’s not clear what would be.

But when we look at the Global Report Card results for the Beverly Hills Unified School District, we don’t see top-notch performance. The math achievement of the average student in Beverly Hills is at the 53rd percentile relative to our international comparison group. That is, one of our most elite districts produces students with math achievement that is no better than that of the typical student in the average developed country. If Beverly Hills were relocated to Canada, it would be at the 46th percentile in math achievement, a below-average district. If the city were in Singapore, the average student in Beverly Hills would only be at the 34th percentile in math performance.

Of course, people don’t think of Beverly Hills as a school district with mediocre student achievement. This is partly because people assume that affluent suburbs must be high achieving and partly because state accountability results inflate achievement by comparing affluent suburban school districts with large urban ones. According to California’s state accountability results, the average student in Beverly Hills is at the 76th percentile in math achievement relative to other students in the state. But outperforming students in Los Angeles, which is only at the 20th percentile in math relative to a global comparison group, should provide little comfort to Beverly Hills parents.

Los Angeles Unified is not the main source of competitors for Beverly Hills students, so the state accountability system encourages the wrong comparison. If Beverly Hills graduates are to have the kinds of jobs and lifestyles that their parents hope for them, they will have to compete with students from Canada, Singapore, and everywhere else. Beverly Hills students have to be toward the top of achievement globally if they expect to get top jobs and earn top incomes.

Results from Affluent Suburbs Nationwide

We can repeat the story of Beverly Hills all across the country. Affluent suburban districts may be outperforming their large urban neighbors, but they fail to achieve near the top of international comparisons (see Figure 1). White Plains, New York, in suburban Westchester County, is only at the 39th percentile in math relative to our global comparison group. Grosse Point, Michigan, outside of Detroit, is at the 56th percentile. Evanston, Illinois, the home of Northwestern University outside of Chicago, is at the 48th percentile in math. The average student in Montgomery County, Maryland, where many of the national government leaders send their children to school, is at the 50th percentile in math relative to students in other developed countries. The average student in Fairfax, Virginia, another suburban refuge for government leaders, is at the 49th percentile. Shaker Heights, Ohio, outside of Cleveland, is at the 50th percentile in math. The average student in Lower Merion, Pennsylvania, near Philadelphia, is at the 66th percentile. Ladue, Missouri, a wealthy suburb of St. Louis, is at the 62nd percentile. And the average student in Plano, Texas, near Dallas, is at the 64th percentile in math relative to our global comparison group.

All of these communities are among the wealthiest in the United States. All are overwhelmingly white in their population. All of them are thought of as refuges from the dysfunction of our public school system. But the sad reality is that in none of them is the average student in the upper third of math achievement relative to students in other developed countries. Most of them are barely keeping pace with the average student in other developed countries, despite the fact that the comparison is to all students in the other countries, some of which have a per-capita gross domestic product that is almost half that of the United States. In short, many of what we imagine as our best school districts are mediocre compared with the education systems serving students in other developed countries.

Pockets of Excellence

While many affluent suburban districts have lower achievement than we might expect, some districts are producing very high achievement even when compared with that of students in other developed countries. For example, the average student in the Pelham school district in Massachusetts is at the 95th percentile in math. That means that if we were to relocate Pelham to another developed country in our comparison group, the average student in Pelham would outperform 95 percent of the students in math. That’s very impressive.

Of course, Pelham is a small district that is home to Amherst College, among other institutions of higher learning, and serves a rather select group of students. But not all college-town school districts are equally high achieving. As we have already seen, Evanston, Illinois, is at the 48th percentile in math in a global comparison. Palo Alto, California, the home of Stanford University, is at the 64th percentile. And the average student in Ann Arbor, Michigan, home to the University of Michigan, is at the 58th percentile in math relative to students in other developed countries. So, the 95th percentile math achievement in Pelham is outstanding, even for college towns.

Spring Lake, New Jersey, has a similarly impressive record of having the average student at the 91st percentile in math. It is a very small and affluent community on the New Jersey shore that has somehow escaped the influence of Snooki and The Situation. Waconda, Kansas, a small rural community, also is at the 91st percentile. Highland Park, Texas, an affluent community near Dallas, is at the 88th percentile.

Interestingly, of the top 20 U.S. public-school districts in math achievement, 7 are charter schools (some states treat charter schools as separate public-school districts). And most of the 13 traditional districts remaining are in rural communities rather than in a large suburban “refuge” from urban education ills.

Pools of Failure

In total, only 820 of the 13,636 public-school districts for which we have 2007 math results had average student achievement that would be among the top third of student performance in other developed countries. That is, 94 percent of all U.S. school districts have average math achievement below the 67th percentile. There aren’t that many truly excellent districts out there.

Of the 13,636 districts, 9,339, or 68 percent, have average student math achievement that is below the 50th percentile compared with that of the average student in other developed countries. Most of our large school districts are well below the 50th percentile. This is especially alarming, because these lower-performing large districts comprise a much greater share of the total student population than do the relatively small higher-performing districts.

The average student in the Washington, D.C., school district is at the 11th percentile in math relative to students in other developed countries. In Detroit, the average student is at the 12th percentile. In Milwaukee, the average student is at the 16th percentile. Cleveland is at the 18th percentile. The average student in Baltimore is at the 19th percentile in math relative to students in other developed countries. In Los Angeles, the average student is at the 20th percentile. The average student in Chicago is at the 21st percentile in math. Atlanta is at the 23rd percentile. The average student in New York City is at the 32nd percentile in math. And in Miami-Dade County, the average student is at the 33rd percentile in math.

Not 1 of the largest 20 school districts is above the 50th percentile in math relative to other developed countries. Those districts contain almost 5.2 million students or more than 10 percent of the country’s schoolchildren. The rare and small pockets of excellence in charter schools and rural communities are overwhelmed by large pools of failure.

Previous Research

The Global Report Card is not the first analysis to compare the performance of U.S. students to international peers. Eric A. Hanushek, Paul E. Peterson, and Ludger Woessmann (see “Teaching Math to the Talented,” features, Winter 2011) used a very similar method to compare the performance of students in each state to students in other countries and arrived at similarly gloomy conclusions. Using state NAEP results for 8th-grade students and PISA results for 15-year-olds internationally, the researchers focused on the percentage of students performing at an advanced level in math. In almost every state, they found that we had far fewer advanced students than most of the countries taking PISA. They also narrowed the comparison to white students in the U.S. and to students whose parents had a college education to show that even advantaged students in the U.S. failed to achieve at an advanced level in math relative to their international peers. More recently, Hanushek et al. updated their analysis to examine the percentage of students in each state and across countries performing at the proficient level in math and reading.  The results were similarly disappointing.

The main difference between the GRC and the Hanushek et al. analyses is that in our study we push the comparison down to the district level. By focusing on white students and children of college-educated parents, Hanushek et al. clearly mean to convey that even students in elite suburban districts have mediocre achievement. Our contribution with the GRC is to name the districts so that people do not indulge the fantasy that their suburb’s record is somehow different from the disappointing performance of others with advantaged students in their state.

There are other important differences between the GRC and the Hanushek et al. analyses. We incorporate test results for U.S. students in all available grades (typically grades 3 through 8 and grade 10) rather than focusing on the grade closest to the 15-year-olds in the PISA sample. We could have focused only on 8th-grade results, as Hanushek et al. did, but in doing so we would have greatly reduced the number of test results on which we were doing the calculations for school districts. We preferred to gain precision in estimating the achievement in each district by increasing our sample size rather than restricting the sample to 8th graders in order to gain comparability in the age of the students under review.

The GRC analysis also differs from those of Hanushek et al. in that the latter focus on students performing at the advanced or proficient level, while we focused on the average student performance in both math and reading. Hanushek et al. concentrated on advanced or proficient performance because they were trying to compare our best students with the best abroad to show that even our best are mediocre. We did the same by highlighting the results for elite suburban school districts. Focusing on the average also avoids any dispute about how “advanced” or “proficient” are defined across different tests.

Gary Phillips at the American Institutes for Research has also conducted a series of analyses comparing state achievement on NAEP to international performance on a different international test, the Trends in International Mathematics and Science Study (TIMSS). Phillips arrives at somewhat less gloomy conclusions about U.S. performance, but that is because the countries included in TIMSS differ from those covered by PISA. Hanushek et al. rightly note that PISA provides a much more appropriate comparison for the U.S.: “Put starkly, if one drops from a survey countries such as Canada, Denmark, Finland, France, Germany, and New Zealand, and includes instead such countries as Botswana, Ghana, Iran, and Lebanon, the average international performance will drop, and the United States will look better relative to the countries with which it is being compared.”

This has sparked a debate among researchers about whether TIMSS or PISA provides a better set of countries against which we should compare the U.S. The Global Report Card circumvents this dispute by developing its own set of countries against which we compare U.S. students. The comparisons provided by TIMSS and PISA depend on which countries decide to take each test each time it is administered. And PISA scales its scores against the results for members of the OECD, which excludes countries like Singapore while including countries like Mexico. Our comparison group depends on PISA results, but it is also based on objective criteria, like per-capita GDP, to identify a set of developed economies that can be reasonably compared with that of the U.S. Our comparison group is a significant improvement on the self-selection of countries that choose to take a test as well as an improvement upon arbitrary membership in an organization like the OECD.

No Refuge

The elites, the wealthy families that have a disproportionate influence on politics, clearly recognize the dysfunction of large urban school districts and have sought refuge in affluent suburban districts for their own children. But the reality is that there are relatively few pockets of excellence to which these families can flee.

In four states, there is not a single traditional district with average student achievement above the 50th percentile in math. In 17 states, there is not a single traditional district with average achievement in the upper third relative to our global comparison group. And apart from charter school districts,  in over half of the states, there are no more than three traditional districts in which the average achievement would be in the upper third.

The elites in those states have almost nowhere to find an excellent public education for their children. But state accountability systems and the desire to rationalize the lack of quality options have encouraged the elites to compare their affluent suburban districts to the large urban ones in their state. These inappropriate comparisons have falsely reassured them that their own school districts are doing well.

This false reassurance has also perhaps undermined the desire among the elites to engage in dramatic education reform. As long as the elites hold onto the belief that their own school districts are excellent, they have little desire to push for the kind of significant systemic reforms that might improve their districts as well as the large urban districts. They may wish the urban districts well and hope matters improve, but their taste for bold reform is limited by a false contentment with their own situation.

But the elites should not take comfort from the stronger performance of affluent suburban districts relative to large urban districts. As the Global Report Card reveals, even our best public-school districts are mediocre when compared with the achievement of students in a set of countries with developed economies.

Of course, the Global Report Card does not isolate the extent to which schools add or detract from student performance. Factors from student backgrounds, including their parents, communities, and individual characteristics, have a strong influence on achievement. But the GRC does tell us about the end result for student achievement of all of these factors, schools included. And that end result, even in our best districts, is generally disappointing.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute. Josh B. McGee is vice president for public accountability initiatives at the Laura and John Arnold Foundation.

LAUSD is being sued by a group that includes Alice Callaghan, a member of the Episcopalian clergy and the manager of Las Familias del Pueblo, a community center for the poor and homeless in downtown Los Angeles. Back in 1996, Callaghan organized 70 Spanish-speaking immigrant parents, who boycotted the Ninth Street Elementary School — calling for an end to failed bilingual-education methods and instead demanding that the school system teach the children of immigrant garment workers academic English as soon as possible.

Callaghan and this different group of parents are suing to enforce the Stull Act. The law goes back four decades and says that the board of trustees of each school district shall evaluate teachers, at least in part, by their student’s performance on the state’s standards-based tests. The law says “shall,” not “may.” It is mandatory that each district do this.

(The law is named for its sponsor, now-deceased Republican Assemblyman John Stull of San Diego, who received bipartisan support at the time for this statutory requirement that teachers be held accountable for the academic achievement of their pupils.)

The attorneys for the plaintiffs are Kyle Kirwan, a prominent Los Angeles litigator, and Scott Witlin, both partners at the law firm of Barnes & Thornburg. Their request for a court order was drafted in consultation with EdVoice, a Sacramento-based education-advocacy group. Before going to court, the plaintiffs sent a letter on Oct. 26 asking the district to comply. The letter stresses that for years the district has engaged in wanton lawlessness. In the letter, the plaintiffs’ attorneys say that the district “refuses to implement the Stull Act in complete abdication of its responsibility to its students, their parents, and the taxpayers of the district.”

The letter says that the district has never evaluated the teachers using student test scores, and, as a consequence, has never told teachers where they stood and counseled them on how to improve in terms of increasing their students’ learning – all of which are required by the law. “In short, the district has never complied with the Stull Act.”

The letter also points to the involvement of the teachers’ union United Teachers Los Angeles (UTLA) in this lawbreaking. Compliance with the law, the letter says, has been “deliberately evaded” through a series of “complicitous” collective-bargaining agreements between the LAUSD and UTLA, at the expense of students — who deserve effective teachers.

Specifically, the district has been pretending that it can avoid compliance with the Stull Act by making collective-bargaining agreements with the teachers’ union that overrule a statute (the Stull Act) passed by the state legislature. It doesn’t work that way. Valid contracts are written under and within the law, not in violation of the law. The lawsuit seeks to end this make-believe in the service of lawbreaking.

In their Nov. 1 petition for a court order, the plaintiffs’ attorneys say that the UTLA has treated the public school system in Los Angeles as “a taxpayer-funded jobs and entitlement program” for adults, even when a teacher‘s performance would be considered “demonstrably unsatisfactory” when judged by pupil results.

The petition described how the teachers’ union adopted a strategy of “stonewalling” when it came to putting the Stull Act into effect. “In collusion with the District‘s governing boards and superintendents,” the petition says, the teachers’ union has blocked lawful evaluation of teachers and the “corrective action” needed to ensure that students get effective teachers.

As a consequence, “the adults‘ collective employment and political interests” are turning the children’s opportunity for learning while in school “on its head” and instead the system is providing job guarantees to teachers as well as “preserving the political power of the Board and the Superintendent.” All of this comes at the expense of children — particularly the “socio-economically disadvantaged.”

These shenanigans by the district and the union have been presented to the public in a way that is designed to pull the wool over people’s eyes: “The result has been a perversion of the evaluation system and a knowing effort to deceive the public using educational jargon.”

Witlin, one of the attorneys, told education policy analyst and blogger RiShawn Biddle: “The school district is supposed to exist for the benefit of the children and not for the adults.”

The teacher evaluation program that is in place in Los Angeles, according to the petition, “does not comply with the Stull Act” and “perpetuates a fraud on the community” by letting teachers get high evaluation ratings whether or not their students are learning the material listed in the curriculum-content standards.

The petition cites damning statements from LAUSD Superintendent John Deasy in which he condemns his own evaluation program for teachers. For example, he recently said: “I would argue that nobody has told me that the current system of evaluation, which is performance review, helps anybody. It is fundamentally useless. It does not actually help you get better at [your] work and it doesn‘t tell you how well you’re doing.”

Superintendent Deasy also stated: “One would have to argue: ‘So … there are schools where 3 percent of the students are proficient at math and 100 percent of the teachers are at the top rating performance.’ That doesn‘t make sense to me whatsoever. And it doesn‘t make sense because the rating performance does not actually help teachers get better.”

In terms of what actually happens, the district is condemned out its own mouth.

Back on March 13, 2011, retired Los Angeles school district teacher Doug Lasken and I wrote an opinion column for the San Francisco Chronicle about non-compliance with the Stull Act in Los Angeles and other California districts – so I could not be happier about this lawsuit, which may finally bring some justice for Los Angeles schoolchildren after years of the district’s deliberate dodging of the law. Success in Los Angeles will mean that districts across California will have to begin evaluating teachers properly and getting struggling employees the extra help they need to become effective teachers.

LAUSD has been negotiating with UTLA to try to put in place a pilot program with three percent of district teachers, who would be evaluated in part on student performance on the state’s standards-based tests. But these negotiations are deadlocked because of the refusal of UTLA to even study the idea of complying with the law.

The plaintiffs in this case reject the proposed pilot program, which has no guarantee of ever having meaningful evaluations that actually count, even for the volunteer participants in the pilot. They point out that LAUSD has a record of “years of non-compliance” with the Stull Act and that there is no reason to believe that the pilot would even expand to the other 97 percent of teachers. “Sadly, the District has abdicated its duty to the children.” The plaintiffs demand instead that LAUSD comply with the Stull Act as soon as practically possible “in its entirety.”

-Bill Evers

Last night was fun for the kids, but today is every education wonk’s favorite holiday: NAEP release day! Kevin Carey is already out with some savvy analysis; let me add some thoughts on the trends in reading.

The big news is that we finally eked out some statistically significant progress in 8th-grade reading. This goal has eluded us before, and has led commentators such as E.D. Hirsch to note that we’re not doing enough to build kids’ content knowledge and vocabulary. Initiatives like Reading First might have helped our youngsters to decode, goes the argument, but that’s not enough to create strong readers, especially as kids get older.

That’s still true, I think, but the NAEP results might indicate that those decoding skills are nothing to scoff at. The middle schoolers who took the NAEP last spring were in first grade in 2004–the heyday of Reading First implementation. It’s possible that scientifically-based reading instruction got them off to a better start as readers, and that head-start has been maintained through elementary and middle school. I can’t prove it (it’s NAEP–no one can prove anything!) but it’s a hypothesis worth exploring. Furthermore, the 8th graders who made the greatest progress since the early 2000s were the lowest-achievers–the very population Reading First was designed to help.

What’s disappointing is that 4th-grade reading results have held steady since 2007–after a big bump up (across all achievement levels) from 2005-2007. This one-time bump might be credited to Reading First (again, stress on “might”). But only a few states have continued making progress; in the last two years only Alabama, Hawaii, Maryland, and Massachusetts have done so. I can’t quite explain Maryland and Hawaii (OK, maybe they DO deserve Race to the Top funds, after all) but Alabama and Massachusetts have some of the most aggressive policies in place to promote research-based reading instruction. And it shows.

Again, these are just guesses. The big question going forward, it seems to me, is whether any of our reform efforts are like to lead to another big bump in test scores anytime soon. Large-scale initiatives like “accountability” and “parental choice” set the context for improvement, but they have rather indirect impacts on achievement. More focused instructional and teacher quality strategies–like implementing the Common Core standards or improving teaching through better evaluation systems–are more likely to result in big gains, it seems to me. Neither of those will be in full force until around 2013 or 2014. So I would expect more steady-state on NAEP scores for the time being, with our next big chance for major gains coming in 2015.

Those are my thoughts. What about you?

-Mike Petrilli

This post also appeared on Flypaper.

I must admit to a bias: I am a strong advocate of national standards, was intimately involved with their first iteration in the 1980s, and am delighted to witness their partial resurrection in a new guise.  As Robert Rothman observes, the new Common Core standards in English language arts and mathematics are not top-down driven reforms (one of the difficulties of the first national standards initiatives) or bottom-up efforts, which have suffered in the past when the states, with the singular exception of Massachusetts, tended to water down their individual attempts to the detriment, rather than the amelioration, of American public education.

The Common Core standards, to quote directly from the book’s introduction, “set expectations for student learning at every grade level,” and the book “describes the development process, the states’ adoption decisions, and early steps by states to implement them.”  Most important, the book explains in depth the content of the standards, what they expect of students, and how the assessment of student results is going to be carried out.

While all of this activity, strenuous and complex as it is, may seem to the educational neophyte to be more theoretical than practical, the fact of the matter is that within six months of the standards being issued in 2010, 43 states and the District of Columbia had adopted them.  Furthermore, they are designed to be “all or nothing.”  (It is difficult if not impossible to adopt some of them.)  They are written with every student in mind, rather than for the gifted few.  Their potential for transforming what is taught and raising the level of academic achievement nationally is truly extraordinary.

Why am I guilty of such unbridled optimism?  First of all, a great deal was learned from the pre-Common Core efforts. The first version of the national standards in the 1980s was vastly too ambitious.  Second, current federal education policy is very favorably disposed towards the common core initiative. Third, international comparisons with other highly developed countries, once shunned, are now fashionable.  They reveal that, no matter how the tests are framed, America is in the middle of the pack, well behind the likes of Finland, Singapore and Japan, in what we traditionally expect of our high school graduates. Fourth, other organizations are in the process of developing common core standards in science (to be released in 2012).  Assuming that they are of the same high quality as their 2010 counterparts, people may be emboldened to do the same for the other basic subjects, and thus escape from the current tendency to narrow the curriculum to the point of no return, a concern of particular moment to Diane Ravitch.  Fifth, unlike the situation in the 1980s, the charter school movement has matured to the point that its growth can provide a nationwide institutional context to pilot the teaching strategies appropriate to implementing the common core strategies.

Finally, there is a significant movement to align teacher education with these new standards.  Ross Perot once said to me “All teacher colleges ought to be torched.”  Such single-minded excoriation may be over the top, but there is no question that the new three R’s of teacher recruitment, retention and renewal are integral to any genuine education renaissance, and are indispensable to the implementation of the common core standards. Let’s hope that all of this really happens.  Robert Rothman certainly thinks there is a good chance it will.  After all, all these favorable circumstances are referenced in this informative volume.

-A. Graham Down

More book reviews by Graham Down can be found here.

For nearly a decade, the No Child Left Behind Act (NCLB) has focused the attention of policymakers and researchers squarely on the achievement of low-performing students, with some apparent success. The math and reading scores on the National Assessment of Educational Progress of the nation’s lowest-achieving 10 percent of 4th and 8th graders have risen sharply since 2000, continuing a trend that began in the 1990s. Yet some may wonder about the potential cost of this focus on higher-achieving students, for whom improvements over the same time period have been modest. Among the questions related to this debate is whether additional programs and resources should be devoted to students on the higher end of the spectrum, those considered gifted.

Three million students in the United States are classified as gifted, yet little is known about the effectiveness of traditional gifted and talented (G&T) programs. In theory, G&T programs might help high-achieving students because they group them with other high achievers and typically offer specially trained teachers and a more advanced curriculum. While previous research indicates that ability grouping is in fact correlated with higher achievement, these findings could be misleading if students placed in high-ability classrooms were likely to be successful for reasons that researchers are unable to measure, such as stronger motivation. To our knowledge, no existing studies offer convincing evidence on the causal effect of G&T programs on student achievement.

Our research begins to fill this gap with two studies of the G&T programs available to high-achieving middle-school students in a large urban school district in the southwestern United States which, to preserve anonymity we shall refer to as LUSD. Since 2007, all 5th-grade students in LUSD have been evaluated to determine eligibility for gifted and talented programs starting in 6th grade. Those students who are deemed eligible often are grouped in classes with other gifted students. They are also permitted to apply for admission to two middle schools that have oversubscribed magnet G&T programs.

The two studies use different methods to ask distinct but closely related questions. The first exploits the fact that eligibility for G&T programming in LUSD is determined by a well-defined cutoff in students’ evaluation scores. By comparing students who score just above the cutoff to those who score just below, the study provides evidence on the effect of enrollment in a G&T program on achievement for those students on the margin of eligibility. The second study takes advantage of the randomized lotteries that determine admission to the district’s two premier magnet G&T programs. By comparing students who win the lottery and attend the magnet G&T schools to those who lose the lottery and attend other “neighborhood” programs, the research provides evidence on whether the magnet G&T programs provide any additional benefits.

The results of both studies will be discouraging for those hopeful that current G&T programs provide a means to accelerate the progress of our most capable students. The first shows that barely eligible students who participated in LUSD’s G&T curriculum for all of 6th grade and half of 7th grade exhibit no significant improvement in test scores across a range of subjects, despite their being surrounded by higher-achieving peers and taking more advanced courses. The lottery study corroborates these results, as students admitted to the G&T magnet schools show little improvement in test scores by 7th grade, despite having higher-achieving peers and being taught by more effective teachers. The lone exception is in science, where students admitted to G&T magnet schools performed at substantially higher levels.

It is difficult to know what accounts for these puzzling results. Our best guess, which we discuss in detail below, is that being placed with higher-achieving peers is not all that it is cracked up to be. Students admitted to both types of G&T programs suffer a large drop in their relative rank in terms of grades within their classes, which could have adverse consequences that offset any benefits of improvements in their educational environment. But we are getting ahead of ourselves. Let’s first take a closer look at the programs and the evidence on their effects.

Gifted Students in LUSD

LUSD is a large school district, with more than 200,000 students. The district is heavily minority and very low income; the minority population is more heavily Hispanic than African American. All LUSD students are evaluated for placement in middle-school G&T programs during 5th grade, including those who participated in the district’s G&T program in elementary school. In order to be deemed eligible for the middle school G&T program, a student must meet the eligibility criteria set forth in the “gifted and talented identification matrix.” The matrix converts scores on standardized tests—the Stanford Achievement Test for English-speaking students and the Aprenda exam for Spanish-speaking students with limited English proficiency—scores on the Naglieri Nonverbal Ability Test (NNAT), average course grades, teacher recommendations, and indicators for socioeconomic status into an overall index score.

While all students who meet these requirements qualify for the G&T program, not all end up being classified as G&T, because parents are allowed to opt out. Some students also enroll in the program initially but later withdraw. Schools in LUSD have a monetary incentive for attracting gifted students, as LUSD provides a funding boost of 12 percent over the average allotment for a regular student.

Gifted students in LUSD are far less likely to be economically disadvantaged and more likely to be white or Asian than other students in the district. They also perform at far higher levels on the Stanford Achievement Tests, which the district administers annually in five subjects: math, reading, language, social science, and science. Their advantage in math and reading test scores in 5th grade is roughly 0.7 of a standard deviation, which amounts to well over two years of academic progress (see Figure 1). By the time the same students have reached 7th grade, these gaps have widened to 1.5 standard deviations in math and 1.25 standard deviations in reading. While this pattern suggests that the students enrolled in the district’s G&T programs learn at a faster rate between 5th and 7th grade, it does not necessarily mean that the G&T programs are the cause. It is to that question we now turn.

Effects on Barely Eligible Students

Our first study examines the effects of participation in a G&T program on students who were just barely eligible to participate based on their overall index scores. We focus on students who were evaluated for G&T eligibility as 5th graders in the spring of 2008 for whom we are able to observe outcomes as 7th graders in the 2009–10 school year. Our outcome measures include Stanford Achievement Test scores and attendance rates, both of which are drawn from administrative data provided by the district. After restricting the sample to students near the G&T eligibility cutoff, we are able to examine these outcomes for roughly 2,600 students.

The method used in the study, known as regression discontinuity analysis, takes advantage of the fact that the district uses a strict numerical cutoff in the index score assigned to students as 5th graders in order to determine their eligibility to participate in the G&T program the following year. Because the students are unable to precisely manipulate their index scores, those scoring just below the eligibility cutoff should be very similar to those scoring just above the cutoff. We can therefore attribute any differences in student outcomes on either side of the cutoff to the effect of having being deemed eligible.

As noted above, not all eligible students end up participating in G&T programs due to factors such as a parent’s decision to opt out. Similarly, some students who do not initially qualify later become eligible through an appeals process that allows parents to submit an alternative standardized test score or through additional evaluations conducted in 6th grade. As a result, we use standard statistical techniques to account for the fact that the cutoff our regression discontinuity analysis exploits is “fuzzy” rather than sharp. This allows us to provide evidence on the effects of actual participation in the G&T program, not simply eligibility for it.

Before looking at student outcomes, we first used the same method to confirm that participation in the district’s standard G&T programs led to measurable differences in students’ educational experiences. Clearly, it did. The average achievement of the peers in G&T students’ classrooms were between 0.25 and 0.33 of a standard deviation higher in each core academic subject. Participation in the G&T program also increased the number of advanced courses in which students enrolled in 6th and 7th grade. We found no evidence, however, that the teachers to whom students in the G&T program were assigned were any more effective, as measured by their impact on student test scores.

Did these improvements in peer characteristics and curricular rigor translate into improved outcomes? Our results indicate that they did not (see Figure 2). Our estimates of the effects of G&T participation for barely eligible students are close to zero in all five subjects and are sufficiently precise to allow us to rule out with 90 percent confidence effects as small as 0.04 standard deviations (sd) in math, 0.07 sd in reading, 0.12 sd in language, 0.10 sd in social studies, and 0.19 sd in science. We also looked at the impact of G&T participation for specific student subgroups defined by gender, race/ethnicity, socioeconomic status, and whether the students had been classified as gifted in elementary school. We found little evidence of differential impacts for students in any of these groups.

The Effects of G&T Magnet Programs

Why does the G&T program in LUSD not yield benefits for students on the margin of eligibility? One reason could be that the qualification boundary is set so low that such students are not able to take advantage of the programs’ purported benefits. Our second analysis, which uses experimental research methods to study the effects of enrollment in the district’s G&T magnet programs, is intended to shed light on this concern.

LUSD has 41 middle schools, of which 8 have G&T magnet programs, and 2 of these are oversubscribed. As a result, the district uses lotteries to determine which students will be admitted as 6th graders. Our analysis compares the performance of students who win the lottery and attend one of the G&T magnet programs to those who lose the lottery and either attend a neighborhood G&T program in the district, a magnet school based on a different specialty, or a charter school. Because the lottery is random, any differences in outcomes between lottery winners and losers can be attributed to the effect of enrolling in the G&T magnet program rather than one of these alternatives. Moreover, the results of this analysis will apply to the entire population of students who chose to apply.

Our lottery analysis is based on the sample of LUSD 5th-grade students determined to be eligible for G&T programs in 2007–08 who applied for admission to one of the two middle schools with an oversubscribed G&T magnet program. This group includes 542 students, 394 of whom were offered admission and 148 of whom were not. We find no statistically significant differences in the observed characteristics of lottery winners and losers, suggesting that the lotteries were in fact conducted in a random way.

The students in the lottery differ both academically and demographically from the students who were included in the regression discontinuity study. Not only do the lottery students have higher test scores than students at the eligibility cutoff, but their test scores exceed those of the average G&T student in the district. Lottery participants are also less likely to be on subsidized lunch, and less likely to be minority.

Of the 542 lottery participants, only 440 students, including 331 winners (84 percent) and 109 losers (74 percent), remain in LUSD by 7th grade. Fortunately, the observed characteristics of lottery winners and losers who remain in the district continue to be very similar. Even so, when analyzing the data we control for students’ demographic characteristics and prior achievement, and use weights designed to make the final sample comparable in terms of its observed characteristics to the set of students that initially applied for the lottery.

One disadvantage of this second study is that the lottery losers have a range of alternative experiences and most participate in standard G&T programs, so the comparison group’s educational experience is less clear than it was in the regression discontinuity analysis. Nonetheless, our data confirm that students admitted to the G&T magnet schools with lotteries seem to have experienced large improvements in their educational environment. Winning the lottery increased the average achievement of students’ classroom peers by as much as a full standard deviation in some subjects. And in contrast to the G&T program as a whole, students admitted by lottery to G&T magnet program were assigned to more effective teachers.

Turning to student outcomes, however, our results provide little evidence that attending a G&T magnet program leads to improvements in student achievement (see Figure 3). The one exception is science test scores, for which we estimate a positive effect of 0.28 standard deviations. Due to the relatively small sample sizes, all of the effects are imprecisely estimated and do not allow us to definitively rule out reasonably large positive effects. Even so, the estimated effects for math, reading, and social studies are negative, and the estimated effect for language is effectively zero.

Discussion

It is difficult to understand why we find little evidence that G&T programs positively affect achievement. A common concern with studies of high-achieving students is that the available achievement measures may not be well suited to discern improvements for this group. This would be particularly worrisome if we were using a state accountability exam targeted toward low-achieving students, but it is less of an issue with the Stanford Achievement Tests. Indeed, we found little evidence of students performing near the maximum levels on these tests in either the regression discontinuity or lottery samples. Although it is possible that the additional course material taught in G&T classes is poorly aligned with topics covered in the achievement test, research documenting the benefits of being placed with higher-ability peers suggests that we should see improvements, even if that were the case.

The effect of being placed in a higher-ability classroom may not necessarily be positive, however, especially for a marginal G&T student. In particular, the drop in ranking relative to one’s peers may have a negative effect: a marginal G&T student is likely to go from being near the top of the regular class to being near the bottom of the G&T class. Even students in the middle of the G&T distribution are likely to experience a loss of ranking in the magnet G&T schools as compared to their neighborhood schools. It may be that students are demoralized by the drop in their relative rankings or that teachers provide more resources to students at the top of the class.

Substantial evidence from educational psychology indicates that students who are placed in higher-achieving groups can suffer psychological harm. A commonly used measure is a student’s “self-concept,” how a student perceives her abilities relative to an objective measure such as achievement. A 1995 study by Herbert Marsh and colleagues compared G&T students to observably similar students in mixed G&T and non-G&T classes and found that G&T students show declines in their math and reading self-concept. More recent research has documented lower self-concept and greater test anxiety among gifted students in ability-segregated classrooms.

Although we do not have direct evidence on student confidence, we can make use of student course grades and rank within the class to probe for evidence consistent with this kind of effect. We evaluate the impact of G&T program enrollment in the regression discontinuity study and of attending a G&T magnet in the lottery analysis. In both cases, we find clear reductions in student grades. For the regression discontinuity sample, grades fall by a statistically significant 4 points out of 100 (3 points changes a grade from a B+ to a B, for example) in math and by 2 to 3 points in other subjects, although these effects are not statistically significant for 7th grade. For the lottery analysis, the grade reductions are even more dramatic, with drops of 7 points in math, 8 in science, and 4 in social studies.

It is also useful to consider how students’ rankings within their peer groups differ by treatment status, as this provides a direct measure of how a student may perceive his position in the overall distribution of student ability. We assume that students mostly compare themselves to their schoolmates who take the same courses in the same grade. Thus, we rank students within each school, grade, and course by their final course grades and then convert these rankings to percentiles. The rankings based on 7th-grade courses exhibit notable drops when students cross the G&T eligibility threshold. Controlling for race, gender, economic disadvantage, LEP (Limited English Proficiency), and prior gifted status, marginal G&T students have a relative rank in 7th grade that is 13 to 21 percentiles lower than similar students who were not admitted. Attending a premier G&T magnet in 7th grade generates a nearly 30 percentile ranking drop in all four of the courses examined.

In short, the necessary conditions are clearly met for a drop in relative ranking to play a role in offsetting the expected positive impact of more rigorous courses, more effective teachers, and higher-achieving peers. The possibility that G&T students are subject to such a mechanism suggests potential constraints on the benefits of programs that provide more similar peers and an increase in traditional education inputs.

One should not conclude from the lack of achievement results, however, that the G&T programs should be scuttled. Our analysis occurs in a district with a large number of relatively high-quality magnet programs, and thus the alternatives to the G&T programs may be strong. There may also be benefits that we are not able to capture, such as impacts on SAT scores, graduation rates, and college attendance. Further, our study examines a G&T program in one district. Certainly, districts vary in the approaches they take to educating gifted students, so it may be that similar studies of programs in other districts would yield different results. Nonetheless, this study does raise questions about the efficacy of G&T programs and the traditional model of ability-segregated classrooms.

Sa Bui is a doctoral candidate in economics at the University of Houston, where Steven Craig is professor of economics and Scott Imberman is assistant professor of economics.

In “Accountability’s End?“, Petrilli divides accountability supporters into four distinct groups, which I found accurate. His categories also make it a lot easier to explain why teachers have so much trouble explaining that we are not against all accountability, so much as we have seen the idea of accountability misused for political reasons. We also have ongoing experience with its unintended consequences.

The truth is that while many teachers disagree strongly with some of the groups described in Petrilli’s article, we are in agreement with others. Opinions among teachers vary, so I won’t claim to speak for all of us, but here are my reactions to each group as Petrilli describes them.

The Tough Lovers: This group wants to make sure that teachers are not unduly shielded from a tough economy and that only hard, competent workers stay on school payrolls. Personally, I don’t mind. I’ve never had a problem with being expected to do a good job – few teachers I’ve ever met have a problem with that. Then again, I have a smart and fair-minded principal who isn’t likely to bully me over some comment I’ve made at a faculty meeting. Not all teachers are so lucky. More so than in the private sector, teachers let go for poor performance – or perceived poor performance – will likely have their careers permanently destroyed. It would be nice to see increased “tough-love” in HR departments balanced out with options for good teachers who get stuck in tough situations.

The World is Flatters: This group supports things like STEM and the Common Core standards as a means to make American education more cohesive and keep us competitive with other countries. I have no major disagreements with this group, and have yet to hear from a teacher who does.

The Tight-Loosers: This group favors results-based accountability as a means to cut back on traditional regulation. On paper, the idea of using some type of end results as a means of giving teachers more autonomy sounds great. For example, I’d love English teachers to be able to read more novels instead of giving expensive, time-consuming, relatively useless bi-weekly assessments. At the same time, teachers get uneasy about the “tight-looser” camp because we’ve seen firsthand that standardized tests don’t tell us everything accountability hawks say they do. Plus, increased emphasis on test results has yet to be matched with more autonomy in most public schools – instead they are the justification for things like replacing novels with bi-weekly assessments. For now, I’d describe the way it plays out as the “tight-tighter” approach.

The Poverty Warriors: This group claims that test-based accountability will keep schools and teachers from shortchanging poor, minority students. It is generally arguments from this group – however well intentioned – that frustrate me the most. Teachers at low-income schools often choose to work there in spite of problems known to impact student achievement. The “poverty warrior” rhetoric has recast these teachers as lazy, racist conspirators against poor kids. It is disingenuous and unfair to suggest that non-teachers in clean, well-decorated offices with all the copy paper they could ever ask for somehow care more about poor kids than teachers who get up at 5AM and break up hallway fights and work with these kids every day.

Teachers have also seen how many accountability measures – even well-meaning ones – have unintended consequences that undermine their stated goals. When we bring this up, however, even if we are really only arguing with one of the four groups above, we get slapped with the label of being against everything all four groups stand for, and thus treated as lazy, against what’s best for children, unrealistic about what kids need to know, and un-caring.

If we had those four characteristics, why, exactly, would we have chosen this profession?

While I don’t agree with every statement in Petrilli’s article, I still hope a lot of people read it. It is definitely a starting point for a more thoughtful discussion of this issue.

-Roxanna Elden

Roxanna Elden is a National Board Certified Teacher in Miami, and the author of See Me After Class: Advice for Teachers by Teachers.

I was prepared for a rant against all things reform when I started reading the New York Times Q & A interview with Maria Velez-Clarke, the principal of the Children’s Workshop School in Manhattan’s East Village, about the school’s C-grade from the City.  The school is “one of several small schools,” said the Times intro, “started in the 1990s by people who had worked at the widely praised Central Park East School.”

Central Park East?  The school started by Deborah Meier, current scourge of standardized tests, charters, accountability, and just about everything associated with Michael Bloomberg and Joel Klein, who initiatiated the school report cards program?  (See the Bridging Differences blog Meier shares with Diane Ravitch and this wonderful 1994 profile of Meier and her hugely successful Central Park East experiment written by veteran NYC educator Sy Fliegal.)  Children’s Workshop offers ballet and yoga, for heaven’s sake!

Instead of a progressive principal complaining about Gotham’s new accountability system squishing her student’s creative impulses, however, we hear an 18-year veteran school leader who was shocked by the C grade the school received in 2010 and determined to do something about it:

I shared it with absolutely no one because it was so devastating to me. I took it home. I sat with my husband and I said, “My God, do you know what this is going to do to morale?” And he looked at me and he said, “O.K., you have the weekend: have a pity party and then move on.”

Velez-Clark actually went to the Transit Museum and bought “C” buttons (for the C train), brought them to school, sat down with her staff and said, “O.K., now what do we have to do here in order to get off the C train and get on the B train?”

She then took her staff on a weekend retreat, where they reviewed every child’s test scores.  And what is most interesting about the school’s response is that Velez-Clark seems unafraid to admit that she has learned something that may be good for her students.

When she and her teachers began to dig into the test scores, for instance, they discovered

….a correlation between attendance and a child’s score. So we worked on attendance. I didn’t always send a note home before, but now sometimes if a child is absent too much, I have to send a letter home saying “this could lead to A.C.S. [Administration for Children's Services] coming to visit your house” or “your child is at risk of being held over because of attendance.”

The school didn’t stop being progressive, it simply integrated cultural history into its ballet lessons and nutrition and science into its yoga classes.

This is how it’s supposed to work.  Congratulations to the staff and children of Children’s Workshop for their B grade this year – and showing that it’s okay to do well in tests.

–Peter Meyer

This post also appears on Flypaper

A  more complete integration of testing, accountability, and teaching would be superior to dealing with the integrity of testing in isolation.   Let’s put the tests out in the sun instead of trying to lock them up in more and more secure rooms.

The use of student outcome measures for accountability is now firmly entrenched and is not about to go away.  But a variety of complaints about the current testing system exist.   First, the tests tend to narrow the teaching to just what is expected on the tests – with excessive teaching to the test and drilling on  practice tests.  Second, the tests are too easy for students in some schools and too hard for others, either wasting the time of some or frustrating others.  Third, using tests for accountability purposes encourages cheating and other ways to evade scrutiny.

To address these different issues, we need to think differently about aims and means.  Here is a brief proposal to deal with all of the problems.  It starts with developing a large item bank of test questions of varying difficulty.  Imagine 1,500 questions for fourth grade math that cover the entire scope of appropriate material from basic to advanced topics.  Next, make all of the test items – not just sample items – publicly available and encourage teachers to teach to the test, because the items cover the full range of the desired curriculum.  Making the items public will also ensure the quality of the test items.  One could invite feedback ratings or open sourcing to provide a path to improving the questions over time.  Then, move to computerized adaptive testing, where answers to an initial set of questions move the student to easier or more difficult items based on responses.  This testing permits accurate assessments at varying levels while lessening test burden from excessive questions that provide little information on individual student performance.  Such assessments would not be limited to minimally proficient levels that are the focus of today’s tests, and thus they could provide useful information to districts that find current testing too easy.  Students would be given a random selection of questions, and the answers would go directly into the computer – bypassing the erasure checks, the comparison of responses with other students, and the like.

This proposal actually follows the current testing by the FAA of knowledge needed to obtain a private pilot license.  While there are commercial books on these tests, replete with questions and answers, the efficient way to prepare for the tests is simply to learn the underlying concepts.  It is not to attempt to memorize the answers, because it is easy to confuse such an attempt.

What are the potential problems?  Some say developing test items would make this too costly, but remember that it is only necessary to have one item bank, not the continually changed banks of today.  Some worry about ensuring that sufficient computers are available in all schools, but with all of the digital devices currently in use, surely there are a range of possibilities to deliver the tests effectively and efficiently.  There is the problem that the testing companies would not particularly like the proposal.   They find they are happy with mindlessly developing slightly different variants of existing tests for different states, years, and administrations.  But, maybe there are more productive ways for them to enter into the process.

The proposed system would yield quick and reliable feedback on student achievement, would deal with the various cheating and gaming issues, and would more effectively define what students should know than the currently available standards.

-Eric A. Hanushek

Eric Hanushek is a senior fellow at the Hoover Institution of Stanford University and member of the Koret Task Force on K-12 Education.

On August 17, 2011 Paul Peterson discussed the findings of this study in a free online webinar. An archived recording of this webinar can be found here.

The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, “America simply is not producing enough of our own innovators, and the cause is twofold—a deteriorating K–12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science.” To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to “raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace.” Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to hire if they can find people with the right skills. As one factory owner put it, “It’s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor.” According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. On the basis of these projections, McKinsey’s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills.

In this paper we view the proficiency of U.S. students from a global perspective. Although we provide information on performances in both reading and mathematics, our emphasis is on student proficiency in mathematics, the subject many feel to be of greatest concern.

Student Proficiency on NAEP

At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established proficiency levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation’s report card.

In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions. Are U.S. schools failing to teach their students adequately? Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade?

One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage—or even a majority—of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries?

In this article, we report results from our second study of student achievement in global perspective conducted for Harvard’s Program on Education Policy and Governance (PEPG). In our 2010 PEPG report, we compared the percentage of U.S. public and private school students in the high-school graduating Class of 2009 who were performing at the advanced level in mathematics with rates of similar performance among their peers around the world (see “Teaching Math to the Talented,” features, Winter 2011). The current study continues this work by reporting proficiency rates in both mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of 2011.

Comparing U.S. Students with Peers in Other Countries

If the NAEP exams are the nation’s report card, the world’s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world’s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries.

Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students and that of their peers elsewhere. But to compare the percentages of students deemed proficient in math or reading, one must ascertain the PISA equivalent of the NAEP standard of proficiency. To obtain that information, we perform a crosswalk between NAEP and PISA. The crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of 2011. Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar for further details.)

What It Means to Be Proficient

According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level “was the result of a comprehensive national process [which took into account]…what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria.” In other words, NAEP’s concept of proficiency is not based on any objective criterion, but reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP says that 8th graders, if proficient, “understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions.”

PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above proficiency  level three on the PISA. (See sidebar for a statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.)

Crossing the Proficiency Bar

Given that definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Performance levels among the countries ranked 23rd to 31st are not significantly different from that of the U.S. in a statistical sense, yet 22 countries do significantly outperform the United States in the share of students reaching the proficiency level in math. Six countries plus Shanghai and Hong Kong had majorities of students performing at least at the proficiency level, while the United States had less than one-third. For example, 58 percent of Korean students and 56 percent of Finnish students performed at or above a proficient level. Other countries in which a majority—or near majority—of students performed at or above the proficiency level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of the individual states within the United States.

Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent.

Only four additional states—Vermont, North Dakota, New Jersey, and Kansas—have a math proficiency rate above 40 percent. Some of the country’s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent).

Click image to enlarge

Proficiency in Reading

According to NAEP, students proficient in reading “should be able to make and support inferences about a text, connect parts of a text, and analyze text features.” According to PISA, students at level four, a level of performance set very close to the NAEP proficiency level, should be “capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text.” (See sidebar for more specific definitions and sample questions.)

Click image to enlarge

The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore,  New Zealand, and Japan (42 percent), Canada (41 percent), Australia (38 percent), and Belgium (37 percent).

Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficiency level in reading. Shanghai students perform at a higher level, however, with 56 percent of its young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and Montana come next, both with 39 percent of the students identified as proficient in reading. The District of Columbia, the nation’s worst, are at the level achieved in Turkey and Bulgaria, while the one-eighth of our students living in California are similar to those in Slovakia and Spain. (See Figure 2 for the international ranking of all states.)

Click image to enlarge

Ethnic Groups

The percentage proficient in the United States varies considerably among students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to students in Belgium, Canada, and Japan.

In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified.

Given the disparate performances among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are due to the presence of a substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community.

While the 42 percent math efficiency rate for U.S. white students is considerably higher than that of African American and Hispanic students, they are still surpassed by all students in 16 other countries. White students in the United States trail well behind all students in Korea, Japan, Finland, Germany, Belgium, and Canada.

White students in Massachusetts outperform their peers in other states; 58 percent are at or above the math proficiency level. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state’s white students are proficient in math, a rate no better than the nation as a whole.  (Results for all states are presented in the unabridged version of the paper.)

In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. Its proficiency rate does not differ significantly (in a statistical sense) from that for all students in Canada, Japan, and New Zealand, but white students trail in reading by a significant margin all students in Shanghai, Korea, Finland, Hong Kong, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado.

Are the Proficiency Standards the Same for Math as for Reading?

Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high?

At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading.

But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the U.S. trails Korea by 16 percentage points, but it’s only 10 percentage points behind Canada. Meanwhile, U.S. performance in math significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 18 percentage points higher. Judged by international standards, U.S. 8th graders are clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects.

A direct comparison of NAEP’s proficiency standard with PISA’s proficiency levels three and four also indicates that a lower NAEP bar has been set in math than in reading. To meet NAEP’s standards currently, one needs to perform near the fourth level on PISA’s reading exam, but only modestly above the third level on its math exam.

Clearly, the experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation’s culture. Those who are setting the common core standards under discussion might well take note of this.

Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading.

What Does It Mean?

Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, “If America is to maintain our high standard of living, we must continue to innovate…. Math and science are the engines of innovation. With these engines we can lead the world.”

But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. That is the wrong message to take away however. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country’s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad.

Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time.

As we proceed into the 21st century, none of these factors remains as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average.

Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation’s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth (see “Education and Economic Growth,” research, Spring 2008). Assuming past economic patterns continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since current average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent.

When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong.

Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, president of the National Academy of Engineering and president emeritus at Massachusetts Institute of Technology, has warned, “America faces many challenges…but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency.”

Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution. Ludger Woessmann is professor of economics at the University of Munich. Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Carlos X. Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University. An unabridged version of this paper is available here.

The legitimacy of test score increases in District of Columbia Public Schools (DCPS),  in particular those at Crosby S. Noyes Education Campus, are the focus of the latest installment in USA Today’s “Testing the System,” a multi-part series exploring the extent and causes of cheating — by teachers, principals and schools — on standardized tests. At the heart of the allegations are multiple erasures — presumably adults correcting test answer sheets — that were detected by the test scanning computers that grade the multiple choice tests. In one classroom at Noyes, USA Today reports, seventh-graders averaged 12.7 wrong-to-right erasures per student. Perhaps the students were lucky, but statisticians say they would have been more likely to pick winning lottery ticket numbers than to make that many wrong-to-right erasures by chance.

If the USA Today allegations are true, then the adults who changed students’ answers did much more than just cheat on a test. They also cheated those students, by allowing them — and their families — to think that they had learned material they clearly hadn’t.

But these deceptions are not new. For decades, less was expected from students attending schools in poor communities. Expectations were even lower for children with disabilities or in special education programs. It was virtually impossible to learn how these children were doing. Because scores were reported only as averages, it was easy to mask the fact that entire groups of students were not learning. Educators and policymakers knew–but the absence of hard data made it easy to turn a blind eye.

The federal No Child Left Behind Act (NCLB), which mandated yearly testing and public reporting of schools’ results in grades 3-8 and once in high school, was written in part to respond to these issues. Lawmakers wanted to ensure that test results would be comparable from student to student and create common standards for all students, regardless of their backgrounds. That’s why the law requires that tests align with state content standards and that students be assessed at their official grade level. It’s also why the law requires that schools report results for smaller groups of students–those who don’t speak English, for example, or those with disabilities–separately.

So, before we heed reactionary calls to end standardized testing, it’s important to remember the decades of willful neglect prior to NCLB. Most notably, the implementation of standards-based reform, first in the states in the 1990s and then by the federal government under NCLB in 2002, spurred an unprecedented focus on the deficiencies of schools that serve poor and minority students–students who were long ignored and whose outcomes were mostly hidden from view.

Still, the law is almost a decade old and its flaws are increasingly clear. Few people would defend the quality of most state tests and the low bar that they set to proclaim a student “proficient.” Part of the solution is a better assessment system. The Obama Administration agrees and is investing $350 million in two different consortia of states to develop these assessments. Ideally, improved assessment practices will show us not only what students are learning but also how they are learning and why they may or may not be gaining particular skills or knowledge. We also need to continue explorations of data of all types (not just test scores), building on, for example, important research that’s helping us develop early warning indicators to prevent students from dropping out.

We need voices and ideas from many places to continue to improve our understanding of how well students are learning in our public schools. But there’s no excuse for cheating, whether it’s done by children or adults. And let’s remember that the system before NCLB also cheated children, denying them a chance to get a good education. Ignoring the past and taking us back to those days–however you score it–is definitely a wrong answer.

-Bill Tucker

(Post cross-posted from the Huffington Post.)

Fischel’s basic argument is that our educational institutions have largely evolved in response to consumer demands.  That is, the consolidation of one-room schoolhouses into larger districts, the development of schools with separate grades, the September to June calendar, and the relatively common curriculum across the country all came into being because families wanted those measures.  And in a highly mobile society, even more than a century ago, people often preferred to move to areas with schools that had these desired features.  In the competitive market between communities, school districts had to cater to this consumer demand.  All of this resulted in a remarkable amount of standardization and uniformity across the country on basic features of K-12 education.

Hearing Fischel’s argument made me think about how ill-conceived the nationalization effort led by Gates, Fordham, the AFT, and the US Department of Education really is.  Most of the important elements of American education are already standardized.  No central government authority had to tell school districts to divide their schools into grades or start in the Fall and end in the Spring. Even details of the curriculum, like teaching long division in 4th grade or Romeo and Juliet in 9th grade, are remarkably consistent from place to place without the national government ordering schools to do so.

Schools arrived at these arrangements through a gradual process of market competition and adaptation.  Parents didn’t want to move from one district to another only to discover that their children would be repeating what they had already been taught or were  inadequately prepared for what was going to be taught.  To attract mobile families, districts informally and naturally began to coordinate what they taught in each grade.  Of course, not everything is synced, but the items that are most important to consumers often are.

That’s how standardization in market settings works and we have a lot of positive experience with this in industry.  VHS became the standard medium for home entertainment because the market gravitated to it, not because some government authority mandated it.  If we followed the logic of Gates-Fordham-AFT-USDOE we would want some government-backed committee to decide on the best format and provide government subsidies only to those companies that complied.

Instead of ending up with VHS, they may well have imposed Betamax on the country, even though market competition would have shown that approach to be inferior.  Sony was the industry leader and if a government-backed committee were in charge they almost certainly would have had the most influence.  The Fordham folks might want to keep this in mind.  A government-backed committee is almost certain to prefer what the AFT wants over what Fordham may envision since the teacher unions are like Sony except only 100 times more powerful.

Even worse, once government-enforced standardization occurs it becomes extremely difficult to change.  If we had a government-backed panel decide on Betamax, we may have been stuck with that format for decades.  We almost certainly would have stifled the innovation that led to DVDs and now Blue-Ray.  Once Sony had entrenched their format, what incentive would they have had to change it?

Similarly, once the Gates-Fordham-AFT-USDOE coalition settles on the details of nationalizing standards, curriculum, and testing, it will become extremely difficult to change anything about education.  Terry Moe and Paul Peterson’s dreams of technology-based instruction may never leave the dream stage because it may fail to comply with certain provisions of the national regime.  If I were the AFT, I’d almost certainly insert those details into the regime to prevent the reductions that technology may bring to the need for teaching labor.  No one should be naive enough to think the Edublob won’t figure out how to use nationalization to block that and other threatening innovations.

I’m also sure that Bill Gates would have preferred being able to get a government-backed committee to enshrine Microsoft-DOS or Windows forever.  But thanks to market competition we have Google innovating with cloud computing.  And I’d bet that Google would love to get government backing for their approach if they could.  Dominant companies almost always favor government regulation.

So I understand why the AFT, USDOE, and Gates favor the current effort to nationalize education.  The mystery to me is why Fordham is protecting the right-flank of this movement or why some conservative governors have gone along.  Don’t they realize that it will enshrine arrangements that favor the teacher unions and are bad for kids?

-Jay P. Greene

In fact, the long report (it is nearly 4,000 words) about how the New York State education department struggled with, and ultimately flubbed, the testing challenge is less a narrative of education failure than a cautionary tale about relying on big government (and the attendant bureaucracy) to run our schools. (See USA Today today:  apparently, six in ten Americans say the government has too much power, and nearly half agree with this alarming statement: “The federal government poses an immediate threat to the rights and freedom of ordinary citizens.”)

“State long ignored red flags on test scores” is the headline in today’s Times.

[E]vidence had been mounting for some time [that the state’s tests] had serious flaws. The fast rise and even faster fall of New York’s passing rates resulted from the effect of policies, decisions and missed red flags that stretched back more than 10 years and were laid out in correspondence and in interviews with city and state education officials, administrators and testing experts.

Paul Peterson and Rick Hess raised red flags long ago, with their research comparing the assessment systems of the states. (See Johnny Can Read…in Some States.) And then there was the classic, to me, front page Times story, from 2002:  The Elderly Man and the Sea? Test Sanitizes Literary Texts, telling the story of the Brooklyn mother who caught the New York State Education Department “cleaning up” (i.e. changing) the texts of famous writers for the state’s ELA tests.

What is amazing to me is that no one goes to jail for this kind of thing. We have become so numb to the news of school failure—and so blinded by the blame-the-victim excuses—that we greet stories like Medina’s with an institutional yawn instead of an arrest warrant.

Part of the problem is that these debates are like TV wrestling: it has some entertainment value.  We can easily get caught up in the testing narrative and miss the essential question: what do we want our children to know? In fact, Medina’s story mentions “curriculum” just once, and then only in an oblique reference to “the citywide math and English curriculums” in New York City.

And that is the second doleful takeaway from this depressing story: that we have handed over our schools to test-makers. Please don’t misunderstand. Tests—and assessments—are critical to learning, but they are the thermometer not the weather. The jobs of educators these days seem to be to trick kids and teachers rather than provide a measure of knowledge.

“In many cases you could not write an unpredictable question no matter how hard you tried,” the Times quotes Daniel Koretz, who “specializes in assessments systems,” according to the Times and who “oversaw the study of New York’s tests that led to the state’s conclusion that they had become too easy to pass.”

We are supposed to have “unpredictable” questions?

This is a system very much lost in the weeds.

There is some hope in the Common Core State Standards Initiative, but until we put the horse in front of the cart and have rigorous, detailed, and broad curricula for our schools and districts, we will continue with a largely useless argument about who sits where in the carriage.

The data used to determine the grades in Figure 1 are available here.

Much ado has been made about setting high standards over the past year. In his first major address on education policy, given just two months after he took the oath of office, President Barack Obama put the issue on the national agenda. They ought “to stop lowballing expectations for our kids,” he said, adding that “the solution to low test scores is not lowering standards—it’s tougher, clearer standards.” In March 2010, Secretary of Education Arne Duncan accused educators of having “lowered the bar” so they could meet the requirements set by the federal education law, No Child Left Behind (NCLB), which requires that all students be proficient in reading and math by the year 2014.

Current conversations about creating a common national standard largely focus on the substantive curriculum to be taught at various grade levels. Even more important, we submit, is each state’s expectations for student performance with respect to the curriculum, as expressed through its proficiency standard. Curricula can be perfectly designed, but if the proficiency bar is set very low, little is accomplished by setting the content standards in the first place.

To see whether states are setting proficiency bars in such a way that they are “lowballing expectations” and have “lowered the bar” for students in 4th- and 8th-grade reading and math, Education Next has used information from the recently released 2009 National Assessment of Educational Progress (NAEP) to evaluate empirically the proficiency standards each state has established. This report is the fourth in a series in which we periodically assess the rigor of these standards (see “Johnny Can Read…in Some States,” features, Summer 2005; “Keeping an Eye on State Standards,” features, Summer 2006; and “Few States Set World-Class Standards,” check the facts, Summer 2008).

The 2009 NAEP tests in reading and math were given to a representative sample of students in 4th- and 8th-grade in each state. NAEP, called “the nation’s report card,” is managed by the Department of Education’s National Center for Education Statistics and is currently the “gold standard” of assessments. Its proficiency standard is roughly equivalent to the international standard established by those industrialized nations that are members of the Organisation for Economic Co-operation and Development (OECD). If a state identifies no higher a percentage of students as being proficient on its own tests than NAEP does, then the state can be said to have set its standards at a world-class level. To ascertain objectively whether state standards are high or low, and whether they are rising or falling, we compare the percentage of students deemed proficient by each state with the percentage proficient as measured by NAEP. The state assessment data used in this report consist of those compiled in 2009 by the 50 states and the District of Columbia.

States have strong incentives not to set world-class standards. If they do, more of their schools will be identified as failing under NCLB rules, and states will then be required to take corrective actions to bring students’ performance up to the higher standard. As a result, the temptation for states to “lowball expectations” is substantial. Perhaps for this reason, a sharp disparity between NAEP standards and the standards in most states has been identified in all of our previous reports. In 2009, the situation improved in reading, but deteriorated further in math.

Every state, for both reading and math (with the exception of Massachusetts for math), deems more students “proficient” on its own assessments than NAEP does. The average difference is a startling 37 percentage points. In Figure 1, we provide a uniform ranking of the rigor of state standards using the same A to F scale used to grade students (see sidebar for the specifics on the methodology we used).

Racing to the Top?

Ironically, Tennessee received an F and had the lowest standards of all states, despite the fact that it is one of the two winners in the first phase of the bitterly contested Race to the Top (RttT) competition sponsored by the Obama administration’s Department of Education. Indeed, Tennessee has had the lowest standards of all states since 2003. Based on its own tests and standards, the state claimed in 2009 that over 90 percent of its 4th-grade students were proficient in math, whereas NAEP tests revealed that only 28 percent were performing at a proficient level. Results in 4th-grade reading and at the 8th-grade level are much the same. With such divergence, the concept of “standard” has lost all meaning. It’s as if a yardstick can be 36 inches long in most of the world, but 3 inches long in Tennessee.

Delaware, the other RttT First Phase winner, also had below-average standards, for which we awarded a grade of C- and ranked it 36th of the 50 states and the District of Columbia. Delaware claimed that 77 percent of its 4th-grade students were proficient in math, when NAEP shows that only 36 percent were. In 8th-grade reading, Delaware said 81 percent of its students were proficient, but NAEP put the figure at 31 percent.

From these findings one might conclude that the Obama administration is having a huge policy impact by getting states like Tennessee and Delaware to set standards they have been unwilling to establish in the past. But Tennessee earned almost full marks (98 percent) on the section of the competition (weighted a substantial 14 percent of all possible points) devoted to “adopting standards and assessments,” even though its standards have remained extremely low ever since the federal accountability law took hold. The proof will be in the pudding. If Tennessee and Delaware and other states now shift their standards dramatically upward, RttT will win over those who think it is performance, rather than promises, that should be rewarded.

Disparities in State Standards

Despite the incentive to lowball expectations, five states—Hawaii, Massachusetts, Missouri, New Mexico, and Washington—have set their standards at or close to the world-class level, earning them an A. Notice that we award grades purely for the expected standard for performance, not actual proficiency. New Mexico earned the same mark as Massachusetts, even though only about one-quarter of its students are proficient, while half of Massachusetts students score at that level. The two deserve equal grades, however, because both are rigorous in their expectations. Another eight states—Colorado, Maine, Minnesota, Montana, New Hampshire, New Jersey, Rhode Island, and Vermont—earned a B for their standards.

President Obama is undoubtedly correct, however, in suggesting that many states are “lowballing expectations.” Of the remaining 38 states, 27 earned a C, and 8—Arizona, Idaho, Illinois, Maryland, Michigan, New York, Texas, and Virginia—a D. Three states—Alabama, Nebraska, and Tennessee—had such low standards that we awarded them an F. All of the states that earned grades of F have been ranked D or below in all three of our previous reports. This suggests that once a standard, however low, has been set, it tends to persist—another reason to be concerned about promises from Delaware and Tennessee.

Changes in Standards

Secretary of Education Duncan is not altogether correct in suggesting that educators are lowering the bar, however. Figure 2 shows that in 2009 the differences between state and NAEP standards shrank by 0.08 standard deviations as compared to the average for the three prior surveys. This is a reversal of the trend of declining standards we observed between 2003 and 2007. Eight states improved the overall rigor of their assessments by a full letter grade or more since 2007: Georgia, Mississippi, Montana, New Jersey, New Mexico, Oklahoma, Utah, and West Virginia. By contrast, we gave just four states—Alaska, California, New York, and South Carolina—grades that were at least a full letter grade worse than they received in 2007.

The reversal in the overall trend is, however, driven wholly by an improvement in the rigor of reading assessments, which set expectations that are higher by 0.49 standard deviations in 4th grade and by 0.26 standard deviations in 8th grade. As a matter of fact, 17 states increased the rigor of their 4th-grade reading assessments by a whole letter grade since 2007, and 17 states did the same for 8th grade. But math standards have slipped by 0.12 standard deviations in 4th grade and by 0.31 in 8th grade. This means that at least some of the state-reported improvements in mathematics proficiency are misleading.

Converging on a De Facto National Standard?

Most changes to standards, as we noted, have been fairly small: only 12 states have made changes to their standards that alter their standing by a whole letter grade. But since our last report two states, Hawaii and South Carolina, have made major alterations to state assessments. The results of these moves have been at odds: while Hawaii’s increased alignment with NAEP raised its grade from a B+ in 2007 to an A, South Carolina dropped from an A to a C-.

States nonetheless seem to be continuing their trajectory of convergence toward standards of similar rigor in math (which, given the slipping standards noted above, constitutes a downward convergence), but are more divergent in reading since 2007, particularly in 4th grade. If the convergence of math standards were to continue, we could gradually attain something like a national standard. But it would take a great deal of national patience to achieve a national standard by convergence creep.

In this report, as in previous ones, we assess the rigor of standards that states set. This is an important task, as it reminds states that whether students have or have not learned cannot be a matter of how the test is designed and where the “proficiency line” is drawn. Rather, setting high standards for proficiency is the first step in the journey toward actually improving the learning of a high percentage of students. According to NAEP, less than one-third of students are proficient in reading and a similar proportion in math nationwide. For the sake of the children of this country, we should be doing much better than that.

Paul E. Peterson is professor of government at Harvard University, senior fellow at the Hoover Institution, and editor-in-chief at Education Next. Carlos Xabel Lastra-Anadón is a research fellow at the Program on Education Policy and Governance at Harvard University.

Everyone knows the early drafts have been the object of much discord. How confident can we be that what will emerge from these tussles and dust-ups will be coherent, complete and sufficiently demanding without being overwrought? If this national standards endeavor were a new drug for fighting swine flu or breast cancer, the FDA would subject it to rigorous long-term “field trials” to determine both its safety and its efficacy before releasing it for widespread use. Yet the Education Department, the White House, the Gates Foundation, the National Center on Education and the Economy and plenty of other parties are sounding and acting as if these standards and assessments had already proven themselves. The high command at Gates seems to assume that all of American K-12 education is going to be reconfigured around them. Secretary Duncan asserts that only states pledging their troth to the Common Core State Standards Initiative (CCSSI) should be eligible for Race to the Top funding. Yesterday, the President declared that future Title I funding for a state should hinge on whether it has embraced the new standards and assessments. And more.

A little humility would seem to us to be in order. If these standards and assessments end up representing a huge improvement over those in use in most states today, then much that’s good may reasonably follow from their installation and use. But what if they don’t? And even if they do, what about those (few) states that have done a creditable job on their own and for which CCSSI may represent either a lateral move or a step backward? In any case, would it not be prudent to appraise their safety and efficacy before demanding that they become the center of America’s new education universe?

“CRTC scandal stuns the state”

“Cheating details revealed”

“Atlanta board calls for cheating probe”

“CRTC” stands for “criterion-referenced competency tests,” and they are administered to students in grades 3-8 to gauge learning.  The problem is signaled in the first few paragraphs of one of the stories:

“One a late June day two years ago, two DeKalb County school administrators panicked.  A few dozen of their elementary school students had just finished high-stakes summer retests–exams first taken in spring but not passed.  With just a glance at the answer sheets, Atherton Elementary Principal James Berry and Assistant Principal Doretha Alexander saw they were in trouble.

“We cannot not make AYP,’ Alexander said. Not making AYP, or adequate yearly progress, meant not meeting a required federal benchmark.  These students, all fifth-graders, also faced being held back if they did not pass.

“‘Okay,’ Barry answered.  He pulled a pencil from a cup on Alexander’s desk.  ’I want you to call the answers to me.’  With that, he began to erase the students’ answers.”

That’s one scene, and it occurred a while ago.  But official investigations have enlarged the problem, and the general picture is this: fully 191 schools in the state of Georgia, 10 percent of the total number of elementary and middle schools, are up for investigation for altering test answer sheets, the story reported.

The next day’s story put the count at one in five Georgia public schools.  More than half of those schools had at least one classroom that displayed abnormal numbers of wrong answers changed to right answers.  In one elementary school classroom, 4th-grade math tests showed an average of 27 answers changed from wrong to right (out of 70 total answers).  In one middle school in Atlanta, nearly 90 percent of classrooms came up suspicious.

The extent of the scandal remains to be seen, and its impact is long-term.  What happens to kids whose tests were flagged, but who might have done much of the answer-changing themselves in the course of taking them?  What about the fate of the governor’s proposal to tie teacher pay to student performance (in other words, is this too strong an incentive to cheating)?  Who is going to investigate all the individual cases and measure out relative culpability and punishments?

This is going to take awhile.

The warm greetings given by both Republicans and Democrats on the committee reflect Duncan’s reputation as a centrist in the ideologically fraught battles over education reform. He has received national attention for moves favored by reformers, such as opening 75 new schools operated by outside groups and staffed by non-union teachers; introducing a pay-for-performance plan that will eventually be in 40 Chicago schools; and working with organizations, including The New Teacher Project, Teach For America, and New Leaders for New Schools, that recruit talented educators through alternatives to the traditional education-school route.

At the same time, Duncan maintained at least a cordial working relationship with the Chicago Teachers Union, and both the National Education Association (NEA) and the American Federation of Teachers (AFT) backed his nomination. He supported the No Child Left Behind law (NCLB), but also called for dramatic increases in spending to help schools meet the law’s targets, and additional flexibility for districts like his own. In nominating Duncan, Obama said, “We share a deep pragmatism about how to go about this. If pay-for-performance works and we can work with teachers so it doesn’t feel like it’s being imposed upon them…then that’s something that we should explore. If charter schools work, try that. You know, let’s not be clouded by ideology when it comes to figuring out what helps our kids.”

Given the strong union support for the Obama presidency, there was great speculation within education circles throughout the fall as to whether the new president would turn out to be a reformer—willing to challenge existing practices and the teachers unions in order to achieve dramatic changes in schools—or play it politically safe by backing programs that brought only marginal changes. A sharp divide among Democrats was in full view at the party’s national convention in Denver, where urban mayors and educators, gathered at a forum sponsored by Democrats for Education Reform (DFER), challenged the dominant role of teachers unions in shaping policy. Newark mayor Cory Booker told those assembled, “We have to understand that as Democrats we have been wrong on education, and it’s time to get it right.”

Even before the national convention, conflicts between the unions and Democratic reformers were intensifying. At a New York fundraiser in 2007, Obama reportedly made a similar point. According to Joe Williams, DFER’s executive director, Obama incriminated the teachers unions when the director of a Harlem charter school asked the then candidate why Democrats threw up so many obstacles.

Williams explained, “We’re at this point where the nation wants to change education more than the unions and the unions are going to have to decide if they’re going to be part of the change or be left out of it entirely.”

Two manifestoes issued during the Democratic primaries laid out competing philosophies on improving student achievement that were intended to influence the eventual Democratic nominee. A “Broader, Bolder Approach to Education,” a letter issued by the liberal Economic Policy Institute, signed by national leaders across much of the political spectrum, and endorsed by the AFT, argued that improving schools alone would not close achievement gaps between disadvantaged and advantaged students. It called on policymakers to provide preschool, afterschool programs, and summer school, and take steps to improve students’ health and social development. Another letter, issued by a coalition called the Education Equality Project, advocated addressing school system failures through greater accountability, school choice, and changes in compensation that would promote teacher quality. Those who signed on to the project, a diverse group of leaders in education, philanthropy, and public service, vowed to “challenge politicians, public officials, educators, union leaders and anybody else who stands in the way of necessary change.”

Obama has allies in both camps. Arne Duncan was one of only a handful who signed both statements. Yet in his confirmation hearing, Duncan left little doubt that the administration wants to make systemic changes.

“We must do dramatically better,” Duncan told the Senate committee. “We must continue to innovate. We must build upon what works. We must stop doing what doesn’t work. And we have to continue to challenge the status quo.”

Advisors to Obama say the rhetorical distinction was overdrawn and that the thrust of the president’s strategy is to make progress without causing further polarization. His education platform reflected that approach. Like many Democrats, he wants to spend more money: on helping students attend college; early childhood care and education; and improving teaching through mentoring and professional development for both principals and teachers. He has criticized NCLB for encouraging teaching solely focused on preparing students to pass tests. But in line with many Republicans and more conservative Democrats, Obama, like Duncan, supports school choice, charter schools, performance-based pay, and alternatives to education schools for teacher preparation (see sidebar). He and his opponent, Senator John McCain, both praised the work of Washington, D.C. schools chancellor Michelle Rhee, who has fought the local union as well as the AFT over tenure and teacher pay.

Economic Stimulus

Widespread agreement that only a massive stimulus package could rescue the U.S. economy presented the new administration with the opportunity to placate both sides of the Democratic divide. The unions and their allies would get a massive infusion of federal funds into the schools that would help offset state and local budget cuts. And this would give Obama cover to push for tougher reforms down the road.

House Democrats, after negotiations with Obama’s team, in mid-January proposed a stimulus package of $825 billion that included between $120 billion and $140 billion for public schools and colleges. Most of the money would have few strings attached.

The spending package would boost federal spending on Title I programs for low-income students and for special education, distributing the money according to current formulas. It would also provide at least $39 billion to offset state cuts in education budgets and $20 billion for capital improvements at schools and colleges. About $15 billion would be available to states as bonuses for efforts such as ensuring that low-performing schools and districts have effective teachers and that the performance of English-language learners and special education students is properly assessed (see Figure 1). One Obama aide said similar incentives would be incorporated into education programs to be introduced later in the spring.

The stimulus package also proposed to boost funding for the Teacher Incentive Fund (TIF), a Bush-era program that provides financial incentives to teachers and principals who raise overall student achievement and close achievement gaps. After Democrats took control of Congress in 2006, they zeroed out funding for TIF but restored $100 million for the following year. In his last budget, Bush requested $200 million for the program, the same amount Obama’s team has proposed.

Thirty-six states plus the District of Columbia already have local or statewide teacher compensation systems that add some sort of financial incentive to the standard step-and-column pay plan, according to the NEA. Former NEA president Reg Weaver cautioned that “while we can be open to alternatives, we should always oppose politically motivated, quick fixes designed to weaken the voice of teachers and the effectiveness of education employees. If they want to talk about changing the way we’re paid, they need to do that with us, not to us.”

In Obama’s platform, he agreed that such plans should be developed in consultation with teachers. Among the promising models is a voluntary pay-for-performance program in place in districts in a dozen states, funded in part by TIF, and implemented by Duncan in Chicago. The Teacher Advancement Program (TAP) provides teachers with professional support, helps them to use data in instruction, holds them accountable for results, and provides bonuses. Teachers in 10 Chicago schools voted to participate in TAP starting in the fall of 2007, and bonuses totaling $340,000 were given out the following year for improved test scores at 9 of the schools. “This is a landmark event for Chicago’s schools—recognizing and rewarding educators for exemplary work and compensating them accordingly,” Duncan said at the time.

The scale of the proposed spending on education is stunning, more than doubling the federal contribution. Of course, even an increase of that magnitude would leave the feds as the junior investors in public education, their contribution dwarfed by current state and local spending. But the funds proposed to offset cuts in state funding would mean that, for the first time, the federal government would be directly covering the cost of basic school operations. That kind of money could buy a lot of goodwill, especially if it helps states avoid laying off thousands of teachers. By December 2008, 19 states had cut K–12 education spending, according to the Center on Budget and Policy Priorities, a liberal research group. Even with the infusion of federal support proposed so far, states may have to make further cuts in their education budgets if the economy does not improve quickly. States spend between one-third and one-half of their budgets on elementary and secondary education, and the revenue available to state and local governments is shrinking fast. By January 2008, states had reported deficits of $350 billion. “If the economy doesn’t get better, schools are in trouble,” said Jack Jennings, founder and CEO of the Center on Education Policy. “For the sake of the schools it’s important that Obama pay attention to the economy.”

Even if the economy recovers and the stimulus package goes through intact, some observers question whether the proposed spending will do enough to address persistent disparities in achievement. Despite past federal support directed toward the needs of low-income students, African American 4th and 8th graders did not make measurable progress on the National Assessment of Educational Progress between 2005 and 2007. “Is the stimulus going to benefit kids in ways that are palpable and real and that improve achievement?” asked Dianne Piche of the Citizens Commission on Civil Rights. As the House was passing its version of the stimulus package (see Figure 1), Frederick Hess of the American Enterprise Institute noted that most of the money simply gave states dollars to keep intact the programs of the past: “It’s like an alcoholic at the end of the night when the bars close and the solution is to open the bar for another hour,” he told a New York Times reporter.

No Child Left Behind

The pressing economic issues, as well as difficult politics, will likely push reauthorization of NCLB into 2010 or even 2011. California Democrat Representative George Miller, who was one of four members of Congress who worked with the first Bush administration on the original NCLB, wants to see it revised and reauthorized. Yet Miller acknowledged to the Washington Post that “at the end of the day, it may be the most tainted brand in America.”

NCLB has been a great success in the sense that no one disagrees with its goals: accountability for results, addressing issues of teacher quality, putting a spotlight on the learning of all students, and better targeting of funds to districts serving the most disadvantaged students. Still, its detractors argue that the law has had unfortunate side effects: too much time spent teaching to narrow tests, schools focused on boosting the scores of students who are just below the proficiency threshold, and some states lowering their standards to reduce the number of schools missing their achievement targets.

“We’ve learned over the past five to 10 years that we have to align curriculum, align standards, and align tests with professional development,” Jennings said. “We’ve also learned that it is very, very hard to do. We’ve also learned that if we really set certain goals…teachers will pay attention to those students who are just below the goal and not pay attention to those who are further down or further up.”

Obama spoke during his campaign at length about the ins and outs of testing and decried teaching to the test. Rather than abandon the testing in NCLB, he has said he wants to invest in improving assessments, so that they measure a broader range of skills than just the basics.

The battle fought over reauthorization of NCLB in 2007 offers a preview of the challenges the Obama team will face. In a speech at the National Press Club outlining his priorities for reauthorizing the law, Representative Miller said, “Throughout our schools and communities, the American people have a very strong sense that the No Child Left Behind Act is not fair, not flexible, and is not funded. And they are not wrong.”

Hope for Reform
Despite the challenges, many in Washington are hopeful that public schools may in fact improve under an Obama administration. Although he cannot ignore the unions that form a key part of the party’s constituency, Obama owes less to them than did past Democratic presidents. The unions did not support him in the primaries and, because he raised so much money on his own, Obama was not as dependent on their money as others have been. Of course, he is hugely popular with teachers, and the staggering amount of money he appears to be willing to spend on education will only make him more so.

In addition, the leaders of the two unions at least appear more willing to be flexible on some long-standing issues. AFT president Randi Weingarten has said several times that “nothing is off the table” except vouchers. Not that much is known about Dennis Van Roekel, the Arizona math teacher who became president of the NEA last summer (see “Same Old, Same Old,” features, Winter 2009). But he was among those who supported Bob Chase, an earlier NEA president, when he tried to get the union to endorse what he called the “new unionism.” Chase wanted the union to experiment with new forms of performance pay and peer review of teacher performance, but the rank-and-file members nationally were reluctant to go along. It remains unclear how far Weingarten and Van Roekel will be able to push their members now to accept changes in compensation, evaluation, tenure, and so on.

Weingarten finds it “very sad” and frustrating that unions are always blamed for opposing reforms. “There’s a lot of demonizing and blame-mongering going on in education and it’s ridiculous…because it just creates excuses,” she said. “It says to me that they don’t think anything can be done because they are looking for the fall guy rather than helping all kids achieve.”

Weingarten expressed hope that Obama would push for more rigorous standards, better curricula, more valid assessments, and investments in helping teachers improve. “You can’t buy it by putting money out there and saying to teachers, ‘if you don’t do it, you’re fired,’” she said, referring to her opposition to Michelle Rhee in Washington, D.C. “We have the responsibility…to recruit and support and retain teachers if they’re doing a good job, and if not, to counsel them out of the profession.”

But Kate Walsh, president of the National Council on Teacher Quality, counters that the unions have resisted that course of action. “I think the unions are up against the wall,” she said. “The whole movement toward the notion that teachers don’t have a basic right to be in the classroom unless they are effective is proving so powerful as an idea that they’re weakened because they’ve run away from it rather than embrace it.”

It is well known that one of the strongest threads in the narrative of Obama’s journey from his childhood to the White House is educational opportunity (see “The Early Education of Our Next President,” features, Fall 2008). Schooled first in Indonesia, he returned to Hawaii because his mother wanted him to get a better education. There, his maternal grandmother and grandfather enrolled him in the private Punahou School, where he studied with the island’s elite. Then, it was on to Occidental College, Columbia University, and Harvard Law School.

“I wasn’t born with a lot of advantages, but I was given love and support, and an education that put me on a pathway to success,” Obama said during a major campaign speech on education last September in Dayton, Ohio. “The reason Michelle and I are where we are today is because this country we love gave us the chance at an education. And the reason that I’m running for president is to give every single American that same chance.”

Joe Williams believes that all of those factors, as well as Obama’s personal commitment to improving education, create a real opportunity to bring about systemic, long-lasting changes. “Everyone says they support the goals of NCLB and if that’s real, then he can use his bully pulpit to say that we’ll do in education the equivalent of saying we’ll put a man on the moon in 10 years.

“He can say that we will make sure that every kid who starts the race will cross the finish line and it will give everyone goose bumps and start a new type of discussion about what the game is. But it only has the potential to change the game if he treats it as an opportunity to wipe the slate clean and inspire people to think very big about what is possible,” adds Williams. “Obama is the only person I’ve seen in the last 20 years who may be up to that job.”

“His vision of education is as a foundation not just of the economy but of a society in which people take care of each other,” explained Stanford University education professor Linda Darling-Hammond, who advised Obama during the campaign and handled education policy for the president-elect’s transition team, in remarks delivered in November 2007 at a National Academy of Education event. “I think we can make great strides in a very short time.”

Although some may worry about the cost of all of the new programs, Darling-Hammond views the amount Obama wants to spend on education as a relatively small part of the overall bailout and recovery package, which could exceed $1.5 trillion.

In his speech last September in Dayton, Obama assured his audience, “We can do it all.”

Richard Lee Colvin is a longtime education journalist and director of the Hechinger Institute on Education and the Media, Teachers College, Columbia University.

The quality of our public schools has been measured in innumerable ways, and stakeholders may draw on any number of sources for rankings to support a particular agenda. Each winter, Education Week issues Quality Counts as a magazine supplement to its weekly newspaper. Report cards track and compare state education policies and outcomes in six areas: chance-for-success; K–12 achievement; standards, assessments, and accountability; transitions and alignment; the teaching profession; and school finance. For example, the grade for transitions and alignment is based on 14 indicators related to “early-childhood education, college readiness, and economy and workforce,”  while the school finance indicators measure spending patterns and resource distribution. Through these report cards, Education Week purports to “offer a comprehensive state-by-state analysis of key indicators of student success.”

The Quality Counts rankings are eagerly anticipated, thoroughly perused, and widely quoted. After the 2009 rankings were released, the Maryland State Department of Education issued a press release touting the state’s place at “the top of the list in Education Week’s tally, just ahead of Massachusetts.” Florida governor Charlie Crist celebrated the news that Education Week’s Quality Counts rated Florida’s schools 10th in the nation, based on its average rating across the six categories that comprise the analysis. Are Florida’s schools among the nation’s best? It depends on what you measure. By November of 2009, two lawsuits had been filed in Florida claiming the state was failing to provide high-quality education to its students. The plaintiffs claimed the state has low graduation rates, frequent school violence, and low levels of education spending and teacher pay compared to other states.

The rankings are also frequently misunderstood. Among the most widely cited of the Quality Counts ranking schemes is the Chance-for-Success Index (CFSI), which attempts to measure a state’s capacity for helping young people succeed. Here’s what Education Week’s Editorial Projects in Education (EPE) Research Center has to say about the index:

The Chance-for-Success Index captures the critical role that education plays at all stages of an individual’s life, with a particular focus on state-to-state differences in opportunities. While early foundations and the returns in the labor market from a quality education are important elements of success, we find that the school years consistently trump those factors. In every state, indicators associated with participation and performance in formal schooling constitute the largest source of points awarded in this category, and help explain much of the disparity between the highest- and lowest-ranked states.

The CFSI’s stated aim is to show the role that education plays as a student moves from childhood through the formal K–12 system and into the workforce, but then the rest of the description is fairly ambiguous. Many states nonetheless interpret the index as a simple measure of school quality. Maryland came in fifth in 2009, with a B+. The Maryland schools’ press release cited above reported that the state “ranked among the nation’s leaders in ‘Chance for Success,’ which looks at how well graduates achieve beyond high school.” Of course, some states choose not to emphasize their CFSI score. For example, the New Mexico education department’s January 2009 press release led with its number-two rank and A grade for transition and alignment policies and buried in the middle its 51st-place CFSI grade of D+.

Does CFSI measure the school system’s contributions to achievement beyond high school? It’s hard to say. Most of its components, described as “key facets of education spanning stages from childhood to adulthood,” are a grab bag of demographic characteristics. The index combines indicators related to family background, wealth, education levels, and employment with schooling measures, including kindergarten enrollment and selected National Assessment of Educational Progress (NAEP) test scores. The 13 components of success are identified in the sidebar. Not all of these have a clear relationship to postsecondary success, and several are beyond the control of state policymakers.

Consider the parental employment indicator and its role in an index that is updated annually or even every other year. Short-run trends in parental employment may not have any impact on the overall quality of a state’s education system; even the direction of possible influence is unclear. Parents who see how difficult it is to get and retain employment without education may stress the value of school completion, but it is also conceivable that underemployed parents may seek to accelerate their children’s entry into the labor force, even at the expense of their education. A similar problem exists with annual income: many factors outside of education quality influence the vitality of a state economy. Even if strong gains in public education are realized, it will be years before the effects are reflected in adults’ annual income. Income trends over the next few years will have little or nothing to do with current levels of education quality.

A Different Approach

Absent a sound theory of action, it is easy to go on a data spree. As seen in the CFSI, the more the merrier. As an experiment, we reconstructed the Chance-for-Success Index. First, we selected a clear standard for our index: we defined “success” as the percentage of young adults, aged 18 to 24, who are productively engaged in postsecondary endeavors (pursuing a college degree, active military service, or full-time employment). We limited the indicators to only those factors for which a reasonable empirical base of evidence shows an association between the indicator and our definition of success and that are plausibly under the control of education policymakers. Five indicators have a clear bearing on education outcomes: preschool enrollment, kindergarten enrollment, 4th-grade reading, 8th-grade mathematics, and high school graduation. Using the same source data as the 2009 CFSI and giving each factor equal weight, we computed new averages for each state and compared the new rankings to the originals.

Our results show marked divergence from the CFSI rankings (see Table 1). Only Maryland (5th) and Arizona (43rd) retained their rankings, although four of the top five stayed within that band. Looking down the list, however, 34 states moved 3 or more places, 21 shifted by 5 or more places, and 13 states moved by 8 or more places. Does our revised index precisely rank states’ public education systems? Probably not. The ideal index would be one that measured how well states and schools did, given their demography. Still, this exercise shows how sensitive the CFSI is to the choice of indicators.

Removing family background characteristics from the index changes states’ rankings substantially. The states that drop the most in the revised rankings are Hawaii, Rhode Island, Indiana, Alaska, Nebraska, and North Dakota. The states that gain the most are Florida, Texas, Maine, Idaho, Arkansas, and Mississippi, mostly poor, rural states.

Is the CFSI largely a measure of parental education? We looked at where the states would fall if we ranked them by individual family background variables. The variable that by itself provides a ranking with the closest fit to the CFSI is percentage of children with at least one parent with a postsecondary degree (parent education). Ranked by that measure alone, only 8 states would move by 8 or more places from their positions in the CFSI. Indicators of family income and adult education levels also produce rankings similar to the CFSI. Ranking states by either the percentage of children in families with incomes at least 200 percent of poverty level (the family income indicator) or the percentage of adults (25–64) with a 2- or 4-year postsecondary degree (adult educational attainment), only 15 states would move 8 or more places.

Measuring the Measure

Report cards must meet a number of conditions if they are to be reliable. First, they need to clearly define the condition or result being examined. None of the descriptions provided by the CFSI editors accomplish this—they never reveal exactly what they take the “chance for success” to be, asserting only that some states provide better opportunities than others. Explained the EPE Research Center’s director, “a child’s life prospects depend greatly on where he or she lives.”

Second, the indicators that are employed should have direct and proven association with the outcome being measured. The CFSI’s current approach mixes inputs such as demographics with outcomes like academic results to arrive at a single score. The result is a tautology: success is the sum of the parts; the parts are by default the components of success.

The editors of Quality Counts gather and report a variety of measures that reflect current education and policy performance across all 50 states and the District of Columbia and, through comparison, encourage states to take actions that would lead to improvements in their ratings. Nowhere do the Quality Counts editors show how or why the Chance-for-Success Index is a good predictor of success. Instead, they provide statistics that divert attention away from the things that actually do matter, such as high-quality teaching, a good range of school options, and success in early elementary schools. There is risk in including variables that have no real impact on the result being studied. States may view the results as motivators to improvement, and ineffective indicators may lead to ineffective attention and investment. Narrowing the scope of the Chance-for-Success Index to factors both causally related to school achievement and under the control of state education officials or school districts would improve its value and deliver the right signals to states.

CREDO at Stanford University supports education organizations and policymakers in using research and program evaluation to assess the performance of education initiatives. The team is led by Margaret Raymond and includes Kenneth Surratt, Devora Davis, Edward Cremata, Emily Peltason, Meghan Cotter Mazzola, Kathleen Dickey, and Rosemary Brock.

A major debate among education reformers over how best to reduce the achievement gap broke out during the 2008 presidential campaign. Most advocates on both sides backed Barack Obama, but they urged him to pursue different policies. The Education Equality Project (EEP) supported a continuation of accountability and other school-focused reforms. The coalition for A Broader, Bolder Approach to Education claimed that the greatest gains could be achieved by addressing health, housing, and other social ills (see “Straddling the Democratic Divide,” features, Spring 2009).

A close look at recent changes in education in the state of Florida sheds light on that debate. One finds in this southern state, a closing of the achievement gap that has eluded allegedly more progressive states. When it comes to education progress, Florida is a star performer. Moreover, its success has come in spite of a challenging student demographic profile and relatively modest resources.

Let us begin with a basic demographic fact often cited by those in the Broader, Bolder camp. Over the past 20 years, the schools of Florida, California, Texas, and New Mexico have all seen rapid growth in their Hispanic populations. Compared to other groups, Hispanic students underperform academically, drop out of school in higher numbers, and attend college in lower numbers. A straight projection of the recent past into the future looks bleak for these students and their educational outcomes.

But demography need not be destiny. Over the past decade, Florida has succeeded in improving student achievement despite its demographic profile. Low-income students (those eligible for free or reduced-price lunch) make up almost half of Florida’s K–12 student body. Florida has a “majority minority” mix of students, with non-Hispanic white students making up 48.3 percent of the total, African Americans about 24 percent, and Hispanics 25 percent. But the educational situation is not as bleak as those statistics might imply: both minority groups have recently made academic strides forward.

Florida has managed to realize such gains although the state’s per-student funding is below the national average. More than making up for its fiscal limitations, the state, led by former governor Jeb Bush, implemented a series of school reforms that together appear to have had dramatic consequences for student performance. Upon taking office in 1999, the governor pursued a multipronged strategy of education reform: an emphasis on reading, standards and accountability for public schools, and new choice options for students. The bulk of the reforms passed in his first year in office. Subsequently, those initial measures were buttressed by additional innovations, including the curtailing of social promotion for students who failed to learn to read in the early elementary grades.

Academic Achievement

Prior to the introduction of those innovations, Florida’s educational record was little short of abysmal. Among the 43 states that in 1998 were gathering information on their students’ performance on the National Assessment of Educational Progress (NAEP), Florida had the fifth-lowest 4th-grade reading scores. But over the next decade those scores moved sharply upward so that by 2007, Florida’s scores were tied for 8th highest among all the states. As the state moved up the leader board, Florida students, on average, were making strikingly larger gains on NAEP exams than the average student nationwide (see Figures 1 and 2). Nor were gains occurring only in reading. Fourth-grade math scores were climbing at an even faster rate. In 8th grade, reading and math gains in Florida were less impressive but they still outpaced the nation.

Those statewide trends could have been masking a widening of the achievement gap between whites and minorities. But exactly the opposite was happening. Far from lagging behind, Florida’s minority students were doing much to drive the overall rise in test-score performance. In the decade after the education reforms began, the average NAEP reading score for Hispanic 4th graders in Florida rose steeply so that by 2007 scores were higher than the average NAEP reading scores of all students (regardless of ethnicity) in 15 states (see Figure 3).

One might think that rising scores among Hispanic students reflect their families’ movement up the income ladder into the middle class. But even Florida’s low-income Hispanic students scored, on average, equal to or higher than nine statewide averages for all students (regardless of income). Average scores of all Florida low-income students, regardless of ethnicity, tied or exceeded the statewide average for all students in seven states. Many of these differences are small and thus within the margin of NAEP sampling error, so should be thought of statistically as ties. The margin of error cuts both ways, however, as the 2007 statewide averages for all students in Georgia, Illinois, Michigan, North Carolina, Rhode Island, and Texas all fell within two points of Florida’s average for Hispanics on the 4th-grade reading exam. Indeed, the national average for all students falls within this same narrow margin.

Comparison of trends in Florida with those in California is particularly intriguing. Both are large states with growing Hispanic populations. California’s median family income is 12 percent higher than Florida’s, meaning families have more resources to devote to their children’s education. But California has largely eschewed the kinds of accountability and choice reforms that Florida adopted.

The consequences for students range between noteworthy and startling, as shown in Figure 4. The chart displays the differences in the NAEP gains 4th- and 8th-grade students made in the two states. As they did in 4th-grade reading, Florida’s Hispanic students outperformed California students in 4th-grade math and 8th-grade reading; they tied the California average in 8th-grade mathematics.

Explaining Florida’s Success

Not everyone thinks that something remarkable has been happening in Florida. The state’s success is only an apparent one, says Boston College education professor Walter Haney. He discounts Florida’s progress on 4th-grade NAEP scores, on the grounds that Florida’s worst-performing readers repeat 3rd grade and thus are not included in the 4th-grade NAEP. Without the low-performing 4th graders who have been held back for a year, the average scores of the remainder jump upward.

Haney’s critique is worth considering. One of the pillars of the Florida accountability reforms has been the policy, introduced in 2003, of not promoting 3rd graders unless they perform at a minimally acceptable level on the reading portion of the Florida Comprehensive Assessment Test (FCAT).

But a more careful look at the data shows that Florida’s NAEP scores were rising before implementation of the retention policy. Between 1992 and 1998, Florida’s average NAEP score in 4th-grade reading dropped by two points. Between 1998 and 2002 (before the social promotion policy affected 4th grade), however, it increased by eight points. One of the reasons for lower retention rates was improved 3rd-grade performance on the state’s examination. In 2002, 27 percent of 3rd graders scored at the lowest level on the reading portion of FCAT, but by 2008 only 16 percent did so, a 40 percent reduction in the pool of students eligible for retention. This helps explain why actual retention rates declined by 40 percent between 2002 and 2007.

One would expect, if Haney’s interpretation is correct, to see an upward spike in 4th-grade test scores in 2003 followed by a steady decline in test performance in subsequent years. But in fact the trend line shows no such spike and decline, only a steady movement upward.

Perhaps Florida’s gains are only apparent for another reason: its low starting point in 1998. Is it possible that gains are realized most easily when scores are initially very low? On this question, opinion is quite divided. Some think gains are more easily realized if students are already accomplished, while others think those with high scores have neared a ceiling, making it difficult to raise their scores further. However that issue is settled in principle, it cannot account for the fact that Florida made striking gains while states with equally low scores did not. For example, on the 1998 4th-grade reading test Florida was near the bottom, with Arizona, California, Hawaii, Mississippi, Louisiana, and New Mexico. In 2007, all those states but Florida were still clustered near the bottom.

But if the Florida achievement gains are genuine, and not imaginary, they might still be attributed to factors over which schools have little or no control, for example, demographic changes in the state. Such is the claim of those who say that demography is destiny. Were demographic change the best explanation, however, student performance in Florida would be worse than ever. According to the National Center for Education Statistics (NCES), 45 percent of Florida children attending public schools in 1998 were of minority background. By 2005, that percentage had climbed to just over 50 percent. Similarly, the percentage from low-income backgrounds (eligible for free or reduced-price lunch) rose from 43 to 45 percent between 1998 and 2007.

Another plausible explanation for the Florida success story is the 2002 passage of an amendment to the state’s constitution mandating universal preschool education for all those who would like to participate. But however valuable the program may prove to be, it cannot explain the gains in achievement observed thus far. The amendment did not require implementation until 2005, and none of the students participating in the Florida early childhood program had reached the 4th grade by 2007, the most recent year for which NAEP data are available.

Much the same can be said for a second constitutional amendment—the class-size reduction amendment—approved by Florida voters in 2002. As in the case of preschool, there is some research evidence that suggests class-size reduction can yield significant gains in student achievement in the early grades. Florida state law now mandates no more than 18 students per classroom in grades K through 3, and no more than 22 students in grades 4 through 8. But the constitutional amendment is being implemented slowly. Through the 2008–09 school year, administrators have been considered in compliance if their schoolwide average class sizes were under the constitutional limits. According to the state department of education, from 2002 to 2008, average class sizes in the early grades were reduced from roughly 23 to 16 students in pre-K to grade 3 and from 24 to 18 students in grades 4 to 8. Still, if class-size reductions had any effect on achievement gains between 1998 and 2007, it could only have been toward the end of the period.

Nor can the gains in education be easily attributed to changes in public school funding. Florida’s average spending per pupil rose from $7,183 in 1998–99 to $7,683 in 2004–05, in constant dollars. This was less than half the increase in the national average over the same period.

One can pretty much rule out these possible explanations for the Florida success story. The gains are not an artifact of the elimination of social promotion in 3rd grade or of the ease with which low test scores can be lifted. Nor can they be attributed to demographic change, the introduction of preschool education or class-size reduction, or greater per pupil expenditure. One must look elsewhere for an explanation. The most likely remaining candidate is school-focused reforms, which have the vigorous support of the EEP side of the education reform debate.

The Florida Reforms

Over the past decade, Florida has introduced a comprehensive program of school reform that has five main points: school accountability, literacy enhancement, student accountability, teacher quality, and school choice. Together, the reforms created a system that appears to have focused teachers and students on the task of learning in a way that has yielded the dividends we have highlighted above.

School Accountability.  In 1999, the state legislature enacted a law that required students in grades 3 through 10 to take annual tests in reading and mathematics, known as the Florida Comprehensive Assessment Test or FCAT. The assessment had two distinctive features lacking in most other accountability systems, including the one prescribed by the federal law, No Child Left Behind (NCLB). First, it gave each school in the state a very clear grade of A to F based on the results from the test and offered a specific fiscal incentive to schools to try to reach as high a grade as possible. Bonuses were given for obtaining an A or raising one’s grade from one year to the next. Conversely, schools receiving an F grade twice over a four-year period were asked to carry out a variety of reforms. The law offered students at “double F” schools the opportunity to attend private schools until a court decision disallowed the practice in 2006.

Beginning in 2002, the accountability system included measures of student progress from one year to the next, a feature not incorporated into NCLB. That gave schools with low-performing students an opportunity to raise their grades without imposing upon them the extremely difficult task of matching the performances of schools whose student body enjoyed a preferred demographic portfolio. Clear, realistic incentives to improve were made available to schools across the state.

Focus on Literacy.  Along with its accountability system, Florida in 2002 introduced a statewide program known as “Just Read, Florida!” The effort created new academies to train teachers in reading instruction and provided for the hiring of 2,000 additional reading coaches. Teachers in grades K–3 took mandatory reading training courses over a three-year period. Students in grades 6 through 12 who demonstrated insufficient reading skills were provided remedial instruction.

Student Accountability. Beginning in 2003, Florida students were asked to pass a more demanding examination if they were to be given a high school diploma. In addition, Florida lawmakers, as discussed above, curtailed the social promotion of 3rd-grade students who performed at very low levels in reading. According to a careful evaluation by Jay Greene and Marcus Winters at the University of Arkansas (see “Getting Ahead by Staying Behind,” research, Spring 2006), the program had a positive impact on the performance of all 3rd graders, including those who were retained in that grade. Apparently, they benefited more from an additional year of instruction than they would have had they been pushed on to 4th grade when they were not well prepared for the more challenging material.

Teacher Recruitment.  Florida enacted new policies for broadening the pool from which teachers were being selected. Previously, teachers were required to earn a certificate by attending one of the state’s schools of education. Florida supplemented that channel of recruitment with a variety of alternative paths. The state opened “Educator Preparation Institutes” to facilitate the transition into teaching. Districts were allowed to offer alternative certification. Today, more than one-third of all new teachers in Florida are coming to the profession through alternative certification programs. The state’s teaching workforce has become the nation’s third most ethnically representative (see “What Happens When States Have Genuine Alternative Certification?” check the facts, Winter 2009).

The alternative certification program may have had a particularly significant impact on Hispanic students. Florida enjoys a large immigrant population that fled from Cuba in the years following the establishment of Castro’s communist regime. Many of the immigrants were middle-class professionals and entrepreneurs, and they have established a strong economic, political, and cultural presence in southern Florida. That population provides a pool of potential educators of high talent who speak both English and Spanish. Just how important alternative certification was to the recruitment of highly qualified bilingual instructors is unknown, but it cannot be ruled out as a potential explanation for the particularly large gains Florida’s Hispanic students have made. On the other hand, it cannot be the whole story, as African American and non-Hispanic white students also made strong gains during this period. Moreover, the percentage of Hispanics of Cuban origin has declined during the past decade (though this may not have affected the size of the pool of qualified bilingual teachers).

School Choice. Florida is well known for the range of school choice legislation it has enacted over the past decade. Charter schools, vouchers, tax credits, and online education all provide students and families with greater choice in 2008 than they had in 1998. For example, 105,329 students were enrolled in the state’s 358 public charter schools in 2007–08. That same year 19,852 students eligible for special education took advantage of the opportunity to use a voucher to attend private schools, and 21,493 students received scholarships averaging $3,750 from a tax credit program that opened private schooling to students from low-income families. The state-funded Florida Virtual School currently offers students more than 90 online courses (ranging from GED to Advanced Placement courses). Middle and high school students anywhere in Florida can access these classes free of charge. The state projects that 168,000 courses will be taken and completed during the 2008–09 school year (see “Florida’s Online Option,” features). Multiple evaluations, by organizations ranging from the Manhattan Institute to the Urban League, have found the choice programs to have had a positive impact on Florida public schools.

Despite the numbers, the school choice programs are not large enough to have had more than a limited statewide impact on the millions of students attending Florida’s public schools. Yet they helped create a climate in which public schools may have wanted to demonstrate their effectiveness for fear that choice opportunities would continue to expand.

Identifying what has caused the rise in Florida student performance cannot be done with perfect certainty. It might have been the accountability system, or the state’s reading program, or its decision to expect more from students, or its alternative certification program, or its plethora of school choice innovations, or some combination of all of them. But the results from Florida do suggest that concerted efforts to improve the quality of an education system can pay dividends for students. It is probably not a coincidence that the one state that has outdone the others in its efforts to reform its schools has made outsize gains in student performance. Exactly which of the many reforms Florida undertook was the key to success may never be known, but the reform package offers other states—and the nation as a whole—a clear path on which they, too, can move forward.

Matthew Ladner is vice president for policy at the Goldwater Institute. Dan Lips is senior policy analyst at the Heritage Foundation.

Cheating by educators comes in many forms, ranging from the subtle coaching of students to the overt manipulation of test results. For instance, a colleague of mine tells of a principal who would begin each morning’s announcements with a greeting to students, such as, “Good morning students, and salutations! Do you know what a salutation is? It means ‘greeting,’ like the greeting you see at the beginning of a letter.” Students learned the meaning of words like “salutation” from the principal’s daily announcements; they probably never learned that his choice of words like “salutation” was done with the vocabulary section of the state-mandated, norm-referenced test in hand. A recent U.S. News & World Report article described a case in Ohio, where one educator is accused of physically moving a student’s pencil-holding hand to the correct answer on a multiple-choice question. Another widely reported case involved a principal in Potomac, Maryland, who stepped down amidst charges that she had gone through students’ test booklets in the classroom and called them up to change or elaborate on their answers.

This isn’t just a recent phenomenon. More than a decade ago, at an evening reception following a conference for school district superintendents in one midwestern state, I happened upon a conversation among several superintendents who, with cocktails in hand, were chuckling and winking about how their quality-control procedures for student testing involved “pre-screening the kids’ answer sheets for stray marks.” What was so funny—I found out later from one of the superintendents—was that stray marks included things like wrong answers. Wink, wink. This practice apparently continues. In Texas, where the accountability system is particularly rigorous, 11 school districts were investigated for an unusually high number of erasures on students’ answer sheets in 1999.

Despite their headline-grabbing nature, such blatant cases of cheating are probably rare. There is usually more subtlety involved, such as when a teacher prods a student to review his or her answer: “Why don’t you take another look at what you wrote down for number 17?” Some examiners cheat by failing to monitor their students properly while proctoring an exam. Others cheat by omission, such as when a teacher reminds students who are likely to attain low scores that it would be okay for them to be absent on the day of the test. A more sophisticated version of cheating by omission occurred in the Austin, Texas, school district in 1999. School administrators entered incorrect student-identification numbers on the answer sheets of low-scoring students, which invalidated their scores and thus raised the schools’ average performance. In states that force schools to give absent students a score of zero for reporting purposes (which eliminates the incentive to encourage them to stay home), all students are encouraged to attend school on test day. But some students are afforded “testing disability accommodations” such as an individual aid, reader, extra time, or other assistance that isn’t usually a part of the student’s educational experience.

Perhaps the largest cheating scandal to date involved teachers and principals in the New York City school district. In December 1999, Edward Stancik, the district’s special commissioner of investigation, released an exhaustive study that found that cheating by 12 educators was “so egregious that their employment must be terminated and they should be barred from future work with the [Board of Education].” For instance, one teacher would have her students first write their answers on a piece of scrap paper. She would then correct their answers before they bubbled in their official answer sheets. The report named another 40 educators who were recommended for disciplinary action; 35 of them had engaged in actions that the investigators judged serious enough to warrant potential termination. The report concluded that the school district had known about the extensive cheating by educators “for years,” and that “educators were not held fully liable for their misconduct.” A follow-up report named another ten educators who had engaged in seriously inappropriate behaviors during testing in New York City, some of them so blatant—such as writing answers to test questions on the chalk board—that immediate termination of employment was recommended.

Some opponents of testing see cheating by educators as a reason to abandon high-stakes accountability systems. For instance, last year Alfie Kohn, a prominent critic of testing, told Congressional Quarterly, “The real cheating going on in education reform is by those who are cheating students out of an education by turning schools into giant test-prep centers.” These critics view cheating as the natural, and not so reprehensible, result of placing undue emphasis on a metric—standardized tests—that yields an incomplete picture of both student and teacher performance. Some even view cheating as a kind of civil disobedience. But, to collect useful information about educational progress, testing is an indispensable if imperfect tool. And if tests are to yield useful information, their validity must be ensured. The answer to cheating is not to abandon accountability. The answer is to limit if not eliminate the cheating. What follows is a survey of what we know about the extent of cheating and some proposals to guard against the subversion of accountability systems.

Testing Guidelines

Cheating can be defined as any action that violates the rules for administering a test. Commercial test publishers produce carefully scripted directions and clear guidelines for administering their tests. The guidelines lay out, in detail, all the actions that would compromise the test results. Similar instructions and rules accompany the customized tests that form the bedrock of many state accountability systems. Some states have strongly worded professional codes of ethics that explicitly define the responsibilities and boundaries associated with mandated testing. What constitutes cheating may even be codified in state law. For instance, the Ohio Revised Code proscribes “any practice that results solely in raising scores or performance levels on a specific assessment instrument without simultaneously increasing the student’s achievement level as measured by tasks and/or instruments designed to assess the same content domain.” The law provides for the termination of an offender’s employment, the suspension of an educator’s license for a violation, and the charging of an offender with a misdemeanor.

National organizations representing various professional associations have also developed standards for educators who administer standardized tests. For example, the American Federation of Teachers and the National Education Association helped to produce the Standards for Teacher Competence in Educational Assessment of Students, which require that teachers “should be skilled in recognizing unethical, illegal, or otherwise inappropriate assessment methods and uses of assessment information.” The most explicit statements regarding cheating can be found in the Standards for Educational and Psychological Testing, a product of collaboration among professional organizations in the fields of education and psychology. Among other guidelines, the standards explain that those involved in educational testing programs must “ensure that test preparation activities and materials provided to students will not adversely affect the validity of test score inferences” and “maintain the integrity of test results by eliminating practices designed to raise test scores without improving students’ real knowledge, skills, or abilities in the area tested.” In short, educators cannot plead ignorance; there has been no dissemination problem regarding what constitutes cheating. Professional codes of ethics that cover virtually every profession whose members work in school settings and state laws that govern those who have a license or credential in the field of education contain strict guidelines for administering tests. Anyone who is connected with testing in American education knows—or should know—how to conduct assessments that yield accurate and credible results.

The Consequences of Cheating

When cheating occurs, testing yields inaccurate information about individual students. The error is compounded when this information is then used for any educational purpose, and specific students wind up paying the price. One student may not receive the remedial instruction in reading that she needs. Another student may be incorrectly assigned to a special program for gifted and talented students that has a limited number of slots. Another may receive a scholarship that should have gone to one of his peers. And yet another may receive a diploma without having learned those minimum skills deemed necessary for success in college or on the job.

From the perspective of educational policy-making, the same invalidities that yield misleading test scores at the individual level also serve to muddle the interpretation of group test performance. Policy-makers and educational administrators have increasingly come to rely on group data to inform their decisions on staffing, curricula, professional development, and teacher-credentialing requirements and to measure the effectiveness of educational reforms. By distorting test results, cheating can lead to ill-advised initiatives, improperly focused resources, and inaccurate conclusions about the course of education reform. Given the confluence of achievement gains—in states ranging from Texas to North Carolina to New York—with the pervasive reports of cheating by educators, it is entirely reasonable to question how much of the former can be attributed to the latter.

Though not attempted in this article or elsewhere to my knowledge, the costs of cheating probably could be measured in dollars and cents. What cannot be measured are the effects of cheating at more fundamental levels. For example, when students learn that their teachers or principals cheat, what is the effect of this kind of role modeling? While fallen professional athletes might be able to say, “Don’t look at me as a role model, I am just an athlete doing a job,” educators cannot: a significant aspect of their job is the modeling of appropriate social and ethical behavior. Also, how might educator cheating affect students’ attitudes toward tests or their motivation to excel? How might it affect their attitudes toward education, their trust or cynicism with respect to other institutions, or their propensity to cheat in other contexts?

Research on Cheating

Shocking anecdotes don’t tell us much about how serious the cheating problem actually is. Just how prevalent is cheating by educators? Only a few studies have directly asked educators whether they have engaged in what have come to be referred to euphemistically as “inappropriate test administration practices.” The most common avenue of research is to poll educators regarding their general perceptions of cheating in their schools. One such study asked 3rd, 6th, 8th, and 10th grade teachers in North Carolina to report how frequently they had witnessed certain inappropriate practices. Of those polled, 35 percent said they had engaged personally in such practices or were aware of others’ unethical actions. The teachers reported that their colleagues engaged in a range of inappropriate practices two to ten times more frequently than they had. The practices included giving extra time on timed tests, changing students’ answers, suggesting answers to students, and directly teaching specific portions of a test. More flagrant examples included teachers’ giving their students dictionaries and thesauruses for use on a state-mandated writing test. One teacher revealed that she checked students’ answer sheets “to be sure that her students answered as they had been taught.” Other teachers reported using more subtle strategies, such as “a nod of approval, a smile, and calling attention to a given answer,” to enhance their students’ performance. In another study, of teachers who were drawn from two large school districts, 32 percent of the teachers surveyed reported allowing students to practice on old forms of standardized tests for two or more weeks.

A total of 40 schools were included in a study that was initiated in order to investigate suspected cheating in the Chicago public schools. Of the 40 schools, 17 served as “control” schools, which were compared with 23 “suspect” schools that had exhibited irregularities in the performances of their 7th and 8th grade students on the Iowa Tests of Basic Skills (ITBS). The irregularities consisted of unusual patterns of score increases in previous years, unnecessarily large orders of blank answer sheets for the test, and high percentages of erasures on students’ answer sheets. The researchers readministered the ITBS under more controlled conditions and found that, even after accounting for students’ reduced level of motivation on the retesting, the “suspect” schools clearly did worse than the “control” schools on the retest. The researchers concluded that they might have underestimated how much cheating was going on at some schools. A study of cheating in the Memphis school district revealed extensive cheating on the California Achievement Test, including a teacher who displayed correctly filled-in answer sheets on the walls of her classroom.

The Perceptions of Educators

The most troubling stream of research on cheating concerns the attitudes of educators toward cheating. They seem increasingly indifferent toward cheating, and there appears to be a growing sense that cheating is a justifiable response to externally mandated tests.

Several studies have attempted to investigate educators’ perceptions of cheating. In a 1992 study, 74 pre-service teachers were asked to judge the appropriateness of certain behaviors. Only 1 percent thought that either changing answers on a student’s answer sheet or giving hints or clues during testing was appropriate, and only 3 percent agreed with the idea that allowing more time than allotted for a test was acceptable. But 8 percent thought that practicing on actual test items was okay; 23 percent judged the rephrasing or rewording of questions acceptable; and 38 percent thought that practice on an alternative test form was appropriate.

The beliefs of pre-service teachers appear to translate into actual practices when they enter the classroom. In 1991, a large sample of 3rd, 5th, and 6th grade teachers in two school districts was asked to describe the extent to which they believed specific cheating behaviors were practiced by teachers in their schools. On the positive side, a majority of respondents (shown in Figure 1) said that, for all of the behaviors listed but one, they occurred rarely or never. Equally noticeable, however, is that in several cases, 15 percent or more of the respondents reported that a behavior occurred “frequently” or “often.” A full 23 percent of the teachers said that they thought teachers often gave hints to children who were having difficulty. Twenty percent said they thought teachers “frequently” or “often” gave students extra time to finish their tests. Likewise, 20 percent said they thought teachers “frequently” or “often” gave students practice on passages that were highly similar to those used on the test.

A 1991 survey examined perceptions about two specific kinds of “test preparation” practices: having students practice for a state-mandated, norm-referenced test using another form of the same test or having students practice on the actual test to be used. The survey polled six groups of educators, including teachers, principals, superintendents, and school board members in California. The results, shown in Figure 2, reveal fairly broad acceptance of these behaviors, even among board members. For instance, 36 percent of teachers in California thought it appropriate to practice with current test forms.

What Can Be Done?

What can be done to address the problem of cheating? At some point, we will need to reconceptualize testing entirely. We must find more effective ways to link, consistently and directly, successful test performance to student effort and effective instruction. If poor performance were accompanied by sufficient diagnostic information about a student’s weaknesses, then all concerned might view identification and remediation of those weaknesses as more beneficial then cheating. Such an initiative will require changing much of the status quo in curriculum, instruction, and assessment. But there are less far-reaching, more pragmatic actions that can be taken immediately. The following list provides a start.

Dissemination. It has been said that we more often stand in need of being reminded than we do of education. As mentioned earlier, every large-scale testing program provides a description of appropriate test-administration procedures; state regulations define the boundaries of legal conduct for test administrators; and education-related associations have produced guidelines for sound testing practice. Nonetheless, those caught cheating often protest that they did not know the behavior was wrong. If only as a reminder, every implementation of high-stakes tests should be accompanied by dissemination of clear guidelines regarding appropriate testing practices. Such reminders should be clearly worded, pilot-tested to refine the meaning that educators take from the guidelines, and distributed and signed by all who handle testing materials.

Procedures. Some minor procedural changes would hinder the ability of educators to cheat. For example, bar-coding or other methods of identifying testing materials plus a system of tracking testing materials would be easy to implement. Federal Express and United Parcel Service know the location of every package at any given time and can reconstruct precisely the hands that a package has passed through. The same tracking can be used for testing materials. Other simple steps would include the sealing of cartons and bundles of testing materials; delaying the delivery of testing materials to schools until just before test administration; and, once delivered, requiring that materials be maintained securely by a named person who is responsible for them.

“Truth in testing.” States with so-called truth-in-testing laws should reconsider their relative benefits. These laws often require that the content of state-mandated tests be disclosed following the administration of a test. They have the best of intentions, but the unforeseen consequence of such laws has been an increase in educators’ use of previous versions of tests for classroom practice, resulting in further narrowing of instruction. Moreover, the economic costs to “truth in testing” states have been staggering. Disclosing each year’s tests renders them useless, making it necessary to develop entirely new monitoring instruments one or more times each year.

Scaling Back. The expansion of testing and accountability systems has elicited two reactionary responses to the concurrent rise in cheating: 1) that large-scale testing for accountability be abandoned; or 2) that testing for accountability rely more heavily on constructed-response formats that, ostensibly, would be less prone to corruption. For instance, it is more difficult to forge or coach a student’s answer to an essay question or a science experiment than to alter a bubbled-in response or to provide the key to a multiple-choice item.

The difficulty with these reactions is that they fail to address the core issues. High-stakes pupil testing arose in the 1970s in reaction to the complaints of some business leaders—along the lines of, “We are getting high school graduates who have a diploma, but can’t read or write!” As UCLA Professor of Education James Popham observed at the time: “Minimum competency testing programs . . . have been installed in so many states as a way of halting what is perceived as a continuing devaluation of the high school diploma.” The public perception was that the gatekeepers were leaving the gates wide open. Perhaps a widespread misunderstanding of the relationship between self-esteem and achievement was to blame. Educators understandably wanted all students to have the personal esteem associated with high achievement. But awarding higher grades in order to boost self-esteem and stimulate further achievement too often had neither effect. The sense that grades weren’t accurate measures of achievement led to the imposition of externally developed and administered tests.

Thus the obvious error in calls to return to the past is that such a strategy only returns American education to the situation that caused accountability tests to be introduced in the first place. Moreover, though current tests are susceptible to cheating, the solution of returning to measures and procedures that are even more easily manipulated is unthinkable.

Nevertheless, we should consider limiting the amount of testing for accountability. Some tests serve solely an instructional function; they serve our need for solid diagnostic information on students in order to make informed decisions about their educational programs. When these tests are used also for accountability purposes, the expanded incentives to cheat can corrupt the information we have on student progress. Likewise, not all tests—especially those designed for purposes of decision-making—need have instructional value. If we clarify the purpose of each test, we can minimize the scope of mandated accountability tests, the time required for their administration, and the opportunities for cheating.

Consequences. In conjunction with limiting opportunities for cheating, we must revise the procedures that are used to unearth cheating and the penalties that are handed down. Many tests are currently administered behind closed classroom doors with little independent oversight; there are strong disincentives for educational personnel to report cheating; and, in most jurisdictions, the responsibility for investigating cheating rests with school personnel who have an inherent conflict of interest in ferreting out inappropriately high student achievement. Revised procedures should include: 1) random sampling and oversight of test sites; 2) increased protections for whistle-blowers; 3) stiffer penalties for cheaters, including permanent disqualification from teaching within a state and more coordinated sharing among the states of information regarding educators who have had their licenses revoked; and 4) assignment of investigative responsibilities to an independent authority.

A Qualified Boon

As we are learning, accountability is not an unqualified boon to American education. Nascent accountability systems have been difficult to implement and have had some undesirable consequences. For instance, in some situations the use of tests as the primary accountability mechanism has resulted in an extreme narrowing of what students are taught. When so much is at stake, educators tend to limit their instruction to the content that is covered on a mandated test. Moreover, some educators perceive the imposition of an externally mandated test as an inappropriate intrusion into an area of professional practice and discretion. They believe that their knowledge of a student’s true ability far exceeds whatever information can be gleaned from a single test; thus the idea that a single test should not be used against any student (for example, to deny grade-to-grade promotion or a high school diploma) is widespread among teachers. Here the same moral reasoning that many current teachers learned in their education courses during the 1970s may come into play: the ends do justify the means; cheating is a justified response to a system that punishes students and teachers on the basis of incomplete information.

But cheating itself leads to inaccurate information and misguided decisions about students. It signals that students have learned the skills we want them to, when in fact they haven’t. It threatens the values that we hope to impart to students via those we have charged with their education. It leads to mistaken conclusions about the efficacy and pace of needed educational reforms. Even if cheating is limited to a minority of educators, as it most likely is, its effects are devastating. It is no more justifiable than telling a sick patient that he is well and then sending him on his way.

–Gregory J. Cizek is an associate professor of educational measurement and evaluation at the University of North Carolina at Chapel Hill and the author of Cheating on Tests: How to Do It, Detect It, and Prevent It (1999).

More than seven years ago, President George W. Bush signed No Child Left Behind (NCLB) into law. Sweeping calls for testing, intervening in persistently low-performing schools, and policing teacher quality made it the most ambitious legislation on K–12 schooling in American history. The law, due for congressional reauthorization in 2007, still awaits legislative action. This spring, the Hoover Institution’s Koret Task Force issued 10 recommendations to guide reauthorization. In this forum, lead author of Learning from No Child Left Behind, EdisonLearning’s John Chubb, and education historian and task force member Diane Ravitch, who declined to sign the recommendations, weigh in on the future of the law.

EDUCATION NEXT: Is NCLB working? Should it be reauthorized?

Diane Ravitch: It is time to pull the plug on No Child Left Behind. It has had adequate time to prove itself. It has failed. After seven years of trying, there is no reason to believe that the results of NCLB will get dramatically better. Now is the time for fundamental rethinking of the federal role in education.

NCLB has produced meager gains in achievement. The National Assessment of Educational Progress (NAEP) assesses student achievement in reading and mathematics every other year. Despite the intense concentration on reading and mathematics required by the law, the gains registered on NAEP since the enactment of NCLB have been unimpressive.

In 4th-grade reading, the gains after implementation of NCLB, from 2003 to 2007, were small (three points) and exactly the same as the gains from1998 to 2003. Fourth graders in the bottom10th percentile of performance had a five-point gain after NCLB, but this did not compare to the 10-point jump in their scores from 2000 to 2002 pre-NCLB (see Figure 1).

In 8th-grade reading, there were essentially no gains from 1998 to 2007. Student performance was a flat line both before and after NCLB.

Mathematics was tested in 1996, 2000, 2003, 2005, and 2007. The gains preceding the adoption of NCLB were larger than those posted after NCLB. From 2000 to 2003, 4 th grade students recorded a nine-point gain in mathematics, compared to a gain of only five points from2003 to 2007. Among 4th-grade students in the lowest decile, there was an astonishing 13-point gain from 2000 to 2003 pre-NCLB; the same group saw a gain of only five points from 2003 to 2007. The same deceleration of student improvement was seen at all performance levels, from top to bottom.

In 8th-grade mathematics, gains also slowed after the passage of NCLB. Eighth graders saw a five-point gain from 2000 to 2003, but only a three-point gain from 2003 to 2007.

John Chubb: NCLB will and should be reauthorized. Absolutely, student achievement has grown much more rapidly in the last decade—the NCLB era—than during the 1990s, especially for the lowest achieving and most-disadvantaged students in the nation. Achievement is what NCLB is all about, so the law has met its most basic test. This is recognized by even the law’s critics which is why the only discussion in Washington is how to mend the law. The Obama administration recognizes that No Child Left Behind aims to help the federal government perform its most important education function: improving the education of students in greatest need. The new president is supported in this view by a bipartisan majority in Congress, which has worked for many years to ensure that poor kids get the help they require. The education needs that NCLB addresses are not going away, nor is the need for funding. Indeed, the economic stimulus bill passed in February increased funding for NCLB by 80 percent, and these provisions of the massive and controversial bill met no objections.

Over half of poor and minority students have reading and math skills far below grade level, whether measured by the tough performance standards of the NAEP or by the standards of the various states. Dropout rates, measured accurately only since NCLB made them part of Title I accountability, hover around 50 percent in many major cities.

NCLB is based on sound principles and should with time improve the achievement of all American children, especially economically disadvantaged and racial minorities. There is empirical evidence these principles are working. The Thomas B. Fordham Foundation recently completed an analysis of the top and bottom 10 percent of all students tested by NAEP. It found that the bottom 10 percent had gained far more than the national average since 2000 in math and reading, more than a full grade level in math. The top 10 percent had gained as well, providing no evidence that schools were ignoring the best students while focusing on the kids below proficient and subject to NCLB sanctions. Both groups of students had also gained more since 2000 than they had during the 1990s. The federal government’s own comprehensive analysis of Title I, mandated by Congress, conducted by RAND among others, and published in 2007 after several years of NCLB experience, found the largest academic gains since 2000 and 2003 among students in high-poverty schools. To be clear, the evidence in total is early, and the research is incomplete. But there is no question that American kids, especially the most disadvantaged, are making progress. It is absolutely mistaken to suggest, that NAEP changes pre- and post-2003 are evidence that NCLB has been counterproductive. Disadvantaged kids are achieving far more today than ever before, and those gains are attributable to higher standards and tougher accountability that began in the states in the 1990s and accelerated with NCLB.

EN: What are the strengths and weaknesses of the NCLB approach to assessment?

DR: Educators and the public are getting wise to the uselessness of the testing regime that has been foisted upon them. A year ago, North Carolina’s Blue Ribbon Commission on Testing and Accountability issued a report recommending a sharp reduction in the number of tests that the state required. The chairman of the commission, Sam Houston, said, “We’re testing more but we’re not seeing the results. We’re not seeing graduation rates increasing. We’re not seeing remediation rates decreasing. Somewhere along the way testing isn’t aligning with excellence.”

NCLB may in reality be dumbing down our children by focusing the attention of teachers and administrators solely on basic skills. Our students are not being prepared to compete with students from high-performing nations in the world. Many are not getting an education based on a coherent, content-rich curriculum in history, geography, the arts, science, foreign languages, and literature. They are not getting a good education. They are getting thin gruel. If we want a future workforce that is smart, creative, independent, and resourceful, we are not educating to get what we want.

JC: Perhaps the single greatest virtue of NCLB’s approach to assessment and accountability is that it shines a bright light on student performance, as measured against explicit standards of proficiency. The nation finally knows which schools are raising proficiency in reading and math and which are not. Before NCLB, such information was spotty at best. A weakness, however, is that the bright light does not shine on all subjects that matter for kids and their future.

The education the nation values is one that is rich in content. NCLB has unwittingly and unfortunately encouraged schools to focus instruction inordinately on reading and math, the subjects that NCLB requires be tested annually and to which it has attached the tough accountability regime. Students, however, need also to understand science, history, geography, civics, and more if they are to succeed in a 21st-century world of intense international competition and technological sophistication.

NCLB already requires science testing once each in grades 3–5, 6–9, and 10–12. This requirement should be extended to include three tests of social science, defined as U.S. history, world history, geography, and civics. The law should further specify that the science and social science assessments be cumulative and comprehensive, and not focused just on the content taught during the tested grade level. NCLB should require that scores be posted on state and district web sites and included in school report cards. State scores should be benchmarked against NAEP, to encourage high standards. But science and social science should not be part of Adequate Yearly Progress (AYP); the process of assessing and exposing performance should be ample to promote attention to these fields.

EN: How should proficiency be defined and measured?

DR: The federal demand that all students will be proficient by 2014 has led states to embrace a very loose definition of proficiency. Most states are now using NAEP’s “basic” achievement level as their definition of proficiency because NAEP’s “proficient” level is far beyond their reach. But many states go even lower than NAEP basic for their definition of proficiency. Tennessee, for example, says that 90 percent of its 4th-grade students are proficient in reading, while NAEP says that only 26 percent are. Only 61 percent of students in Tennessee are at basic or above, according to NAEP. Similarly, North Carolina tells the public that 86 percent of its 4th graders are reading proficiently, but NAEP says only 28 percent are (and 36 percent score “below basic”). These states and many others make inflated claims to satisfy NCLB’s ridiculous requirements.

JC: There is much room for improvement in how proficiency is defined and measured by NCLB—and we have practical suggestions for improving both. But the fundamental principles that NCLB advances represent a huge step forward for the nation. NCLB asks the nation to define what all students should know and be able to do in reading and math, and then measures progress toward these performance standards. This is a boldly democratic and egalitarian expectation and the very first time that the nation has asked its schools to perform at an explicit level. We should proudly defend these principles.

On a practical level, “proficiency” should describe the knowledge and skills necessary to be “college and career ready” in the 21 st century. Proficiency should capture the “common core” of competencies deemed necessary for all students to have a chance at success after high school.

NCLB should authorize the U.S. Department of Education to fund—after a competitive bidding process—up to three multistate consortia to develop standards, tests, and performance levels that support the overarching goal of college and career readiness. With federal funding, states will buy into one of the systems of national standards and tests, saving the huge expense of developing new standards alone. NCLB could, through these recommendations, give the nation standards both achievable and worth achieving, while preserving the rights of the states to determine what “national” standards should be.

EN: Are the law’s “remedy” provisions—including public school choice and supplemental educational services—working?

DR: The remedies the law prescribes—choice and tutoring—have proven to be ineffective. Less than 5 percent (and by some estimates, as low as 1 percent) of eligible students choose to leave their “failing” school to transfer to a school that made AYP. Some say it is because the students and families did not get adequate notice, but more likely students are not choosing to leave for other reasons. In many suburban and rural school districts, there may be no other school to transfer to. But perhaps more important, most students will not leave their school even if there is another school that is presumably better, by NCLB’s definition, and that is accessible. That is because most students are not in the group that is failing to make progress, and if they like their school, they don’t want to be separated from their friends.

The law assumes that the schools are bubbling over with discontented kids who are eager to escape, but that assumption is probably wrong. Or at least there is no evidence for it based on the lack of response to the choice provisions of NCLB. We have long known from polling data that the public is concerned about the quality of American education, but most parents are satisfied with their own children’s school. The failure of choice in NCLB reminds us of that consistent finding.

The other remedy in NCLB for failure to make AYP is tutoring, and that too has proved to be ineffective, though it has turned into a half-billion-dollar bonanza for tutoring companies. Evaluations in several states, including Maryland, Virginia, Tennessee, Alabama, Georgia, Michigan, and Kentucky, have reported that students who received tutoring did no better on state tests than their peers who did not receive tutoring. Only about 15 percent of eligible students have signed up for tutoring. Even when tutoring is free, conducted after school, and provided in a convenient location (sometimes in their own school building), most students don’t want it. Maybe it conflicts with their afterschool jobs or their sports or other commitments. Maybe they just don’t want to study for an additional hour or two when the school day is done. We need to know more about why 85 percent of eligible students avoid tutoring. We need to know why most eligible students are not showing up to be tutored, and why those who do show up are gaining so little from it.

JC: We know from ample research that choice can boost the achievement of students who avail themselves of it. We also know that tutoring is an effective means of remediating achievement deficits. RAND recently affirmed the effectiveness of SES tutoring in a well-controlled study. But choice and tutoring are not working nearly as well as they could in NCLB. This has nothing to do with the ideas of choice and tutoring but rather with the way NCLB provides for them.

Students in failing schools simply do not have enough choices. The law currently limits choice to schools not in improvement status, which often eliminates all nearby options. NCLB should increase the choices available by permitting families to judge school shortcomings for themselves. A school failing a single subgroup or barely missing AYP, for example, might be a better choice for a student in a school that is failing badly. Yet today those choices are not available.

NCLB should offer additional charter school start-up grants in any school district where failure is rampant, such as a district not making AYP. Students should be able to choose schools in neighboring school districts, subject to district approval. And private schools should be eligible to receive choice students, provided those schools charge no extra tuition and participate in the state testing program.

Students in failing schools should also have greater access to tutoring, sooner. There is no more effective way to help students who are struggling than to get them extra, focused, individualized attention. Yet only 20 percent of students eligible for tutoring under NCLB are receiving services, and the services often fall short of the quality offered in the private marketplace. This should be remedied.

First, make Supplemental Educational Services (SES) available as soon as schools are declared in need of improvement, the same time as school choice is offered. Second, ensure that students have access to the best possible tutors. Grant districts the right to provide SES, even if the district is failing to make AYP, but also require districts to provide a fair and competitive marketplace for all providers. Whatever access the district itself has to families, students, and facilities, it must also provide to private tutors—or the district loses the right to be a provider. To reinforce these measures, NCLB should require states to provide information on eligible students to approved providers. The states should be required to collect and post comparative information on the effectiveness of all tutors.

EN: Are NCLB’s sanctions for persistently failing schools effective? Are they fair?

DR: The law’s punitive sanctions are ineffective. By year six of failing, the schools may be turned into charter schools, taken over by the state or private management, closed, or restructured (e.g., replacing the entire staff). None of these sanctions had a research basis to justify its inclusion in the law. They were hopes or hunches, based on ideology, not evidence. Most states and districts choose the least onerous of the sanctions, which is restructuring. According to a 2008 report from the Center on Education Policy, restructuring itself needs to be restructured because there is no sure-fire way to turn around a chronically low-performing school. The federal Institute of Education Sciences recently published a research summary on how to achieve this admirable goal, but not one of its four recommended strategies was supported by evidence.

JC: Currently, NCLB’s escalating sanctions apply identically to schools that have failed massively and to schools that barely miss. This is a big mistake—but one that is easily fixed. NCLB should differentiate school improvement needs. Over time we expect more and more schools to succeed with the majority of their students, but to struggle with certain extra-needy subgroups. It is vital, as the nation expects increasing percentages of students to achieve proficiency, that we identify schools accurately for their performance. The Department of Education has approved nine states’ requests to implement “differentiated accountability” plans. NCLB should build on this good work and institutionalize a simpler system for all states.

Schools should be placed into one of two categories of “needs improvement.” “Limited” improvement is for schools whose shortcomings involve less than one-third of the student body. Limited improvement would offer students in year one of their school’s acquiring improvement status (two years of missing AYP) choice of another school and SES. If schools remain in limited improvement status, NCLB would require, in year four of improvement, that states develop with schools “limited corrective action plans.” Schools with limited improvement status should not face restructuring; states should have the flexibility to work with schools with limited problems as they see fit.

“Schoolwide” improvement is for schools that miss new AYP growth targets for all students or for subgroups that total more than one-third of school enrollment. Schoolwide improvement would require schools to proceed through restructuring, but NLCB should be revised to include only three means to restructure: First, a school may be reorganized as a charter school, giving it new governance. Second, the school’s management can be contracted out to an independent school management company, changing day-to-day control of the school. Finally, a school may be closed and reopened with 100 percent of the teaching staff and administration replaced. Each of these measures ensures a new day for the school and its students.

EN: Is NCLB’s goal of universal proficiency by 2014 one that should remain in a reauthorization of the law?

DR: The demand that all students be proficient by 2014 is absurd. This laudable goal has never been reached by any other nation or by any state. The only way it can be met is by defining “proficiency” to mean minimal literacy and numeracy. Meanwhile, the expectation that all schools will achieve this goal has created a trajectory of failure that guarantees a steady increase in the number of schools that are stigmatized for not making adequate yearly progress. In the 2007–08 school year, nearly 30,000 schools—or 35 percent of all public schools—joined that abysmal list; this was a 28 percent increase in the number of “failing” schools over the previous year. In Massachusetts, which has the highest-scoring students on NAEP in the United States, nearly half the public schools in the state were rated as being “in need of improvement.”

It does not take a statistician to figure out that NCLB is a recipe for disaster for American public education. An article in Science magazine last fall predicted that nearly 100 percent of all elementary schools in California would be failing schools by 2014.

JC: Universal proficiency is perhaps the most important principle of NCLB—certainly the most audaciously democratic one. It should and will be preserved. Who, after all, will be willing to say whose children should be proficient and whose should not? And, this is not just a matter of principle—the goal is doable.

But the states need to come together around standards that are worth accomplishing, that represent the common core of knowledge and skills that every child needs to be prepared in the 21st century for college or a productive career. Students with special needs or just beginning to learn English need to be provided alternative means to demonstrate proficiency. Universal proficiency in practice may mean 90 to 95 percent proficient, a high number but not an unattainable one. Finally, schools must be given time to realize goals worth achieving.

NCLB should extend the 2014 deadline for universal proficiency by six years—half the original NCLB timeline—to 2020, but only for states willing to adopt new high national standards.

EN: Has the federal leadership embodied in NCLB been a help or a hindrance to school improvement?

DR: Washington does not have the institutional knowledge or capacity to reform our nation’s schools. Congress is not the right institution to reform the nation’s schools. The U.S. Department of Education lacks the capacity to tell the nation’s schools what they should do to improve. Washington is too remote from schools to take responsibility for improving them. In their edited volume, No Remedy Left Behind, Chester E. Finn Jr. and Frederick Hess wrote that NCLB “amounts to a civil rights manifesto dressed up as an accountability system. This provides an untenable basis for serious reform, rather as if Congress declared that every last molecule of water or air pollution would vanish by 2014, or that all American cities would be crime-free by that date…. NCLB’s dogmatic aspirations and fractured design are producing a compliance-driven regimen that recreates the very pathologies it was intended to solve. It’s time to relearn the lessons of the Great Society, when ambitious programs designed to promote justice and opportunity were undone by utopian formulations, unworkable implementation structures, and a stubborn unwillingness to acknowledge the limits of federal action in the American system. In the end, Washington is not well-positioned to effect change to a program that depends on state and local action, or successfully to require states and districts to adopt measures whose efficacy hinges on gusto and creativity rather than compliance.”

A few tweaks here and a little tinkering there cannot fix this fundamentally flawed legislation. The time has come to discard it altogether and begin to think afresh about how the federal government can provide useful assistance to states, districts, and schools that are trying to improve. What we need is a clear recognition of the appropriate federal role in education and a deeper understanding of the meaning of a good education. Perhaps with a sense of the limits of federalism and of the limitless potential of education, we might be able to free ourselves from the sterility, rigidity, dogmatism, and narrow anti-intellectualism of NCLB.

JC: NCLB embodies a delicate balance between federal leadership and state execution. Despite the hue and cry from critics about federal over-reaching, NCLB provides ample discretion to the states. The role that NCLB sets out for the federal government—setting national goals while leaving states and districts to decide how to reach them—is sound, and surely superior to the hodge-podge of state accountability systems that preceded it. The challenge now is to improve how our federal-state partnership works. Experience can be a powerful guide.

Let’s face facts. The nation needs to boost its achievement even more now than when the law was passed. Our economic welfare depends more and more on education. We should learn from the law—as it is beginning to help our children learn—and not expect 50 uncoordinated states to get the nation where it needs to be in the demanding world of the 21st century.

What, in addition to what we have already suggested, would improve the federal-state partnership? A practical remedy on which all sides now agree: change how the law measures academic progress. NCLB currently recognizes achievement only when it lands a student above a state’s proficiency bar. The act does not recognize student progress by the lowest achievers, growing from, say, below basic to basic. The act also fails to recognize the growth of the nation’s top students: a school gets zero credit toward AYP for upper-end success. The Department of Education has approved 15 states’ requests to use “growth models” to measure achievement. NCLB should be revised to make growth the only measure of achievement. The act should require that each student’s achievement be judged, for purposes of determining AYP, against one simple standard: is the student on track to be proficient or better by the time of her last reading and math tests in high school? Those tests must be passed for high-school diplomas to be awarded.

Listen to the Podcast

Focus of School Reform Shifting to Teachers (12/17/09)
Are Middle Schools or Middle Schoolers the Problem? (12/10/09)
Biggest Spender in Politics: The NEA (12/4/09)
Saving Jobs or Stimulating Reform? (11/24/09)
Election Postmortem (11/19/09)
Will Congress Reroute the Preschool Juggernaut? (11/4/09)
Voters Choose Neighborhood Schools over Socioeconomic Diversity (10/29/09)
The Nobel Committee Isn’t the Only One Giving Speculative Prizes (10/22/09)
Will Michelle Rhee Triumph? (10/14/09)
Will the Federal Role in Education Double? (10/8/09)
Charter Schools Narrow Achievement Gaps in New York City (10/1/09)
What Congress Is Not Working On (9/24/09)
Charter Schools, Unions, and Linking Teachers with Student Achievement Data (9/17/09)