Cutting to the Chase



By 01/28/2013

4 Comments | Print | NO PDF |

As the U.S. education world eagerly awaits more information about the new assessments that two consortia of states are developing to accompany the Common Core standards, dozens of perplexing and important questions have arisen: Once the federal grants run out, how will these activities be financed? What will it cost states and districts to participate? Who will govern and manage these massive testing programs? What about the technology infrastructure? The list goes on.

The assessment questions that weigh most heavily on my mind these days, however, involve “cut scores.” For if the Common Core is truly intended to yield high school graduates who are college and career ready, its assessments must be calibrated to passing scores that colleges and employers will accept as the levels of skill and knowledge that their entrants truly need to possess. Adequately equipping young people cannot wait ‘til twelfth grade, nor can the assessment sequence. The tests in use from Kindergarten through eleventh grade need to have passing scores that denote true readiness for the next grade and that cumulate to “college and career readiness.”

That’s a daunting challenge for any test maker, but it’s further complicated by widespread fears of soaring failure rates and their political consequences, as well as by Arne Duncan’s stipulation (in the federal grants that underwrite the assessment-development process) that the states belonging to each consortium must reach consensus on those passing scores (in government jargon, “common achievement standards”). All this means, in effect, that Oregon and West Virginia (both members of the “Smarter Balanced” consortium) must agree on “how good is good enough” for their students, as must Arkansas and Massachusetts (both members of PARCC). Can that really happen?

The angst is palpable among state officials—especially the elected kind—over the threat of soaring failure rates, and not just among the poor and dispossessed. We already know from national assessment data that about half of eighth graders with college-educated parents fail to clear the “proficient” bar on NAEP. If (as mounting evidence suggests) “NAEP proficient” is roughly equivalent to “college ready,” and if the new assessments hew to that level of rigor and honesty, many millions of American youngsters will be found unready—and millions more will learn that they’re not on track toward readiness. Such a cold shower should benefit the nation over the long haul, but in the short run, it’s going to feel icy indeed.

Yet that’s only the start. Here are some other perplexing challenges in this realm:

*When will the chilly water hit? PARCC says it doesn’t intend to do any level-setting until after scores come in from the first test-administration in 2015, a classic example of psychometric considerations overriding real-world political considerations. How can states and districts possibly prep their students, their educators, and their publics for new standards (and heightened risk of failure) if nobody knows in advance what sort of performance will be deemed passable? The folks at Smarter Balanced say they’ll set cut scores in advance of that first administration but “confirm” them afterward. What exactly does that mean?

*A single cut-score or several? NCLB says states must set at least three cut scores on their assessments, but the Education Department’s RFP for the new Common Core assessments makes no such demand. It refers only to “college and career readiness.”I had the honor of chairing the National Assessment Governing Board (NAGB) twenty years back when it was struggling to set “achievement levels” for NAEP. Al Shanker (and others) warned that we’d be wrong to establish a single level. It would either be too high, he cautioned, in which case far too few kids would reach it, or too low, meaning it would amount to little improvement over state-set “minimum-competency” levels. In the end, NAGB set three achievement levels, familiar today as “basic,” “proficient,” and “advanced.” We declared “proficient” to be the level that every student ought to reach—but then, as now, fewer than two in five actually achieved that. “Basic” was intended to be a solid marker along the road to proficient and “advanced” was meant to represent “world class”. Will the Common Core assessments do something similar?

*Will these cut scores be static or will they rise over time? Some states (Texas comes to mind) have had good results by slowly elevating the cut scores on their own assessments. Doing something of the sort with the new Common Core assessments would ease the political backlash. On the other hand, the promise of “college and career readiness” would then remain hollow for years to come.

*Speaking of which, who is doing what to ensure that colleges and employers will actually accept these standards—and cut scores—as evidence of readiness? And how exactly is “readiness” being construed? (The federal procurement was nebulous and, while the Common Core standards themselves are substantively very ambitious, the “cut scores” on their assessments are not obliged to be.) For college-bound youngsters, I see “readiness” as the ability to enter credit-bearing courses in the appropriate subjects on non-selective-admission campuses—i.e., to avoid remediation. But I’ve no idea how to define “career readiness.” Who does?

*What happens in states (about half of them) that already have statewide graduation tests (e.g., Massachusetts Comprehensive Assessment System and the Ohio Graduation Test) with minimum passing scores? These have typically been pegged to levels of academic prowess around old-style ninth- or tenth-grade curricula and have been required (of almost all students) as prerequisite to getting a diploma. Are said states supposed to replace these with the new assessments pegged (presumably) to much tougher demands? Administer both? What about a two-tiered diploma system, at least for awhile? Set the new cut scores high to denote true “readiness,” get the colleges to accept them as such, and confer a “readiness diploma” on youngsters who meet that standard. But for some period of time also continue to hand out “regular” diplomas to those who meet the state’s existing graduation requirements, typically a mix of course completions, test scores, and the like. Phase out the latter if possible. (In the old days, New York State did something like this, distinguishing between an ordinary diploma and a “Regents’ Diploma.”)

*As the “credit-recovery” industry grows, often abetted by online delivery of make-up courses and such, how will such arrangements intersect with the new standards, assessments, and cut scores? (The same question may fairly be asked of the G.E.D.) It may turn out that stiffer academic expectations drive more kids to flunk, drop out, etc., and that credit recovery becomes more important than ever. But will that path turn out to be a short cut to a meaningless diploma or an honorable avenue to meet the higher standards and demonstrate one’s “readiness” for job and college?

*If the ACT and College Board folks build the Common Core into their widely used college-admissions tests, as seems likely, will these new consortium-based assessments even be needed at the high school level? For that matter, what would a “cut score” on the SAT or ACT look like?

I hope some smart people are figuring all this out—but if they are, the answers haven’t yet reached my eyes or ears. Time is getting short.

-Chester E. Finn, Jr.

This blog entry first appeared in the Fordham Institute’s Education Gadfly Weekly.




Comment on this article
  • Ze'ev Wurman says:

    “For college-bound youngsters, I see “readiness” as the ability to enter credit-bearing courses in the appropriate subjects on non-selective-admission campuses.”

    Non-selective-admission colleges will take anyone who passes its enrollment requirements, such as: Community colleges will take anyone who graduates high school — is that a true college-readiness in Checker’s mind? Non-selective 4-year state college will accept anyone who passes a small set of courses with barely a passing grade — is that a true college-readiness in Checker’s mind? But none of that will get students within a mile of being accepted to the California State University, not to speak of University of California — is that a true college-readiness in Checker’s mind?

    So can Checker explain why do we need a new Common Core “college-readiness” test that doesn’t prepare anyone for a real college? Is this his justification for the federal takeover of education in the states? Just wondering …

  • Paul Hoss says:

    Ze’ev has essentially asked my question. First of all there’s a college out there for anyone/everyone as long as the parents have a bank account. We all know, higher education in this country has become big business. Does college/career ready mean ready for Harvard, the University of Massachusetts or Bunker Hill Community College? Who, if anyone will define this? And “cut scores,” is that the same as “thresholds of proficient” on state NCLB, the ones that too many states fudged so their students wouldn’t be embarrassed?

    Now for a naive question: How could anyone associated with this massive undertaking allow for two different consortia, PARCC v Smarter Balanced? And with two different “passing” scores? Clearly, one of the primary goals of CCSS was to get away from fifty states going in fifty different directions with standards, assessments, and thresholds for proficient. Now we’re sanctioning two? What’s wrong with one direction for all students across the country in the 46 enrolled states?

  • ccssi says:

    We’ve responded to CRESST’s interim analysis of the 2 consortia’s progress on assessments with our own interim analysis:

    http://ccssimath.blogspot.com/2013/01/consorting-with-consortia.html

  • Jeanne Allen says:

    Checker my dear friend — So after creating this monster you are only now recognizing the disastrous consequences that can and most likely will occur when hundreds of millions of dollars are rendered useless by ridiculous cut scores that will inevitably be set and accepted? This is Virginia standards and assessments, year 4, Mass year 3, and so on. We got NCLB because states started messing with perfectly good standards. Now we are getting Common Core and PARCC because states started messing with NCLB. What next, my friend? National standards by executive fiat?

  • Comment on this Article

    Name ()


    *

         4 Comments
    Sponsored Results
    Sponsors

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform

    Sponsors