In a new Public Impact policy brief, A Better Blend: A Vision for Boosting Student Outcomes with Digital Learning, which we co-authored with Joe Ableidinger and Jiye Grace Han, we explain how schools can use blended learning to drive improvements in the quality of digital instruction, transform teaching into a highly paid, opportunity-rich career that extends the reach of excellent teachers to all students and teaching peers, and improve student learning at large scale. We call this a “better blend”: combining high-quality digital learning and excellent teaching.

The Promise of Blended Learning …

The potential of blended learning to improve student achievement arises from two benefits of blended models that build on each other. One is the power of digital instruction to personalize learning. The other is the capacity of blended models to let schools reach more students with excellent teachers who ensure that students achieve ambitious, personally fulfilling goals.

… Is Not a Guarantee

Technology in our classrooms is nothing new. At various points in the past century, leaders have hyped new technologies in schools, which have generally failed to meet the lofty expectations. Even blended models and other recent digital-learning initiatives have yielded mixed results. And other promising, recent reforms have shown that a lack of focus on teacher quality typically leads to disappointment.

Today’s blended models will likely fall short as well, unless they include excellent teachers playing instructional and team leadership roles that maximize technology’s impact in tandem with their own.

How Schools and Policymakers Can Create a Better Blend, Right Now

For a better blend of technology and teachers, schools must first focus on implementation to combine excellent technology and teaching. It would be easy to move toward blended learning while leaving students’ access to great teachers exactly as it is today. Instead, schools should shift to blended learning while enhancing teaching effectiveness, through:

• Selectivity: Hiring selectively based on indicators predictive of outstanding teaching
• Reach: Extending the reach of excellent teachers to more students, directly and through team leadership
• Freed Time: Scheduling to give teachers time to collaborate, develop, and analyze student learning data during school hours
• Accountability: Giving excellent teachers credit and accountability for the growth of all students under their purview, including those taught by the teachers on teams that they lead
• Authority: Vesting excellent teachers with control of the digital content they use, allowing them to continuously drive improvements in instructional materials in ways never possible previously
• Rewards: Investing savings in paying teachers far more for achieving excellence with more students, making stronger recruitment and enhanced selectivity possible.

Then, to achieve excellent learning at scale, state policymakers must change state policy to enable and incentivize a better blend in large numbers of schools, through:

• Funding that is flexible and weighted by student need, so that schools may invest in the people and technology that best advance their students’ learning
• People policies that let schools hire, develop, deploy, pay, advance, and retain excellent teachers and collaborative teaching teams to reach every student with excellent teachers
• Accountability, using increasingly better measures, that drives teaching and technology excellence and improvement, so that excellent teachers and their teams get credit for using blended learning to help more students, and schools have powerful incentives for a better blend
• Technology and student data that are available for all students, allowing differentiated instruction for all students without regard to their economic circumstances
• Timing and scalability, including implementing a better blend from the start in new and turnaround-attempt schools—when schools often have more freedoms to implement new staffing models that do not over-rely on the limited supply of outstanding school leaders. This also includes helping new schools develop systems for scale, and giving excellent new schools incentives to grow.

Digital learning may be life-changing for students and career-boosting for teachers, but only if schools and policymakers commit to a better blend.

–Bryan Hassel and Emily Ayscue Hassel

This blog post first appeared at EdTech Digest.

They do two things really, really well. First, they take part in on-the-ground, let’s-solve-this-problem human-capital activities. In partner cities across the nation, they train and certify teachers, develop and implement new evaluation systems, help administrators improve observations, and much more.

Chances are, if you’re hearing about interesting, innovative teacher or leader work in an urban area, TNTP is involved.

The second is that they put out these superb little reports. They’re always short and punchy, visually pleasing, terribly informative, and, in one way or another, unexpected. Teacher Evaluation 2.0 was a valuable how-to guide for discriminating policymakers, The Irreplaceables was a teacher-retention wake-up call, and, of course, The Widget Effect was a game-changer.

The organization is at its influential-powerful best when it combines its smarts and muscle—when it can use its research and analysis to inform the field and then help implement the change. For example, TNTP’s findings on the appalling state of teacher evaluations helped shape the Race to the Top application, precipitated a wave of state-level statutory changes, and kicked off some of TNTP’s most meaningful partnerships with states and districts.

Leap Year, the organization’s latest offering, follows in this fine research-meets-practice tradition.

It looks under the hood of the first year of teaching. The conventional wisdom holds that all teachers are lousy out of the gates, so we treat the rookie season, says the report, “like a warm-up lap.”

But there’s much more to this story.

Using its “Assessment of Classroom Effectiveness” (ACE) tool, a multiple-measures evaluation system designed specifically for new teachers, TNTP assessed new educators via observations, student surveys, growth data, and principal ratings.

Among the lessons learned: Not all teachers struggle from the start; in fact, nearly 25 percent score in the top two categories (out of five) in their first observation.

Similarly, while most teachers improve throughout their first year (.2 points on a five-point scale for each observation), many do not. One-quarter of those later denied certification started off poorly and actually got worse over the year.

In fact, just like a charter school’s early performance can accurately predict its later performance, a teacher’s first-year performance tells us a great deal about his/her ability to improve. Teachers who received certification after their first year had an average score of 3.14 on their first observations; those denied certification scored around 2.50, on average, on that first observation.

In fact, writes the report, “Teachers who are performing poorly in their first year rarely show dramatic improvement in their second year.” This includes even those teachers who—thought to have potential despite early struggles—were given a second year (an “extension plan”) to earn certification.

“After more than a year in the classroom, not a single extension-plan teacher earned an observation score in the (top two) categories.”

There are plenty more fascinating tidbits throughout the report; you’ll learn about training and norming observers, using student surveys, adjusting for the inflation of principal ratings, and cultivating early skill sets in teachers.

But probably my favorite new fact relates to improving observations. It turns out that more observations aren’t the key; more observers are. “When assessing tradeoffs between adding observers and adding observations, the evidence is fairly clear—adding observers gives the greater boost to reliability. Giving teachers three different observers, instead of the same observer for each round, significantly increases the reliability of observations.”

The only complaint I had with the report is actually a complaint about an element of the underlying system, specifically, the names of the five rating categories—in order: “Ineffective, Minimally Effective, Developing, Proficient, and Skillful.”

Give 100 reasonable people those names and ask them the best, second-best, etc., I would happily gamble that less than half would choose this exact order.

Complaining about the discrepancy between a classification title and its content may seem like semantics, but it’s more than that. We have such troubled evaluation systems, I believe, partly because we still don’t have honest conversations about effectiveness. By muddying what’s meant with these indecipherable category names, we contribute to the problem.

But this is a minor matter when compared to the serious strengths of the report.

What’s most exciting is that, unlike evaluation and tenure reform, which required new laws in most states, most state departments of education can singlehandedly (or with their state boards) alter certification rules through regulation.

That means an enterprising state chief could swiftly turn this report’s findings into policy. Don’t approve prep programs graduating candidates unprepared for that critical first year; make sure early professional development builds foundational skills; prioritize additional observers over additional observations; and make permanent certification contingent on proof of success.

I’m all but certain a number of states will take this report’s lessons to heart, and once again it will be said that TNTP influenced for the better our educator policies and practices.

-Andy Smarick

This blog entry first appeared on the Fordham Institute’s Flypaper blog.

I taught the usual load minus three hours allotted for attending seminars. I was also expected to mark papers, prepare to teach classes, and observe fellow teachers on a regular basis while concocting elaborate lesson plans for 10 observed teaching visits. I was initially undaunted and looked forward to becoming an expert in my craft.

The first step involved constructing lesson plans, an exact science in NRW, down to the verbs that are permitted when describing pupil progress in the space of a single lesson (“students are able to summarize, to analyze,” etc.). Germans are great fans of the scientific method and enjoy being able to measure and quantify things, a laudable trait. But is it ever possible to prove that all pupils have learned in a given hour what the teacher set out to teach? In this unexpectedly Kafkaesque world, I was stymied as to how one might go about doing this. This led to the first furrow in my brow.

Next we learned the desired lesson format. One should begin each lesson not by asking to see homework but with introductory material, such as a video clip designed to jump-start class discussion. The material should lead the students to state the aim of that day’s lesson themselves, an interesting reversal of those dinosaur days in which the teacher would write the aim on the blackboard. I frequently spent 10 minutes trying to get my students to intuit the question I’d had in mind. Was this really time well spent, I wondered? Furrow number two made its appearance.

Once the question of the day is sorted out, the class discusses how to go about answering it. Students, rather than the teacher, decide whether a debate, role play, mind map, or some other method best suits the topic at hand. Furrow number three was born.

Students then work in groups, jointly preparing their results. I hated group work when I was in school, as I knew who would take responsibility for completing the assignment. Mandatory here, group work is intended to build social skills.

After the presentation phase, members of the class summarize what has been accomplished (“What have we learned today that we didn’t know before?”) and apply the results to an analogous situation, a step referred to as “transfer.”

At this juncture, the teacher may assign a thoughtful homework assignment that encourages in-depth transfer while not overburdening the students. Enter furrow number four.

Despite my confusion about what was expected of me, I looked forward to my first observed teaching visit, in which the class was to compare and contrast speeches by Martin Luther King Jr. and Malcolm X. My students picked out themes and metaphors and persuasive techniques like nobody’s business. At the end of the hour, I floated out of the room, expecting to hear high praise. The wise reader will have a sense of foreboding at this point.

The observer quickly cut to the chase: “So what would you say the students did during this lesson they hadn’t done before?”

“Um,” I stammered. “They read and analyzed speeches by two important civil rights activists.”

“Am I to assume, then, that they were unable to read before?”

“No, of course not.” I regrouped. “They did a close text analysis and compared and contrasted the use of rhetorical devices in the texts.”

“Were the students unfamiliar with such devices before?”

“Well, no, we had previously worked on alliteration, metaphors, and similes,” I admitted meekly.

“So what is it you would say was actually learned by your students in the past hour?”

I stared at him. Technically, applying what one has already learned did not qualify, so I bowed my head in resignation and understood. My students had learned nothing from me. Nothing at all.

Tamara Braunstein is an educator and writer from Brooklyn, New York, who (usually!) embraces intercultural exchange, having taught high school in Senegal and Germany.

Parts of this story have been told before, starting with TNTP’s 2009 Widget Effect report and since replicated in Florida and Tennessee. Beyond adding another state to the list, my paper uses a unique dataset to add some new elements to the story. For example, my paper is the first to include not just teachers but also principals, superintendents, and support staff like librarians and janitors. The data show that districts have trouble evaluating their employees across the board, not just teachers.

In addition, I had access to the actual words and terms that districts use to label their performance categories. It turns out that, like Eskimos with snow, Washington school districts can only talk about what they can see. They have about twice as many terms for positive than negative performance, because they almost never see poor performance. Districts struggle even to create labels for poor performance, let alone to place an individual employee in one of the low-performing categories.

As I articulate in the piece, if an employer can’t differentiate between their employees, they’re likely to treat them all as interchangeable widgets when it comes time to decide on how to help them improve, how much to pay them, or which ones should be retained.

If there’s one strain of criticism to this argument, it comes from hypothetical questions about how many ineffective employees we should expect schools to identify. Recent pieces, from Aaron Pallas and Matthew Di Carlo and The New York Times’ Jenny Anderson, explore this issue.

I have three basic responses. One, I mostly think this question is just an abstraction at this point. School districts across the country are still primarily relying on either/or evaluation systems where all employees are rated satisfactory or unsatisfactory. And, even the places that have implemented new evaluation systems, like Florida and Tennessee, still identify 97 or 98 percent of teachers as satisfactory. Unless you think 1-2 percent of employees is the right number of low performers (which American Federation of Teachers president Randi Weingarten implies in the Times piece), we have work to do.

Two, it’s unfortunate that we aren’t asking the opposite question: How many truly excellent teachers are there? How many teachers and principals should receive extra compensation, be protected from layoffs, be given additional responsibilities, and encouraged to stay on the job? There are two ends to every distribution, but we seem to pay an inordinate amount of attention to the negative side.

Three, there is no “right” answer to this question. It should ultimately be decided by value judgments made by local communities, which should reflect their unique needs. If student performance was low and flat in certain schools, especially compared to similar students in other schools, that community might want to hold more adults accountable. If students at a particular school achieve at high levels and show strong growth, that school probably doesn’t have the same urgency around identifying poor performers.

Dana Goldstein points out that New York City has had this particular fight before, but in most districts the distribution of evaluation ratings isn’t public information, so communities by and large haven’t had this discussion yet. Until they do, and until we start seeing something approaching real differentiation, the question about the “right” number is premature.

Washington has enacted a series of legislative and regulatory reforms improving district evaluation systems. They’ve mandated that districts use four-level rating evaluation systems instead of simple either/or determinations that most districts had been using. And they’ve introduced new elements like requiring districts to use a high-quality evaluation rubric and to factor in student growth into their ratings of teachers and principals. While these are undoubtedly positive steps, the lessons of other states suggest that merely tweaking old evaluation systems is not sufficient to change a culture that doesn’t value performance.

Read the full report here.

-Chad Aldeman

Chad Aldeman is a senior policy analyst at Bellwether Education Partners.

A version of this blog entry originally appeared on The Quick and The Ed.

—Education Next

Behind the Headline
No Substitute for a Teacher
Education Next | Spring 2013

New data from the U.S. Department of Education suggest that teacher absenteeism is becoming a serious problem, with about one in three teachers missing more than 10 days of school each year. June Kronholz takes a close look at the problem, and at the quality of teaching that takes place when teachers are absent, in a new article for Education Next.

-Education Next

That’s why I was so impressed with (and proud of) the recent memo sent out by my former colleagues.

I’ve said many times before that educator evaluation policy got far ahead of the practice. This memo shows that the NJDOE has been assiduous in trying to bridge that gap.

The graphic on page 3 shows how they’ve used multiple sources to continuously inform their work. The timeline on the final page shows how they’ve choreographed the various activities over a long stretch of time to ensure that the work progresses—but prudently.

The heart of the memo is a summary of what they’ve learned from these various sources to date and how the department is responding to the lessons.

I may be biased, but this is—in my opinion—top-notch, grown-up policymaking by a state department of education: Take a broad policy directive, start a pilot, develop multiple external assessors, integrate this work with mid-stream RTTT-3 funds and a new tenure law, make course corrections, act with transparency about findings, and push on.

I would commend this memo to just about anyone in our field, but particularly groups like TNTP that do this work day in and day out, officials at USED interested in witnessing the difficulty of bringing an Administration priority to life, the Gates Foundation MET team (who I’ve been pestering about next steps), academics who study policy implementation, and anyone else with an interest in today’s work on educator effectiveness.

Finally, I’m including a few links to somewhat unflattering news articles associated with the memo’s release. These should serve as lessons to those who want to do serious policymaking. Do your job thoughtfully and well, and take pride in that—but know that the aspects likeliest to be covered will be those that generate the most heat, not the most light.

My congratulations to my superior former boss Commissioner Chris Cerf, my amazing former colleague Chief Talent Officer Pete Shulman, and their colleagues.

When people look back on this era of ed reform, I’m sure they will remember the big pieces of legislation and the political fights. That’s wonderful theatre for sure.

But the day-to-day work to animate cold words in a statute book is what matters most.

-Andy Smarick

This blog entry first appeared on the Fordham Institute’s Flypaper blog.

This surely is a key question that will come up this spring in statehouses across the nation as they face another tough budget year. District officials, if they are wise, will not just rely on the same old belt-tightening maneuvers. Indeed, perhaps the only viable option is seriously addressing policies toward educator salaries.

To put the issue in perspective, let’s start with some basic history about the budgetary picture for schools. The recession of 2008 was a rude shock to state and local governments, and especially to schools. Coming off a century of continuous growth in spending per pupil, districts were slow to adjust to the possibility that the revenue collapse might actually put them on a more perilous spending path.

Initially, they were saved from tough decisions. The federal government, faced with a weakened overall economy, charged in to make up for fiscal shortfalls of school districts through a stimulus package designed to get the macroeconomy moving again. This package provided a bridge that kept spending in most states from falling at the pace of lost revenues.

In the 2009-10 school year (the latest with available data), when the stimulus was in full swing, the overall story was still not rosy. State and local revenues per pupil fell in real terms in 39 states. When federal stimulus dollars were added, overall real spending per pupil still declined in 23 states.

Moreover, the stimulus package was explicitly a temporary fix that was designed to end quickly—and it did. To the extent that they thought about it, most states and districts implicitly presumed that, as the stimulus funds were phased out, their own funds would return, and that they could then resume on their prior path. For that reason, in many states, the initial response after the stimulus money stopped flowing was largely to try to do what had been done before the recession and to wait out the storm.

The shock came after the stimulus went away. Even as state revenues began to recover, school spending generally did not return to its previous trajectory.

In reality, state and local revenues continue to be slow to recover, and states have found themselves facing deficits (many of which are illegal, according to state constitutions). Given overall state demands, a number of states are dealing with deficits by allowing school spending to fall. Moreover, most projections suggest that general fiscal pressures on schools are likely to last for some time.

A first response has been to resist any spending decreases, generally arguing that schools should be exempt from fiscal shocks. For some, this resistance has included going to court (e.g., in New Jersey and Kansas) to argue that reduced funding violates established state constitutional requirements for providing students with an adequate education. Nonetheless, a majority of the states have simply fought out their funding battles in their legislatures—with few states returning to the spending growth of the past.

This leads back to the simple question: Isn’t it possible that forced spending reductions will make the education system more efficient? Since there is spending in many districts that is not contributing much to student learning, can’t we simply squeeze out this inefficiency by cutting the funding to schools?

Although we do not have a definitive answer to that question, my response is most likely no.

If school districts had a line item in their budgets for “waste, fraud, and abuse,” we could just reduce that to deal with the budget pressures. Unfortunately, we do not find such itemized inefficiency.

It is possible, as the Council of the Great City Schools has convincingly done, to document wide differences in costs of various management functions, ranging from finance operations and procurement, to safety and security, to transportation. Many of these differences are large enough to explore further, but acting on them is unlikely to solve long-term fiscal problems, since each is a relatively small budget item.

The big money still resides in instructional personnel, meaning mainly administrators and teachers. Salary and benefits funding for instructional employees represents the largest spending area in the typical district, bringing to mind the old Willie Sutton adage about robbing banks “because that’s where the money is.” The case for inspecting this spending, however, runs much deeper.

First, as is widely recognized, teachers and principals have the largest impact on student performance, implying leverage on the achievement side of the efficiency equation. Second, numerous studies have shown that teacher pay based on degrees and experience is unrelated to teacher effectiveness, implying leverage on the cost side of the efficiency equation. Addressing issues of inefficiency almost certainly demands addressing the fact that salaries, and by implication, total compensation paid by schools, are unrelated to student outcomes.

Dealing with either side of this efficiency equation has no historic precedent, and districts are unlikely to focus on these issues just because funding is cut. Partly because of existing state labor and education laws, these issues are often simply not on the table even in times of fiscal stress. Yet, that is not the full story, because even when these constraints are not binding, we see little systematic movement toward rationalizing instructional spending and performance.

A number of states have moved forward by at least eliminating pure LIFO—last in, first out—rules for reductions in the teaching force, which are designed to protect more-experienced teachers during layoffs. Wisconsin got the bulk of the publicity for its work on this front, but Colorado, Florida, Indiana, Oklahoma, and others have also opened the way for decisions more closely related to teacher performance. These states have taken the first necessary steps of reducing teacher-tenure guarantees and calling for better evaluations of teachers and administrators.

These actions, however, will probably be insufficient. There is considerable inertia in local districts. There are contracts that restrict action. There is resistance from teachers’ unions. There is little experience or political will to change.

And there is the final defense of the status quo: “Surely reform would cost money, and we do not have it.” But, yes, districts do have it. They are currently spending money in ways unrelated to achievement, and reform means ending that, not just adding on to what was done before.

The only way that efficiency will be significantly improved is by strengthening the relationship between salaries and performance. Currently, we dramatically underpay our best teachers while dramatically overpaying our worst. Efficient policies imply paying significantly more to the best teachers—not just giving small, temporary bonuses for student achievement—to keep them in the classroom longer. Additionally, it probably also means having them teach more students, because dealing with tighter budgets and paying significantly higher salaries will most likely require slightly larger class sizes. At the other end of the performance spectrum, we cannot reduce the pay of the worst teachers enough, and we simply must move them out of the classroom. The impact of the small numbers of unacceptably ineffective teachers is disproportionately large and represents a huge drain on both achievement and finances.

These are not things that happen easily or automatically. As we embark on a new budget season, we should not delude ourselves that just cutting school budgets will lead districts to a new, more efficient place. Left to their own devices, districts are much more likely to do what they have always done, but on a somewhat restricted scale. This path will lead neither to more efficiency nor to better results—and in fact could significantly harm students.

Improving outcomes—either with fewer or more resources—requires significant change. It will be virtually impossible to get such change without active state policies that push for the alignment of salary budgets with classroom performance.

Eric A. Hanushek is the Paul and Jean Hanna senior fellow at the Hoover Institution at Stanford University. His writings can be found at hanushek.stanford.edu.

This blog entry was first published in Education Week, Vol. 32, Issue 20, Pages 28-29, 31

Instead, he became a substitute teacher.

In the college town where he was living, an astonishing 47 percent of the school district’s 721 teachers were absent more than 10 days during the school year, according to data the district reported to the U.S. Department of Education for a 2009–10 study. That number rose to 61 percent in an elementary school with one of the district’s highest percentages of black, Hispanic, and low-income children.

Even at that, the district’s absences don’t appear to be record setting. U.S. teachers take off an average of 9.4 days (roughly 1 day per month) each during a typical 180-day school year. By that estimate, the average child has substitute teachers for more than six months of his school career.

Those absences provided full-time employment for my son. With a month-old bachelor’s degree, he taught history and Spanish, his majors; calculus and literature; 2nd and 4th grades (after his second day on the job, the district asked him to take the 2nd-grade class for the rest of the year); tennis (no, he doesn’t play); and gym to a class of severely disabled high schoolers. Once, he worked as a secretary at the alternative school; none of the four teachers assigned to the school showed up that day.

The district didn’t pay much: $60 a day. But it also didn’t ask much in the way of credentials: no teaching certification, teacher education classes, or training beyond a three-hour orientation that focused mainly on administrative details like time sheets. That isn’t unusual either: in some of the country’s larger school districts—including Maryland’s Baltimore County, Florida’s Hillsborough County, Georgia’s Cobb County, and Colorado’s Jefferson County—substitutes need only a high-school GED.

My son taught a high-school unit on World War II, his intellectual passion. But most often, teachers left behind worksheets, quizzes, and videos for him to monitor, amounting to what University of Washington professor Marguerite Roza calls “a lost day for most kids, regardless of the qualifications of the sub.” Indeed, many schools are looking for someone just to keep order rather than to teach differential equations.

“A lot of times, principals are just praying for basic safety,” said Raegen T. Miller, who has studied teacher absenteeism as associate director of education research at the Center for American Progress and as part of a Harvard University team.

No problem there: my son is, after all, a Marine.

Counting the Days

The education department reported after the 2003–04 school year that 5.3 percent of U.S. teachers are absent on any given day, and that’s still the number most researchers use. But districts account for absences differently: some would count the tennis coach absent if he left his gym classes in the hands of a sub to attend an out-of-town tournament with his team; others wouldn’t. Some count professional development days when subs are hired to take the class; others don’t.

The U.S. Bureau of Labor Statistics employs a weekly absence measure and reports that in 2011, nationwide, 3 percent of the workforce worked less than a 35-hour workweek because of absences. Among public-sector workers, rates were 3.9 for federal workers, 4.2 for state, and 3.6 for employees of local governments.

Geoffrey Smith, who studies substitute-teacher management and founded the Substitute Teaching Institute at Utah State University, says, “A lot doesn’t get called in.” Each of Utah’s 42 school districts counts teacher absences differently, he told me, which means there’s little consistency in the data. Still, he said his surveys suggest that between 8 and 10 percent of teachers are absent on any given day, and there’s some anecdotal evidence on his side.

Last summer, for example, the Camden, New Jersey, school board outsourced its substitute hiring to a private vendor because the job was so onerous: between teachers calling in sick or on leave, the district needed to find subs for up to 40 percent of its teachers each day, it told the local newspaper. In a 2011 report for the Providence, Rhode Island, school board, researchers at Brown University’s Urban Education and Policy program found that the district’s 1,321 teachers took off an average of 21 days each per school year.

In the education department’s 2009–10 report—assembled by its Office for Civil Rights from surveys of 57,000 schools—on average, half the teachers in the 208 Rhode Island schools surveyed were absent more than 10 days during the year, surpassing teacher absences in Hawaii, Arkansas, Oregon, and New Mexico by only a whisker. Nationally, 36 percent of teachers were absent that often. And even in Utah, which reported the lowest absence rates to the department, 20 percent of teachers took off more than 10 days each school year.

Who are those teachers? Harvard researchers Raegen Miller, Richard Murnane, and John Willett studied a district they identified only as large, urban, and northern. Duke University researchers Charles Clotfelter, Helen Ladd, and Jacob Vigdor analyzed data from North Carolina schools. Both studies concluded that teachers in bigger schools were absent more often than those in smaller schools. Elementary-school teachers took off more time than did those in high school. Tenured teachers took off 3.7 more days than did those without tenure.

Teachers in traditional districts seem to take off more than those in charters. Using the education department’s Office for Civil Rights data, Miller estimates that about 37 percent of teachers are absent more than 10 days at district elementary and middle schools compared to 22 percent at charters.

Female teachers under age 35 averaged 3.2 more absences each school year than did men. Teachers who had a master’s degree or graduated from a competitive college took less time than those who didn’t. And teachers in low-income schools were absent more often than those serving higher-income families. One in 4 middle schools in the Duke study were among those with the highest absence rates, but that dropped to 1 in 12 among middle schools serving the district’s most affluent students.

Teachers argue that they’re absent as often as they are because they’re subject to all kinds of infections from sniffly-nosed youngsters and to intense stress in tough schools. Teaching—and particularly elementary-school teaching—is still a majority-female occupation, and child care still falls overwhelmingly on mothers, they add. When a teacher’s child is out with the flu, she may have little choice but to stay home, too.

But other research contends that teachers’ frequent absences are driven by generous leave provisions in their contracts, which typically include time off for illness and personal choice and, in many cases, family deaths, voting, religious observation, union business, conferences, cancer screening, even driver’s license renewal. The National Council on Teacher Quality (NCTQ), which maintains a database on collective-bargaining agreements in 113 large school districts, reports that the contracts give their teachers, on average, 13.5 days of sick and personal leave per school year.

In Columbus, Ohio, the contract allows teachers 20 paid days off, in addition to school holidays and summer breaks. Teachers have 21 days in Boston, 25 days in Hartford, and up to 28 days in Newark, according to NCTQ. By contrast, only 73 percent of private-sector employers provide any sick leave in addition to paid vacation, according to the U.S. Labor Department, and they offer an average of eight sick-leave days during a 12-month work year. In New York City, even substitutes qualify for sick days, one per month.

Teachers certainly are exposed to all manner of classroom germs, but there’s also evidence that a lot of absences are discretionary. The Harvard study found that the highest percentage of absences at that northern, urban district were on Fridays, when 6.6 percent of teachers took off, providing themselves a three-day weekend. Only 4.9 percent took off Tuesdays. More than half the absences that the study examined were for “personal illness,” and more than half of those were for only a day or two. Perhaps coincidentally, the district required a doctor’s excuse for an absence of three days or more.

Substitutes typically earn less than $100 a day. But even at that, Raegen Miller puts the cost of substitute teachers at $4 billion a year, or about 1 percent of total K–12 spending. In Fairfax County, Virginia, whose 13,000 teachers are offered 11 days off a year, the district budgeted $19 million for substitutes in 2012. Cleveland, Ohio, whose teachers may take 18 days off, is budgeting $10.8 million for substitutes this year.

University of Washington’s Marguerite Roza calculated what districts would save yearly on substitute pay if teachers took leave at the same rate as other professionals, that is, 3 days during a comparable 180-day year. Her conclusion: $43 per pupil in savings, or about one-half percent of school budgets.

Cost to Learning

The costs are far more than just financial, of course. The Duke researchers found that being taught by a sub for 10 days a year has a larger effect on a child’s math score than if he’d changed schools, and about half the size of the effect of poverty. Columbia researchers Mariesa Herrmann and Jonah Rockoff concluded that the effect on learning of using a substitute for even a day is greater than the effect of replacing an average teacher with a terrible one, that is, a teacher in the 10th percentile for math instruction and the 20th percentile in English instruction.

There’s no research on how long that effect lasts. But because learning is cumulative, “you would expect that the effect would aggregate to a larger loss of achievement over an entire school career,” Mariesa Herrmann told me. In other words, “A teacher not in a classroom is a missed opportunity for learning,” says Kate Walsh, president of NCTQ.

Some of that learning loss comes from disruption to the classroom: subs don’t know the kids, the classroom routine, the school culture. They have no skin in the game, nothing to win or lose if no one learns Chaucer. Classroom management breaks down. Miller told me that even janitors know when there’s been a sub: “There’s more crap on the floor.”

Teachers often leave busywork behind, or no work at all. In math, particularly—where the roiling debate over how to teach basic computation continues—subs often are cautioned not to teach anything at all for fear of setting the class back. Given testing pressures and school-wide lesson planning, there’s little time to reteach a lesson.

Then, too, districts set fairly low standards for their subs, although the weak economy and teacher layoffs seem to be bringing more certified teachers into the sub pool. Of the 113 large districts in the NCTQ’s database, less than one-quarter require that subs hold any teaching credentials. Only 37 districts require a college degree; 1 in 11 asks for only a high-school diploma or GED.

Out of curiosity, I perused the Denver Public Schools “Substitute Teacher Handbook.” It told me subs can’t wear bedroom slippers to work, that they’re paid $90.40 a day, and that they can ask for, but shouldn’t expect, an evaluation. It didn’t say anything about their qualifications to teach.

Carrots and Sticks

With school budgets strained and learning loss evident, I wondered why districts didn’t try to claw back some of the days they’ve granted their teachers for illness and personal leave. Miller has calculated the learning loss attributable to teacher absences to be equal to about 5 percent of the achievement gap between black and white students. “If you had an intervention that would close the gap that much, it would be worth doing, wouldn’t it?” he asked.

The problem, NCTQ’s Kate Walsh told me, is that teacher quality has been ignored as a reform issue until fairly recently. Now that the focus has shifted, superintendents have so many bigger issues to confront—teacher-evaluation systems, tenure, differential pay—that “you can understand why they don’t go after this benefit,” she said.

“This is small change” to most districts, Miller added—they’re facing budget gaps way larger than that. And the issue touches such a nerve with teacher groups that “there’s profound reluctance to get into it at the bargaining table,” he said. “It’s an entitlement.”

“You should have seen the hate mail I got” after publishing a recent report on teacher absences, Roza told me.

Instead, districts have been turning to incentives to keep teachers at their desks. Almost all districts allow teachers to accumulate unused sick and personal days, and to cash them out when they retire or leave the district. The Detroit schools paid out $12.5 million for unused sick days in 2010–11, which was twice what it spent on subs. New York City employs hundreds of full-time subs who report daily to a “home” school and fill in where they’re needed. They provide some continuity for the school and soak up teachers who have been laid off by budget cuts or enrollment declines. Even so, says Columbia’s Herrmann, these “absent teacher reserves” account for only 10 to 15 percent of the teachers that New York needs every day.

But some districts, facing such huge eventual payouts, have begun capping the number of sick days teachers can accumulate, posing a use-’em-or-lose-’em dilemma for teachers. And for younger teachers, “in deciding whether to be absent, are you really thinking of your retirement?” Columbia’s Herrmann asked.

Researchers have proposed that districts pay teachers a bonus for the days they don’t take off, or give their schools the money that would have been spent on subs as a collective incentive, or set up a reward system for teachers with good attendance (the Columbia study found that only 3 percent of teachers had perfect attendance). The Duke researchers proposed increasing teacher salaries by $400 a year and then charging teachers $50 for each day they take off. They estimated that the scheme would reduce absences by about one day per teacher and largely pay for itself. (Among the arguments raised against the proposal: it would hit female teachers with children harder than it would hit men.)

Many private schools and some charters simply don’t hire subs. Colleagues fill in for absent teachers during their own nonteaching hours. That keeps the class on pace when, say, one 4th-grade social-studies teacher can fill in for another, especially since they’re likely to have drafted the lesson plan together. It also means that one teacher is imposing on another, which creates some accountability, or at least discomfort for the teacher calling in repeated excuses.

But union contracts often limit how many hours a public-school teacher must be in the classroom: that’s why a school may hire a substitute librarian rather than send everyone back to their homerooms when the full-time librarian is out. And some contracts require districts to pay their teachers to sub, usually at rates higher than they would pay a substitute. The Wichita district pays its own teachers $20 an hour; a full-day sub earns $99.

Research also shows that absences increase where districts install automated absence-management systems instead of leaving the job to school secretaries. Teachers log onto the system’s website to report they will be absent. Subs log onto the same site to choose the class they’ll teach.

But districts are adopting the systems anyway, as school support staffs are slashed and technology becomes cheaper. Among the largest of the systems, privately owned Aesop is in 3,000 districts. Aesop claims on its website that it saves districts money: its “fill rate”—that is, the number of classrooms it fills with a sub—is so high that schools don’t need to use more costly downtime teachers. The company adds that its data reports enable principals to track who’s frequently absent and “to work with teachers” who are.

But the automated systems mean that teachers no longer have to talk to the principal, and perhaps explain that they’re taking a day off for a wedding-gown fitting or an auto tune-up. The automated systems also give schools less control over who will fill their classrooms: schools still can call favorite subs, but when those aren’t available, an opening is listed on the website and anyone on an approved list, including the GED holder, can claim chemistry class.

Researchers have found that teachers are absent more often when their fellow teachers are, too. That can suggest there’s an “absence culture” in the school, as in “heck, everyone else is doing it.” It also suggests a struggling school, where teacher absences and student absences feed off one another until neither group shows up. Or it may suggest weak management and unhappy workers. “If you’ve worked in an effective organization, people show up. If you’ve worked in a dysfunctional organization, they take off,” NCTQ’s Walsh observed.

I wondered about that when I looked at the education department’s 2009 report on absenteeism and paged to high-performing Montgomery County, Maryland. The district reported that only 6.8 percent of teachers were absent 10 or more days per year at one school with a high percentage of black, Hispanic, and low-income children. But at two other schools with similar demographics, 42 percent and 19.6 percent of teachers took off that much time.

I asked the district about that. Then I asked again. As in every district I asked about teacher absenteeism, no one answered.

June Kronholz is a former Wall Street Journal reporter and a regular contributor to Education Next. Her son has resumed active duty with the U.S. Marine Corps.

“No,” says a boy named Max in a black rock band T-shirt. “He was on PCP.”

A few nod their heads in agreement. They could be talking about a sophomore who got wasted over the weekend or a senior who got busted in the parking lot, by all appearances, but they’re actually discussing the president of a road-racing company, a man whose crimes had nothing to do with illegal substances. He earned the attention of these students through a poorly written letter, one that caught the eye of Ms. Andrea Bassett, an Honors English teacher at Needham High School in Massachusetts.

Max reads a printout of the letter to the other boys as if he were dropping meat into a shark tank. “‘In trying to formulate what to say in regards to yesterday’s events,’” Max quotes, “‘I realized that what I said over and over to the folks I helped get on returning shuttle buses was exactly what should be said to all.’”

“What?” someone exclaims. Everyone laughs. “He just throws in words!” Max says. He goes on to finish the opening paragraph.

“‘While it became repetitive, it was no less from the heart in any one time from the other:’”

“He ended with a colon,” says a boy who didn’t shave that morning.

“You can pretty much revise the first paragraph,” says Mike, his cheek on his hand.

A stocky kid named David chimes in. “That’s not just bad grammar,” he says, indignant. “That’s, like, bad PR.”

His comment catches the attention of Ms. Bassett, who is making rounds to each cluster of students. “David,” she says, “the life lesson here is that bad grammar is bad PR. You guys remember that.”

Ms. Bassett is the newest faculty member of the English department at Needham High, a lean, athletic blonde who chose to show this letter to her students as a good bad example. It was an apology for a poorly managed 15K, a race that Ms. Bassett herself ran, averaging a 10-minute mile. In the letter, the president of the road-racing company tried to explain how the runners had gotten misdirected and why there was no water at the finish line. Ms. Bassett thought the greater indignity was enduring an apology from a president whose prose waddled along for 40 paragraphs, weighed down with extra words and never-ending sentences.

“He would definitely fail a grammar assignment in this class,” she says, to wide classroom approval. Ms. Bassett is part of a department that has decided to take grammar seriously. Too many students were claiming that nobody had ever taught them the rules. Needham High School’s seniors, mostly from upper-middle-class families, were graduating without knowing the parts of speech or parts of a sentence. They would sometimes write “u” instead of “you” in their essays, or a lowercase “i” instead of “I.” The high school, like many others, had been suffering from a lack of standardized grammar instruction throughout the grades. Over the summer of 2011, the English department created a series of PowerPoint presentations to coordinate grammar instruction across the grades, hoping to provide their students a better, more uniform understanding of the rules. The goal was to set a baseline for Needham High students, allowing them to review old lessons and master new ones through the slides.

“They actually like it. They like something in front of them that’s task-oriented,” says Ms. Bassett. The PowerPoint slides look like blueprints, with their simple, white-on-blue form, and they lay the rules out in a straightforward way. Needham High’s teachers have been using them for more than a year, and Ms. Bassett believes that they have made a subject that was once confusing “concrete and quantifiable.”

Battling Barbarism

Sloppy English usage may seem like a modern problem, but the laxness that has led to this moment in grammar’s history bears a strong resemblance to the atmosphere in early-18thcentury England. At that time, decades had passed since the golden age of English, with its production of the King James Bible (1611) and the plays of William Shakespeare (1564–1616). Many began to fear that the language was going to the dogs.

“Our language is in a manner barbarous,” poet John Dryden complained in 1693. Theologian Thomas Stackhouse agreed. “We write by guess, more than any stated rule,” he said in 1731, “and form every man his diction, either according to his humour and caprice.”

Dryden and Stackhouse weren’t complaining about rule breakers, as Needham’s teachers do; they were complaining about a lack of rules in the first place. In the early 1700s, no English-specific grammar or dictionary existed. Writers worried that in a few generations their work would become as unintelligible as Old or Middle English was to them. As Jonathan Swift, author of Gulliver’s Travels, put it in 1711, “Such as Chaucer is, shall Dryden be.”

Much of the concern sprang from the English Civil War (1642–1651). The overthrow of the monarchy and the turmoil that followed had dirtied the image of English, a green, unsure language at the time. With Oliver Cromwell leading the country and the king himself beheaded, the King’s English was in jeopardy. An expansion of printing during the war had allowed writers of less means to publish material. “Such an infusion of enthusiastic jargon prevailed in every writing, as was not shaken off in many years after,” said Swift. “To this succeeded the licentiousness which entered with the restoration, and from infecting our religion and morals fell to corrupt our language.” Religion, morals, language—they had all grown shoddy by the 1700s, many thought. The English language needed help. Fast.

Generation Gap

In the English department lounge at NHS, teachers sit at a long table, sipping on water bottles and pulling out home-packed lunches. They too believe that English needs help, and they want to fix it. Jonathan Cooke thinks the decline of grammar is a recent one. The only person with gray hair in the room, he’s a former lawyer who switched to teaching 15 years ago. He remembers a time in the early 1970s when virtually every student could identify a direct object. “I learned that all through middle school,” he says. “By the time I got to high school, it was more funky. You could take a course in just satires.”

Brent Concilio, a young, Dartmouth-educated teacher with a turkey wrap in his hand, thinks the shift in the 1960s came from the ideas of John Dewey (1859–1952), a reformer who pushed for a child-centered education. “In the interest of making English class more ‘relevant’ to students’ lives, we began having students read contemporary novels and talk about how those novels made them feel.”
“Wicked cool,” says Cooke.

“But any time you make room for something, something else has to go,” says Concilio. “And what went was the systematic teaching of grammar.”
This shift in priorities was only one of the factors in the abandonment of grammar instruction. Another factor was a public campaign against the concept of a single correct way of speaking.

The Conference on College Composition and Communication in 1972 stated that students had a right “to their own patterns and varieties and language.” The resolution, which was adopted in 1974 by the National Council of Teachers of English (NCTE), went so far as to say that correcting language was “immoral” because it was really an attempt by one social group to exert dominance over another. Suddenly, grammar was oppressive. It was stodgy. It was all but banished from many classrooms. The pendulum swung far away from the prescriptive, rules-oriented English once taught in schools.

After the sixties, grade-school students, by and large, didn’t learn grammar the way their parents had, and now, decades later, they don’t reinforce the rules very well with their own children. Without this reinforcement at home, much of the burden to teach students correct English lies with teachers.
The problem with that idea, of course, is that many teachers today didn’t learn much grammar when they were in school, either. “It’s now been gone for a generation,” Concilio says. “A lot of people, I think, really don’t understand the value of it.”

Rules of Order

The pushback we have seen over the past few decades could have been less severe if the architects of English grammar had set up the rules to be more respectful of actual usage. The early grammarians were reacting to disorder, though, and they weren’t afraid to leave a few people behind in their drive for structure.

One of the first of the language reformers was the writer Dr. Samuel Johnson. In 1755, he published A Dictionary of the English Language, a mammoth work of scholarship that he spent nine years writing. The dictionary was a tremendous step toward preserving the language, but Johnson complained that he had to create it with “no assistance but from general grammar,” meaning Latin grammar, because nobody had systematized the English language yet.

The call for a unique English grammar grew louder. It was the greatest void in the language, now that a dictionary had been written. Eighteenth-century scholars and politicians believed that such a grammar would dignify the language on the world stage, helping to emphasize England’s political autonomy from the European continent.

Writers were begging for standards not only for their own guidance, but for their legacies.

Robert Lowth stepped up to the challenge. Lowth, a clergyman and eventual bishop of London, believed that correct grammar was next to godliness, and that the King James Bible was the gold standard of the language. English, he said, was becoming far too loose, and it needed “stiffening up,” a claim that would resonate several centuries later with Needham High School’s English teachers.

Lowth’s Short Introduction to English Grammar, published in 1762, was not the first English grammar ever written, but it outsold all the others on the market. The most notable of the competing guides was a descriptive grammar by theologian and chemist Joseph Priestley (1733– 1804). Robert Lowth’s grammar proved more popular because Britain in the 18th century, still recovering from the English Civil War, wanted prescription, not description; rules, not the reality—especially not the reality of the lower classes.

The only problem was that “stiffening up” the language left English a bit too stiff. Lowth often looked to Latin for inspiration rather than to customary usage when he settled a question. For example, he frowned on the expression “It is me” because it ended in the objective case. “It is I” matched the Latin construction, and was therefore better, according to Lowth. It has remained the rule for proper usage ever since, but has always been too awkward to gain traction among most English speakers.

Lowth also disapproved of prepositions at the end of sentences. “The placing of the preposition before the relative is more graceful,” he said. The rule worked for Latin, but not so well for English speakers, whose sentences ended naturally in prepositions. Lowth at least acknowledged that this tendency was “an idiom which our language is strongly inclined to,” showing that the inclination bent strongly in his direction, too.

He called double negatives improper, and grammarian Lindley Murray (1745–1826) later proved this claim with algebra, even though Shakespeare was known to use a double negative occasionally. Lowth also preached against verbs that had merged tenses over the years. He preferred strong verbs that had a distinct past tense: drink and drank, write and wrote, for example. Verbs whose past tenses merely ended in “ed” were the result of a natural streamlining of the English language. Lowth wanted to fight against this tendency and supported usage that kept verb tenses distinct and intact, like Latin verbs, which were in no danger of merging because the language had been dead for centuries.

Lowth’s ideas pleased the class conscious because his rules were too pedantic for the lower classes to adopt. They allowed social climbers a clever way to blend in with the upper class. They fit the zeitgeist because 18th-century England, with its zeal for classical ideals of logic and reasoning, was fertile ground for anyone who wanted to explain something rationally, even something as irrational as the English language.

A Short Introduction to the English Language ran 22 editions in the 18th century and led several decades later to an important spin-off grammar by Murray, which became a staple in 19th-century schools on both sides of the Atlantic. What began as one man’s guidelines eventually became hard rules, enforceable with a switch. Even when Americans began producing their own textbooks, in the mid-19th century, they rehashed most of Lowth’s and Murray’s ideas.

To be sure, Lowth and his fellow reformers stabilized the language, but their prescriptive, top-down approach also set the stage for the instability we have now. The gap between proper written English and actual usage is wider today than Needham High School’s football field.

Today’s Torchbearers

Grammar instruction has been mocked and marginalized for decades, partly because the rules were too cold and unfeeling. Lately, the rules have been making a bit of a comeback. Educators are starting to believe that English grammar, even with its quirky rules, is far better than nothing, after they’ve seen the results of nothing. The SAT added grammar questions to its format in 2005 in response to pressure from college administrators. Parents have begun to push for more English language instruction. The NCTE has softened its position, and now we see a growing number of teachers bringing grammar, the forgotten spinster of school subjects, back to the party.

“In the work force, grammar will be as important as this training of analyzing literature,” says Ms. Bassett. “[These students] are not going to be paid in 20 years for analyzing literature.

They’re going to be paid to present something to their company.”

Her colleagues list several benefits that come from grammar instruction: clear cover letters, stronger writing skills, and an easier understanding of a foreign language, to name a few. If there is a bias toward one “correct” way of speaking, well, they want their students to learn it.

And so the legacy of the English language lies heavily with teachers like Ms. Bassett, a recent convert to grammar herself, and her students. They may go too far in their reforms, as their predecessors have, or they may achieve a balanced approach. At any rate, the appearance of today’s grammarians, in their hoodies and sneaks, bears little resemblance to that of their forerunners.

A boy named Leo, in a Red Sox cap, raises his hand to make a suggestion in Ms. Bassett’s class. “You could put an em dash here: ‘Our race director quickly came up with a contingency plan—real time, on the spot—in the horror of what could have been a disaster.’”

“Oh, my gosh, you are an em-dash king. Nicely done,” says Ms. Bassett.

David raises his hand. “This is the dumbest thing,” he says, pointing to a paragraph in the memo: “‘Finally, we start the race. What happens next defies belief, absolutely and completely!!!’ Like, why are there three exclamation points?”

“What sort of tone does it create to use three exclamation points?” asks Ms. Bassett.

“Colloquial,” a few answer back.

“Yeah,” she says. “Third grade. Like a tween looking at Justin Bieber.”

At the end of the hour, as Max, Mike, and David put their pens away and zip up their bags, Ms. Bassett warns her students that there are consequences to becoming successful and writing with poor grammar: “You’ll get ridiculed in my class.”

And in a society that has neglected grammar for so long, mockery may be just what grammar needs to come back into vogue. Only now, the ridicule is coming from the bottom up, from 17-year-olds who specialize in snark, who know the rules better than their future bosses, who write clean sentences but don’t appear very close to godliness. They may be Robert Lowth’s best hope.

Elise Hahl contributed to Choosing Motherhood (Cedar Fort, Inc., 2013) and has written for the online magazine “Outside In Literary & Travel.”

Data from the Bureau of Labor Statistics (BLS) show that school district costs for teachers’ health insurance rose at an average annual rate of 4 percent above inflation from 2004 to 2012. In 2004, health insurance costs tacked 11.4 percent onto teacher earnings; in 2012, they added 15.5 percent. At roughly $560 per pupil per year, the national average masks wide variation across states, as districts in some states have relatively low insurance costs while costs borne by districts in other states are quite high. The data do not include health costs for other school employees and retirees, which can be quite substantial.

In this study, we examine BLS data to compare the costs to districts for teacher health insurance with similar costs to private-sector employers. We find that insurance costs for teachers are 26 percent higher than they are for private-sector professionals, and this is partly explained by greater unionization in the public sector. We also examine data newly available from Wisconsin to quantify the impact of that state’s recent change in collective bargaining law: we find a reduction in district costs of 13 to 19 percent, the result of lower-cost policies and higher teacher contributions.

Comparing Employer Costs

We begin with a basic, high-level question: How do employer health care costs for teachers compare with those for private-sector professionals? The most comprehensive national data published on employer costs, the BLS National Compensation Survey (NCS), provide estimates of employer insurance costs on a “per-hour-worked” basis for 180 groups of employees, broken down by occupational groups, industries, ownership (private industry or state and local government), and other characteristics. These data do not separate health from other insurance costs (life and disability) for teachers, but these other components are small (approximately 5 percent of the total), so this does not significantly affect our results.

We focus our comparisons on K–12 teachers and private-sector professionals. Using unpublished data provided to us by the BLS, we multiply the hourly employer insurance costs by the number of hours worked to obtain annual costs for each group of workers. Some 97 percent of K–12 teachers work full-time, while 83 percent of private-sector professionals do so. Because part-time workers are less likely than full-time workers to have health insurance from their employers, we adjust the private-sector comparison data to match the percentage of teachers who work full time.

Click to enlarge

We estimate from these data that the national average of annual employer insurance costs in 2012 was $8,559 for K–12 teachers, and $6,803 for private-sector professionals. The difference between the figures has increased since 2004. Annual employer insurance costs for K–12 teachers rose 67 percent, compared to 49 percent for private professionals. The gap between employer costs was just 12 percent in 2004 but rose to 26 percent by 2012 (see Figure 1).

Our estimates for employer insurance costs average the expenditures across those employees who are covered by an employer’s plan and those who are not. Employees may not be covered either because no plan is offered (an issue for part-time employees in particular) or because the employee chooses not to participate (e.g., because coverage is available through a spouse’s employer). According to the NCS Employee Benefit Survey (EBS), 87 percent of K–12 teachers participate in a health insurance plan (medical, dental, vision, or prescription drug) through their employer, compared to 80 percent of private-sector professionals (our estimate, adjusting for the part-time percentage). Consequently, the difference between teachers and private-sector workers in employer health cost per participating employee is 16 percent ($9,838 vs. $8,492).

The EBS also collects data on premiums for medical insurance (a slightly narrower category than health insurance). The medical premiums are broken out by single and family coverage, so these data allow us to examine the cost of comparable policies. We find that for single coverage, employer costs for private-sector professionals are 82 percent of those for teachers ($4,496 vs. $5,494), but for family coverage, private-sector costs are 104 percent of those for districts ($11,116 vs. $10,728), slightly higher. This is a notable shift in the last few years. As recently as 2009, the employer cost for single coverage was $1,361 higher for teachers than for private-sector professionals, compared to $998 today, and for family coverage it was $29 higher for teachers instead of $388 lower. This suggests that some school districts have begun to adjust their policies toward private-sector norms.

Click to enlarge

Employee Contributions and Total Premiums

The EBS data on medical insurance also include information on employee contributions. Together with employer costs, these data indicate that, for both single and family plans, total premiums are higher for teachers than they are for private-sector professionals. For single coverage, teachers pay a smaller share (13 percent) than do private professionals (19 percent). For family coverage, teachers contribute more (34 vs. 29 percent), which is enough to cover the higher cost of their plan. In other words, the total premium for teachers’ family coverage is more expensive than it is for private-sector professionals, but the share coming from teachers more than covers the difference (see Figure 2).

In addition to premiums, employees incur out-of-pocket costs, such as deductibles and co-payments. The EBS data indicate that one reason teachers’ insurance plans are more expensive is that features of the plans (such as lower deductibles) reduce out-of-pocket costs. Although it is accurate to say that teachers pay more to get more in the way of family coverage, it is more precise to state that they pay more up front in premiums and then pay less out-of-pocket.

Union vs. Nonunion Employees

The NCS data allow us to compare medical insurance coverage and premiums for union vs. nonunion workers, where union status is defined by whether the employee belongs to a collective bargaining unit. These breakouts are not available for K–12 teachers or private-sector professionals, but they are available for the state and local government (public) sector and the private sector. The comparisons are still informative because teachers’ health care costs track those of the public sector to some extent.

These data indicate that about 95 percent of union workers have access to employer-provided medical insurance in both the public and private sectors, and their participation rate is essentially the same in both sectors (78 to 79 percent). Nonunion workers are less likely than union workers to participate in a medical plan through their employer, in large part because their employer is less likely to offer them one. The difference from union workers is smaller in the public sector, however, where the nonunion participation rate is 68 percent, compared to 48 percent in the private sector.

In the public and private sectors, for both single and family coverage, the employer cost is higher for union workers than for nonunion workers. The total premium is significantly higher in all cases except for family coverage in the private sector, where it is about the same for union and nonunion workers. Finally, employee contributions are lower for union workers, except for single coverage in the public sector.

These patterns are the same for the state and local government sector vs. the private sector, with union and nonunion combined: higher employer costs, higher total premiums, and lower employee contributions, for both types of coverage. The unionization rate is higher for the public sector than for the private sector (50 percent vs. 14 percent in the EBS data), suggesting that unionization explains some portion of each of these patterns (see Figure 3).

Click to enlarge

But these are not the patterns we observed between K–12 teachers and private-sector professionals: they are similar for single coverage but not for family coverage. Whatever impact unionization may have, there are other factors at play.

There is one state in which we have a seemingly natural experiment in changing teacher union strength: Wisconsin. If union strength results in higher employer costs, higher total premiums, and smaller employee contributions, then the removal of teacher health benefits from collective bargaining in Wisconsin might be expected to have the opposite effect: lower employer costs, lower total premiums, and larger employee contributions. This is exactly what happened.

Wisconsin Before and After Act 10

Wisconsin was the first state in the nation with public-sector collective bargaining and has long had one of the nation’s strongest teachers unions. It has also long been a state with very expensive teacher medical insurance. Average district costs in 2011 were $8,311 and $19,356 for single and family coverage, respectively. These costs were about 50 percent and 80 percent higher than the 2011 national averages for teachers, which were $5,500 and $10,723. Although Wisconsin is in a region with higher-than-average medical premiums, this geographic factor accounts for only a minor part of the gap between Wisconsin’s district costs and the national average.

Wisconsin’s high district costs reflected both the choice of expensive plans and low teacher contributions. In 2011, teachers made no contribution at all for single coverage in 43 percent of the state’s districts, nor for family coverage in 31 percent. By comparison, the noncontributory rates in 2011 among teachers in the national data discussed above were 39 percent and 16 percent, respectively. Among private-sector professional employees, the noncontributory rates for single and family plans were lower yet, 17 percent and 9 percent.

Act 10, proposed by Governor Walker and enacted by the legislature in 2011, removed benefits from local collective bargaining, thereby giving districts greater freedom to shop for less-expensive plans and to negotiate premiums. The law also allowed districts to establish higher employee contributions. Among the provisions of Act 10 was a 12 percent floor on the employee contribution rate, which applied directly only to the state-administered plan, but now serves as a benchmark that many school districts have followed.

These changes were intended to achieve savings on district benefit costs, through adoption of plans with lower premiums and increased teacher contributions. We examine the change in medical insurance costs for the school year ending in 2012, the first to be affected by Act 10, using data from the Wisconsin Association of School Boards (WASB). These results may not represent the total impact, as not all districts have renegotiated insurance contracts. Some are under contracts with insurers predating Act 10, including those with pre–Act 10 collective bargaining agreements that have not yet expired.

Click to enlarge

We calculate estimates of yearly changes using only districts for which data are available in consecutive years. The main finding from the WASB data is a sharp drop in employer costs in 2012 after years of steady growth. District payments for their employees’ medical care increased every year from 2003 to 2011. But from 2011 to 2012, average district costs for family coverage fell by an estimated $2,010, while district costs for single coverage declined by $1,042 (see Figure 4).

These figures underestimate the district savings attributable to Act 10, since premiums were steadily rising prior to Act 10 and were expected to continue doing so. When we account for this expected growth (using average growth from 2007 to 2011), we estimate savings of $2,614 for family coverage and $1,304 for single coverage. These estimates represent declines of 13 to 19 percent from the projected district costs for 2012.

Districts saved on teacher medical insurance costs in 2012 for two reasons: reductions in total premiums and increases in the portion paid by teachers. As discussed above, Act 10 did not directly raise teacher contributions, but the 12 percent minimum it established for the state plan set a standard that districts were now free to follow. For single coverage, between 2003 and 2011 the average share of medical insurance paid by teachers drifted up slightly, from about 3 to 4 percent, followed by a jump to more than 10 percent in 2012. Similarly, for family coverage, the average teacher contribution drifted up slightly over the period, to about 5 1/2 percent, and then jumped in 2012 to more than 10 percent. These figures now place Wisconsin in the vicinity of the national average contribution rate for teachers with single coverage of 13 percent, but still far below the average for family coverage of 34 percent.

In dollar terms, teacher contributions for family coverage rose by $939 in 2012, relative to the previous trend, while total premiums for family coverage declined by $1,674. Our estimate of $2,614 for the impact of Act 10 on district costs reflects these changes. The estimated impact on total premiums accounted for two-thirds of the reduction in district costs, and the act’s impact on employee contributions comprised the other third. We find a similar breakdown for single coverage.

These data have two important limitations. First, they likely understate the share of district savings attributable to higher employee costs because some (maybe most) of the reduction in total premiums is due to a rise in employee out-of-pocket payments (such as higher deductibles). Second, these data do not tell us anything about the quantity and quality of health care provided. Efficiency may have been enhanced as employees paid more of the cost and as employers became free to shop around, but we have no hard data on this.

As a check on the WASB data, we examined data from the Wisconsin Department of Public Instruction (DPI) on districts’ fringe benefit costs for teachers. Unlike the WASB data, these data are available for all districts but do not separate out health benefits from other fringe benefits, including retirement contributions, Social Security, and life insurance. The impact of Act 10 captured by these data will therefore include not only the effect on health insurance, but also the shift of about one-half of retirement contributions from employer to employee as mandated by Act 10.

The DPI data show a steady rise in fringe benefit costs from 1998 to 2011, in both dollar amounts and as a percentage of teacher salary, with the latter measure rising from 34 percent to 51 percent over the period. After Act 10, the average benefit rate dropped 8 percentage points to 43 percent. This is still quite high by comparison with the private sector, but markedly reduced. It is likely that at least one-half and perhaps two-thirds of the $4,500  drop in district fringe-benefit costs reflects the shift in retirement contributions, but virtually all of the remainder represents the reduction in district health-benefit costs. Thus the DPI data suggest a drop of $1,500 to $2,200 in average annual district health costs per teacher.

The DPI and WASB estimates show broadly consistent evidence of a large first-year impact of Act 10 on district costs for teacher health insurance, but we can only speculate on what the future effect will be. As mentioned above, some districts have not yet been able to use their new powers because of unexpired collective bargaining contracts or insurance policies, so there are more savings to be had. Many of the underlying drivers of rising health-care costs are independent of Act 10, and over the long term these will push Wisconsin employer costs back up, but from a significantly lower starting point. Moreover, as districts gain more experience in the open health care market, unfettered by collective bargaining, it is possible that they will be able to lower the rate of growth.

It is important to note that even with the dramatic savings from Act 10, district costs and total premiums in Wisconsin are still well above the national average for teachers. Indeed, by some estimates, prior to Act 10, a number of Wisconsin districts had insurance plans that were set to trigger the federal tax on “Cadillac plans” under the Affordable Care Act of 2010, scheduled to begin for 2018. This may still be true. Thus, there will be continuing pressure to reduce costs toward the national average, especially if and when the luxury tax is implemented.

Conclusion

The national data from the Bureau of Labor Statistics indicate that annual employer insurance costs are 26 percent higher for teachers than for private-sector professionals; adjusting for higher participation rates among teachers reduces the difference to 16 percent. Direct estimates of employer costs for medical plans present a mixed picture: higher employer costs for single coverage but not for family coverage. For both categories, total medical premiums are higher for teachers than they are for private-sector professionals, but for family coverage the teachers incur the extra expenditures themselves.

Unionization is associated with higher total premiums, higher employer costs, and lower employee contributions in both the public and private sectors. This suggests that the high unionization rate among teachers plays an important role in their employers’ higher average cost. Equally important, differences in teacher union strength across states help explain the wide variation in employer and employee health-insurance costs. In some nonunion states, teacher medical benefits are not particularly generous, owing to either low-cost plans (e.g., those with high deductibles) or high teacher contributions. In Arkansas, teachers typically pay 65 or 70 percent of the premiums for family coverage (the national average is 34 percent). In other states, with strong unions, such as Wisconsin, district insurance costs can be very expensive. It is in those states that the opportunities for district cost reduction are most promising, as data from Wisconsin so clearly show.

District cost reduction would ideally derive from changes that enhance efficiency, such as greater competition for health insurance. There should be no illusions that such efficiencies will come easily. In all likelihood, a great deal of any district cost reduction will take the form of higher teacher payments toward their health care through higher contributions and increased out-of-pocket expenses. This raises the question of the role of teacher health benefits in the total compensation package. The overall size of the package will continue to be the subject of debate. It is worth briefly commenting, however, on the importance of the structure of the package.

There are three reasons that efficiency might be enhanced by reallocating some of the compensation package from employer-paid health benefits to salary. First, efficiency in health-care expenditures is more likely enhanced when employees pay for services, since price signals provide the consumer with appropriate incentives. Second, shifting compensation back to salary (in the aggregate) provides greater opportunity for districts to use salary differentials to retain and recruit higher-quality teachers. Finally, as a matter of consumer choice, not all employees may want their employers to devote, say, $20,000 out of a $70,000 compensation package to medical insurance. Take-up rates well below 100 percent suggest that many teachers ascribe less value to the medical benefits offered than they cost. Thus, both efficiency (in attracting recruits) and equity (toward non-participants) might be enhanced by such a shift. Employers can offer greater choice among health plans of varying cost, with lower subsidies, fixed in size, and higher salaries that allow employees to choose how much they want to spend on higher-cost plans. As districts under fiscal distress increasingly turn to cost-cutting measures, such potential efficiency enhancements will become all the more important.

Robert Costrell is professor of education reform and economics at the University of Arkansas and fellow at the George W. Bush Institute. Jeffery Dean is distinguished doctoral fellow at the University of Arkansas. This paper is drawn from a chapter in A Bigger Bang for Education’s Bucks (George W. Bush Institute, forthcoming).

In the following essays, we identify some of the key findings from the sixth annual Education Next-PEPG Survey, a nationally representative sample of U.S. citizens interviewed during April and May of 2012 (for survey methodology, see sidebar). Highlights include

• the Republican tilt of the education views of independents

• the especially high marks that Hispanics give their public schools

• strong support among the general public for using test-score information to hold teachers accountable

• lower confidence in teachers than has previously been reported

• the public’s (and teachers’) growing uneasiness with teachers unions

• the shaky foundations of public support for increased spending

• majority support for a broad range of school choice initiatives.

In addition to the views of the public as a whole, in this year’s survey special attention is paid to Hispanics, African Americans, parents, and teachers, all of whom were oversampled in order to obtain a sufficient number of observations. And in an effort to assess the sensitivity of respondents’ opinions to information and question wording, we embedded in this survey, as we have done in previous ones, various experiments. Responses to all questions are posted on our website, educationnext.org.

Independents lean Republican in their views of teachers unions and school spending—and support private school choice.

With Barack Obama and Mitt Romney running neck and neck, the nation’s eyes are trained on independent voters, who will likely decide the presidential election. And in the days leading up to the national conventions, education policy, though hardly at the top of the public agenda, did assume a more prominent role in both campaigns. Which candidate is best positioned to use education to bring undecided voters into the fold? The answer may be surprising.

Just one-third of independents report that President Obama has done an “excellent” or “good” job of handling education issues, while the rest assign him a “fair” or “poor” rating. And on the education policy issues that most clearly divide the parties—the role of teachers unions and support for school spending—the views of independents hew closer to those of Republicans than of Democrats. Moreover, independents are more supportive than members of either party of expanding private school choice for disadvantaged students, the centerpiece of Governor Romney’s proposals for K–12 education reform.

Whereas 25 percent of respondents to the EdNext-PEPG survey report that they are Republicans and 34 percent say that they are Democrats, fully 41 percent claim no affiliation with either major party. Of this group, 52 percent claim that they lean Democratic, while just 40 percent lean Republican. On key education issues, however, these independents express views that better align with Republicans.

No single education issue divides Republicans and Democrats more sharply than the role of teachers unions (see Figure 1). Seventy-one percent of Republicans report that the teachers unions have a generally negative effect on schools, as compared to just 29 percent of Democrats. Though independents come down in between, a majority of them (56 percent) agree with Republicans that unions have a negative effect.

Republican and Democratic voters also diverge in their preferences on school spending and teacher salaries. Figure 2 shows that when not provided with information about current spending levels, 79 percent of Democrats say that spending on public schools in their local district should increase, as compared with 50 percent of Republicans. Among independents, 57 percent support increased spending, again placing them closer to Republicans in their view of the issue. And when respondents are informed about current spending levels, the gap between Republicans and independents vanishes: 39 percent of both groups support spending increases, compared to just 51 percent of Democrats.

The same pattern holds for teacher salaries: when respondents are not provided with information about current salary levels, 60 percent of independents support increasing teacher salaries, placing them closer to Republicans (54 percent of whom support increases) than to Democrats (75 percent). Providing information on current teacher salaries in their state reduces support for salary increases among independents to 34 percent—exactly the same as among Republicans. Information also shrinks the share of Democrats supporting salary increases to 41 percent.

Governor Romney has made the expansion of school choice for disadvantaged students central to his campaign, calling for the expansion of the Washington, D.C., voucher program and for allowing low-income and special education students to use federal funds to enroll in private schools. It is perhaps surprising, then, to find that Republicans are less supportive of this concept than are Democrats (see Figure 3). Just 42 percent of Republicans express support for the idea, compared to 52 percent of Democrats. Voucher support among independents appears to be as high as (or greater than) it is among Democrats, at 54 percent.

Hispanics like public schools but not all union demands in contract negotiations.

Increasingly, both the Republican and Democratic parties have sought ways to court Hispanics. Though they lean Democratic—63 percent of Hispanic adults approve of the way President Barack Obama is handling his job as president—they are not as blue as is the African American community, 92 percent of whom give Obama a thumbs-up.

Those seeking the Hispanic vote in 2012 should know that education is an issue that resonates with the Latino community. Almost 60 percent of those we surveyed say they are “very” or “quite a bit” interested in education issues, as compared to less than 40 percent of African American and white voters.

On many topics—including school vouchers, charter schools, digital learning, student and school accountability, common core standards, and teacher recruitment and retention policies—the views of Hispanic adults do not differ noticeably from those of either whites or African Americans.

But in certain domains—estimates of school costs and school quality, support for teachers unions, teacher tenure, and teacher pensions—the views of Hispanics differ rather substantially. Their judgment of the American school is generous, perhaps because they compare public schools in the United States to much less effective institutions in Mexico, Cuba, and other parts of Latin America. They also underestimate the costs of running public schools, though they revise their thinking rather substantially about the merits of spending increases once they learn the facts. They are less supportive of unions and union demands than are African Americans.

Nearly 40 percent of Hispanic adults give the nation’s public schools a grade of an “A” or a “B” on the traditional scale used to evaluate schools (see Figure 4). When asked about the public schools in their community, no less than 55 percent give such favorable assessments. By comparison, whites and African Americans express significantly less enthusiasm about the nation’s schools. Less than 20 percent of whites and African Americans accord the nation’s schools an “A” or a “B,” and only around 40 percent give the schools in their community one of these two top grades.

Hispanic respondents think American students perform better academically than is actually the case. American 15-year-olds ranked no better than 25th among the 34 developed democracies participating in the latest round of international tests. Yet nearly 45 percent of Hispanics say student math performance in the United States ranks among the top 15 countries in the world in math. Only 30 percent of white and African American respondents place the United States that high.

Hispanic respondents also think the public schools cost a lot less than they actually do. While annual per-pupil expenditures run around $12,500, Hispanics, on average, estimate their cost at less than $5,000. Whites and African Americans estimate the costs to be more than $7,000.

The same goes for teacher salaries, which average about $56,000 a year. On average, Hispanics think teachers are paid little more than $25,000 a year; blacks, on average, think they are paid around $30,000 a year; and whites estimate salaries at $35,000.

When told just how much schools cost, however, Hispanic respondents adjust their thinking quite dramatically. When informed about actual per-pupil expenditures, Hispanics’ support for higher taxes to fund spending increases drops from 46 percent to 25 percent. When given the actual amount teachers receive, their support for higher salaries plummets nearly in half—from over 60 percent to little more than 30 percent.

Although learning the truth about costs and salaries has a similar impact on white opinion, African Americans remain more committed to higher spending. Thirty-seven percent of African Americans favor higher taxes, even when told how much is currently being spent, only a slight dip from the 42 percent favorable when that information is withheld. When given the facts about teacher salaries, African American support for higher salaries drops 20 percentage points—from 74 percent to 54 percent.

Like other ethnic groups, Hispanics do not appear especially sympathetic to teachers union demands in collective bargaining negotiations. Sixty-two percent of Hispanic adults think teachers should pay 20 percent of their pension and health care costs, as do 56 percent of African Americans.

By an overwhelming margin (87 percent), Hispanic respondents favor proposals to condition teacher tenure on their students’ making adequate progress on state tests. Whites and African Americans also favor such proposals but not to the same degree (75 percent and 80 percent, respectively). When it comes to whether teachers unions are playing a more positive or a more negative role in their local community, Hispanic adults come out in the middle—at 59 percent in support, they are more supportive than whites (45 percent) but less supportive than African Americans (75 percent).

Use test scores for evaluations, says the public (but not the teachers).

Teachers have long been paid primarily on the basis of their academic credentials and years of experience, creating in most parts of the country a lockstep pay scale that does not account for a teacher’s classroom performance. This approach is often justified on the grounds that it precludes favoritism on the part of principals, school board members, and other administrative officials.

As teacher effectiveness has become an increasingly visible policy issue, standard approaches to salary and tenure decisions are undergoing substantial change. More than 20 states now require that student test-score gains be used in key personnel decisions, often including tenure and salary determinations. Four states go so far as to prohibit a teacher from receiving a top rating if students do not exceed a certain level of accomplishment, while another 10 require that achievement gains constitute at least 50 percent of each teacher’s evaluation.

Is the public onboard with these changes? And what do teachers think about them? To find out, we randomly divided those interviewed into two groups (see Figure 5). The first group was given a stark choice: How much weight should be given to test scores and how much should be given to principal recommendations?

Given this simple dichotomy, the public says test-score gains should be given more than half the weight (62 percent) in making salary and tenure decisions. Teachers, by contrast, are prepared to place only a quarter of the weight (24 percent) on this information, with the other three-fourths of the weight being given to principal recommendations.

The second half of the sample was asked a more complex question, which required giving weights to test scores and evaluations from four different sources: principals, parents, students, and fellow teachers.

When the question was posed this way, the public and the teachers once again disagree. The public would place about one-third of the weight (32 percent) on test scores, but teachers would assign them less than one-fifth (19 percent). Conversely, teachers would give principal recommendations nearly half the weight (44 percent), while the public would give their recommendations less than one-quarter (23 percent).

Perhaps surprisingly, teachers are unenthusiastic about being evaluated by their fellow teachers. Like the rest of the public, they divide up the remaining weight more or less equally among the three remaining sources of evidence (students, parents, and fellow teachers).

An even bigger gap between teachers and the public emerges on the desirability of releasing information about teacher performance to the public at large. In both New York City and Los Angeles, newspapers have published such information, provoking an outcry among teachers, who felt their privacy had been invaded. When we asked respondents about this as a general practice, 78 percent of the public expresses support, compared to just 33 percent of teachers (see Figure 6).

When given the option of expressing neutrality on the issue (as another randomly chosen half of the sample was), 60 percent of the public still says it supports the publication of information about teacher performance, while only 13 percent is opposed, the remaining 27 percent taking the neutral position. Teacher opinion is almost the mirror image. Fifty-four percent oppose making information on test-score impacts publicly available, 30 percent express support, with the remaining 16 percent not taking a clear position either way.

Are teachers unions undermining teacher popularity?

Teachers have long held a cherished place in American popular culture. In such films as Blackboard Jungle, Stand and Deliver, and Dead Poets Society, Hollywood has highlighted the power of teachers to utterly transform the lives of their students.

But is this now changing? Are Waiting for Superman, Bad Teacher, and Won’t Back Down (forthcoming “A Takeover Tale,” cultured, Winter 2013) harbingers of a new, more skeptical depiction of teachers? At first, it would seem that public trust in teachers is widespread. When we asked half of the respondents in our survey whether they “have trust and confidence in the men and women who are teaching children in the public schools,” no less than 72 percent say “yes” (see Figure 7). This is almost exactly what Phi Delta Kappan (PDK), a publication sympathetic to teachers unions, found in its 2012 poll and about the same as in PDK polls in previous years.

When we expand the possible response categories, however, a somewhat different picture emerges. Only 4 percent of the American public has “complete” trust and confidence in teachers, and just 38 percent has “a lot” of trust and confidence in them. Meanwhile, 49 percent has “some” trust and confidence, and 9 percent has “little” trust and confidence. In other words, 58 percent of those surveyed express less than “a lot of trust and confidence” in the teaching force.

Since this is the first time the public has been asked to break its assessment of teachers into four categories, we cannot document any trends over time. But we do know that public opinion toward teachers unions—and teachers’ opinions of them, too—has turned in a negative direction. The portion who thinks that teachers unions have had a positive effect on their local schools has dropped by 7 percentage points over the past year. Among teachers, the downward shift is no less than 16 percentage points.

In this year’s survey, as we have done in the past, we asked the following question: “Some people say teachers unions are a stumbling block to school reform. Others say that unions fight for better schools and better teachers. What is your opinion? Do you think teachers unions have a generally positive view on your local schools, or do you think they have a generally negative effect?” Respondents could choose among five options: very positive, somewhat positive, neither positive nor negative, somewhat negative, and very negative.

In our polls from 2009 to 2011, we saw little change in public opinion. Around 40 percent of respondents took the neutral position, saying that unions had neither a positive nor a negative impact. The remainder were divided almost evenly, with the negative share just barely exceeding the positive.

This year, however, the teachers unions lost ground. While 41 percent of the public still takes the neutral position, the portion with a positive view of unions dropped 7 percentage points in the last year, from 29 percent to 22 percent.

The drop is even greater, in both magnitude and significance, among our nationally representative sample of teachers. At a time when, according to education journalist and union watchdog Mike Antonucci, the National Education Association has lost 150,000 members over the past two years, and projects to lose 200,000 more members by 2014, teacher discontent appears to be rising. Whereas 58 percent of teachers had a positive view of unions in 2011, only 43 percent do so in 2012. Meanwhile, the percentage of teachers holding negative views of unions nearly doubled during this period, from 17 percent to 32 percent.

But when that same question was posed in either/or terms to the public as a whole, respondents split down the middle: 51 percent say unions had a negative impact, while 49 percent say their effect was positive. Teachers, meanwhile, offered a more positive assessment. When forced to choose between just two options, 71 percent of teachers claim that unions are a force for good, whereas 29 percent see them as a stumbling block to reform.

Support for school spending is shaky.

With the U.S. economy trying to crawl back to recovery, an unemployment rate above 8 percent, and state and local governments facing the prospect of insolvency, many school districts have found it necessary to cut expenditures and personnel. In California, the cities of Stockton and San Bernardino have declared bankruptcy. In Michigan, the financially bankrupt Muskegon schools have been handed over to a for-profit charter organization. Cuts in arts programs and extracurricular activities are becoming commonplace. Nationwide, the number of school employees has drifted downward by as much as 5 percent in the past few years.

Still, the American public continues to support increasing spending on local public schools. Or at least it appears to do so (see Figure 8). Sixty-three percent of the general public says it prefers an increase in school expenditures in the local district, well up from levels in 2007 when only 51 percent of the public called for expenditure increases. Not surprisingly, teachers are even more enthusiastic about increasing expenditures, 68 percent of whom like the idea.

When one investigates the issue just a bit further, however, fractures can be detected in the public’s willingness to spend more on public schools. Though most Americans still offer their support for spending increases in the abstract, their enthusiasm ebbs rather substantially when the taxes needed to pay for the increased expenditures are broached and when information about actual expenditures and salaries is provided.

Part of the explanation for this is the widespread ignorance on the part of the general public about just how much already is spent on public schools. When asked to estimate per-pupil expenditure in their district, Americans guess that expenditures are about $6,500 annually, when in fact they are around $12,500. That is only a slightly better set of estimates than the ones given in 2009, when Americans thought $4,231 was being spent per pupil and the reality was closer to $10,000 (see “Educating the Public,” features, Summer 2009).

When respondents are told the correct figure, support for spending on public schools shifts sharply downward. Support for increased spending on our standard question drops by 20 percentage points, a much bigger drop than what was observed in 2009, when support for increased spending fell only 8 percentage points (from 46 percent to 38 percent).

In another sign of less-than-wholehearted support for an education spending spree, only 35 percent of the public says taxes should increase to fund the schools. Support drops by another 11 percentage points—to just 24 percent—when those interviewed were first told how much was currently being spent.

Teachers, who stand to benefit from increased expenditure, remain committed to more spending when told the realities of the expenditure situation in their district. Their support slips only 8 percentage points from the high of 68 percent when no information is supplied about current expenditures. But even teachers are 17 percentage points less likely to support higher taxes to fund increases in education spending.

When the subject turns from per-pupil expenditures to teacher salaries, the same pattern emerges (see Figure 9). When asked without any accompanying information, nearly two out of three Americans think that teacher salaries should go up. Among teachers, support for a salary boost registers at no less than 85 percent.

As they do on per-pupil expenditures, however, Americans hold markedly inaccurate views about actual teacher salaries. When asked to hazard a guess, Americans estimate that public school teachers in their states receive, on average, about $36,000 in salary annually. The true figure, even without accounting for benefits, pensions, and the like, sits at about $56,000 nationwide.

Support for higher salaries plummets, however, when Americans are told how much teachers actually make in their states. Of those given the facts, only 36 percent favor an increase, which amounts to a whopping 28-percentage-point decline from the 64 percent favoring an increase when no information is supplied.

When teachers were given accurate information about salary levels in their state, their support slips by only 10 percentage points, probably because they are thinking about their own paycheck. Also, they have a better sense of teacher salaries in their state than the public has, estimating them to be about $44,000 annually.

Is public support for charters really that much higher than for vouchers and tax credits?

As a policy reform, school choice shows no signs of slowing. The number of states with school-voucher and tax-credit programs has escalated since 2010, the number of students attending charter schools climbs steadily year by year, and new technologies for online learning are being promoted by a cascade of new entrepreneurs.

The contours of elite debate about school choice, however, are not replicated in the larger public. While charter schools and digital learning are thought to be the safest choice options for political elites to promote, tax credits are even more popular than charters, and vouchers, the most controversial proposal, also command the support of half the population when the idea is posed in an inviting way.

Vouchers and tax credits. When it comes to school vouchers, apparent levels of public support turn on the wording of the question. For the past two years, PDK has asked whether respondents “favor or oppose allowing students and parents to choose a private school to attend at public expense.” Even with the rather loaded “at public expense” phrasing, PDK reported that support shifted upward from 34 percent to 44 percent between 2011 and 2012.

If one asks the question in a more inviting manner, as we have, support jumps further still (see Figure 10). Told about a proposal “that would give low-income families with children in public schools a wider choice, by allowing them to enroll their children in private schools instead, with government helping to pay the tuition,” 50 percent of the American public comes out in support and 50 percent expresses opposition.

Still, support for vouchers does not match public willingness to back tax credits, even though most economists think the difference between vouchers and tax credits more a matter of style than substance. Nearly three-fourths (72 percent) of the public favors a “tax credit for individual and corporate donations that pay for scholarships to help low-income parents send their children to private schools.” We find little evidence that support for tax credits has changed significantly since 2011.

Charters. Figure 11 shows that when given a choice of supporting or opposing charter schools, 62 percent of the public says it favors ”the formation of charter schools,” nearly identical to what PDK finds (66 percent favoring ”the idea of charter schools”). Support for charters, however, is softer than it might seem. When respondents are given the opportunity to take a neutral position that neither supports nor opposes charters, no less than 41 percent choose that option. Among the remainder, the split is nearly three to one in favor of charters.

Meanwhile, public knowledge about charters remains as impoverished as ever. As our survey did two years ago, we asked respondents a variety of factual questions: whether charter schools can hold religious services, charge tuition, receive more or less per-pupil funding than traditional public schools, and are legally obligated to admit students randomly when oversubscribed. We found little change in the level of public information over the past two years. Large percentages of respondents still say they don’t know the answers to these questions. Among those who hazard a guess, they are as likely to give the wrong answer as the correct one. Although teachers do a better job of accurately identifying the characteristics of charter schools, even a majority of teachers get many of the answers wrong or say they don’t know.

Online education. As major universities—Harvard, Stanford, MIT, and others—are joining community colleges and state universities in a nationwide dash toward online learning in higher education, many states are exploring ways of incorporating new digital technologies into secondary schools.

A substantial share of both the public and the teaching force seems ready to consider the expansion of online learning. When asked if high school students should be allowed to take “approved classes either online or in school,” opinion splits down the middle, with a bare majority (53 percent to 47 percent) favoring the idea. Teachers are more enthusiastic, among whom no less than 61 percent feel students should be given an online option.

The public, however, is not equally enthusiastic about all uses of the online tool, nor is support for the idea gaining strength. The most popular uses are for rural education and advanced course taking. Fifty-eight percent of those surveyed think students in rural areas should have online opportunities, with only 14 percent opposing the idea. That is down modestly from the 64 percent who supported this use in 2008. Similar percentages of support and opposition are expressed for advanced courses taken online for college credit. But once again, levels of support have slipped since 2008 (from 68 percent to 57 percent in 2012).

Less popular are online courses for dropouts and home schoolers. Only 44 percent favor, and 30 percent oppose, using public funding to help dropouts take courses online. Many home schoolers find online courses to be a valuable tool, but the public remains dubious. Only 28 percent favors public funding for such uses, and 38 percent opposes it. Those percentages have not changed materially since 2008.

William G. Howell is professor of American politics at the University of Chicago. Martin R. West is assistant professor of education at the Harvard Graduate School of Education and deputy director of Harvard’s Program on Education Policy and Governance. Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution.

Silly question—it might seem. How could someone be expected to teach piano if they do not know how to play themselves? Yet that is what the public schools are about to ask of teachers more generally. In two years, most public schools will administer new student assessments in reading and math, pegged to the higher academic standards of the Common Core. Although performance standards—how high a student must score to be proficient—are still being set, the bar will likely be at the National Assessment of Education Progress (NAEP) “proficient” level. This will be a huge challenge for students, particularly in high school, as NAEP proficiency is the equivalent of an SAT score in verbal and math of nearly 1200—or 200 points higher than the average student taking the SAT today achieves.

The Common Core standards will be an even greater challenge for America’s teachers. The average SAT score of public elementary teachers today is less than 1000; for secondary teachers a bit higher. In effect, our public schools are asking teachers to help students reach standards that are far above the standards that they have achieved  themselves. A 200 point gap on the SAT scale is huge; statistically, it amounts to two standard deviations. It’s like saying that teachers achieving at the 50th percentile need to help students achieve at the 90th percentile—like asking ordinary pianists to train virtuosos.

Of course, America has many brilliant teachers. And aptitude is but one measure of teaching ability. Lots of teachers will be able to help their students achieve the new higher standards. But numerous teachers are also below average in both aptitude and teaching ability. So, the general point holds: America has not built a teaching force anywhere near the standards that are being set for tomorrow’s students. Our presidential candidates are fond of saying that the U.S. will be number one in the world in education—that our students will achieve with the best in the world. That will never happen if our teachers are not also the best in the world.

In my new book, I make precisely this argument—and suggest how we can do better. The Best Teachers in the World: Why We Don’t Have Them and How We Could observes, as is now customary, that nothing is more important for student achievement than teacher quality. The quality of a nation’s schools cannot rise above the quality of its teachers. The book argues—and this is far from customary—that the nation is just not serious about teacher quality. Our current policies have not a prayer of producing the best teachers in the world. We spend the most per pupil on public education in the world, but over 20 nations compensate teachers more highly. We draw teaching candidates on average from the least able high school graduates. We train teachers in non-selective colleges and universities. We do not recognize merit on the job. These are not practices that attract and retain the best and the brightest in teaching.

Doing better will require much more than currently favored reforms. New teacher evaluation systems, the bipartisan rage today, are a step in the right direction, but a small step. We can’t evaluate and fire our way to a great teaching pool. We must elevate the status of the profession, the attractiveness of the work, and the compensation for doing it. We need to recognize that quality teaching is intellectual demanding, requiring deep academic knowledge, research-based training, and rigorously guided practice. Teaching must uphold standards commensurate with those we aim to hold for our students.

But these are all just platitudes—and familiar ones—if not backed by measures that can truly move the profession. The book makes three recommendations: a much smaller, selective, intellectually engaged, and better compensated teaching force supported by technology; an open, transparent, and accountable system of preparation and professional development that drives out inferior providers and rewards success; and increased responsibility for teacher development in the hands of principals, who may be the strongest determinant of teacher quality on the job. These reforms are decidedly not more of the same. Each would be devastating to some element of the status quo. Each will be fought tooth and nail by vested interests. So be it. Our willingness as a nation to fight for the best teachers in the world is the best measure of our commitment to the future achievement of our children.

-John Chubb

Many parts of the schooling model are also still evolving as the school learns what does and does not work. Uncertainty exists, and teachers are both teaching amidst the uncertainty and helping to create and refine the school model itself on the fly. Because new innovations rarely emerge fully baked and launch with perfect success, this is both natural and good.

One of the school leader’s questions in this first-year of operations was whether he had hired the right teachers—and what profile of a teacher would be right for the model. His early hypothesis was that the right teachers would have the same profile and skills as those who were successful in other “No Excuses” charter schools, but he said he wasn’t sure if he should be looking for other attributes as well.

I suspect that his initial hypothesis has some merit, but I think one of the theories that we call the “Schools of Experience” could be helpful in refining it further.

As Clayton Christensen, James Allworth, and Karen Dillon recount in How Will You Measure Your Life, most companies that hire assume implicitly that there are some innate talents that candidates either have or don’t have that can predict success. In essence, employers list the skills correlated with success and search for those candidates with them. The surest sign of this talent they think is in the resume: if a candidate shows success after success, she must just have the “right stuff” to be successful in any job. If this were true, however, then why do we so often see executives with a successful track record in one company fail in another?

Morgan McCall, a professor at the University of Southern California, developed a different view that moves beyond the skills correlated with success to a circumstance-based theory. His model asks whether someone has actually wrestled with a problem similar to the one she will need to wrestle with in the new job. In essence, it looks to see if potential employees have taken different “courses” in their “schools of experience” that will prepare them to tackle and succeed in the new job.

As Christensen et al write, “McCall’s thinking is not based on the idea that great leaders are born ready to go. Rather, their abilities are developed and shaped by experiences in life. A challenging job, a failure in leading a project, an assignment in a new area of the company—all those things become ‘courses’ in the school of experience. The skills that leaders have—or lack—depend heavily on which ‘courses,’ so to speak, they have and have not taken along the way.”

Through the lens of McCall’s theory, the job when hiring shifts from identifying candidates with stellar resumes to asking whether the potential employees have had the right experiences that prepare them to be successful in the job.

An example helps make the pitfalls of the first approach and the merits of the second clearer. When launching an internal startup, many established companies will bring on board star performers from the parent company—people with stellar resumes. But often these people will not have had any experience in launching a new venture. None have had to adjust a strategy when the first one didn’t work because they were in a company with established processes and a successful strategy. Staffing the new venture with some people who have had the right schools of experience increases the odds of success.

Turning back to the blended-learning school at hand, the school is a startup in which there aren’t existing processes to get things done and people are inventing the school’s culture as they teach. This suggests another criteria for the school leader to be including as he hires teachers: looking for people who have had experience in uncertain situations where there were not firm rules to follow—and they had to create and establish new processes and tweak them as they went along. High flyers who have high expectations for students but who have never been in an organization with an emergent strategy might not be ideal off the bat. Of course, as the school evolves and is hopefully successful, processes will emerge that define how to do things and what the school’s culture is. As this happens, the ideal schools of experience a candidate will need will change as well.

This lesson is not just for this particular school leader. As innovation increases in education in the years ahead, the way we prepare some teachers may need to change as well. Some will need a new “course” to prepare them to succeed in the new and uncertain schooling models being developed and tweaked in the years ahead. The life courses that allow entrepreneurs to succeed may also be important for teachers.

-Michael Horn

This blog entry first appeared on Forbes.com

In 1970 many disabled and mentally handicapped students were denied access to public education. Today these students are guaranteed a public education until the age of 22. Also in 1970, about 5% of the U.S. population was foreign born, compared with about 20% today. Many of these children enter the education system with limited English skills and are provided services to improve their mastery of English. Such services were unheard of in many parts of the country even 20 years ago.

It is obvious from these statistics that many more special-education teachers and English-language specialists are counted in the teaching profession now as compared to 1970. Mr. Greene claims that math and reading scores of 17-year-olds are unchanged since 1970. I would submit that the teaching resources devoted to students, excluding teachers of special education and limited-English speakers, is close to unchanged since 1970.

There is a plausibility to this argument, but special education and ELL can neither account for the 50% increase in teachers nor can they be ignored when considering the stagnation in student achievement. Special education teachers constitute about 14% of the teaching work force and disabled students constitute about 13% of the student population. So, if we imagine, as the letter writer does, that many of these disabled students were denied access to public education, then the addition of teachers was roughly commensurate with the addition of disabled students. Excluding all disabled students and teachers, the reduction in student-teacher ratios between 1970 and 2012 would still have been roughly from 22 to 15. If you wanted to use as the starting point 1980, 5 years after the start of federally mandated special education, the ratio still drops from 18.6 to 15.2.

But of course not all disabled students were denied access to schools before federal legislation. Outside of the most severely disabled, the bulk of students now classified as disabled would have been present in school in 1970; they just weren’t being served very well. So, if we added a large number of special education teachers to better educate students who were always present but who we now consider disabled, it should have resulted in much better outcomes for those students. But overall outcomes are flat.

There is a disturbing habit among people who make the argument represented in the WSJ letter to act as if special education is a black hole from which no progress can or should be expected. Yes, they say, we hired more teachers, but that was for more special education students and you couldn’t expect that to result in any progress. But this is entirely wrong. Special education can and should result in greater academic achievement, so even teachers added in that category should be contributing to better aggregate outcomes.

All of these arguments also hold true for ELL except that ELL is much smaller and involves fewer teachers than special education. A critic could note that the world has given the US public education system more ELL students because of higher immigration, although the same cannot really be said of special education. Other than the exclusion of severely disabled students, whose numbers are quite small, the distribution of disabilities in the public school student population should be roughly the same today as it was back then given that most disabilities are genetic in their origin. It’s just that we didn’t serve many of those students well in the past and therefore should expect that achievement should be rising as we devote more resources to them. More teachers should be producing more achievement.

And yes, more ELL students might require more teachers to produce the same achievement. But in other ways our student population has become easier to educate. Unless students have become significantly more difficult to educate across all dimensions, it’s not possible to explain away the facts that we have 50% more teachers without any meaningful improvement in outcomes.

Several years ago Greg Forster and I addressed this in our Teachability Index, in which we tracked 16 indicators of the advantages or disadvantaged that students bring to school and found that overall students are somewhat less challenging to educate now than they used to be. And for a forthcoming book I have updated and improved upon that analysis and still find that students are somewhat easier to educate, so it should not require many more teachers to get the same results.

We can’t blame special ed and ELL to account for the lack of productivity in education as we’ve hired more teachers. The problem is that we’ve ignored the trade-offs between teacher quantity and teacher quality.

-Jay Greene

In the largest study of instructional practice ever undertaken, the Bill & Melinda Gates Foundation’s Measures of Effective Teaching (MET) project is searching for tools to save the world from perfunctory teacher evaluations. In our first report (released in December 2010), we described the potential usefulness of student surveys for providing feedback to teachers. For our second report, the Educational Testing Service (ETS) scored 7,500 lesson videos for 1,333 teachers in six school districts using five different classroom-observation instruments. We compared those data against student achievement gains on state tests, gains on supplemental tests, and surveys from more than 44,500 students.

So far, the evidence reveals that my son’s strategy when choosing a team of superheroes makes sense for teacher evaluation systems as well: rather than rely on any single indicator, schools should try to see effective teaching from multiple angles.

Achievement Gains and Predictive Power

A teacher’s track record of producing student achievement gains does one thing better than any other measure (even if it does so imperfectly): it signals whether a teacher is likely to achieve similar success with another group of students. Not surprisingly, this is particularly true when the outcomes are being measured with the same test. In comparison to classrooms of students elsewhere with similar baseline achievement and demographics, a teacher’s achievement gain in one year is correlated at a rate of .48 in math and .36 in English language arts (ELA), with the average growth of students in another year. Such volatility notwithstanding, a track record of achievement gains is a more reliable predictor of the gains of future students than classroom observations or student surveys.

Unfortunately, state tests do not measure every outcome parents and taxpayers (and students) expect from schools, and cost is a factor in determining what gets measured. Given the higher cost of scoring constructed-response items, many states rely heavily on multiple-choice items to measure student achievement. The shallowness of the items on the test does not necessarily translate into shallow teaching. (For example, although spelling can be tested with low-cost items, a language teacher may find it useful to briefly summarize the reach of the Roman Empire while explaining the appearance of many Latin roots in the English language. A conceptual understanding can provide a framework for learning the fact-based knowledge examined on state tests.) In our study, the teachers with larger gains on low-cost state math tests also had students with larger gains on the Balanced Assessment in Mathematics, a more-expensive-to-score test designed to measure students’ conceptual understanding of mathematics.

Our results did raise concerns about current state tests in English language arts, however. Current state ELA assessments overwhelmingly consist of short reading passages, followed by multiple-choice questions that probe reading comprehension. Teachers’ average student-achievement gains based on such tests are more volatile from year to year (which translates to lower reliability) and are only weakly related to other measures, such as classroom observations and student surveys.

We supplemented the state tests with an assessment requiring students to read a passage and then write short-answer responses to questions about the passage. The achievement gains based on that measure were more reliable measures of a teacher’s practice (less variable across different classes taught by the same teacher) and were more closely related to other measures, such as classroom observations and student surveys. In order to provide clearer feedback on teacher effectiveness, states should hasten efforts to add writing prompts to their literacy assessments.

We expect schools to do more than raise achievement on tests, however. Parents hope their children will learn other skills that lead to success later in life, such as an ability to work in teams and persistence. Just because these skills are hard to measure and are not captured directly on any state test need not imply that effective teachers are ignoring them. Indeed, building student persistence may be an effective strategy for raising achievement on state tests. Recent evidence suggests that the teachers with larger student-achievement gains on state tests also seem to have students with greater long-term career success. As Raj Chetty, John Friedman, and Jonah Rockoff reported recently (see “Great Teaching,” research, Summer 2012), being assigned to a teacher with a track record of student achievement gains is associated with higher earnings and rates of college going.

In sum, the “superpower” of the student achievement–gain, or growth, measure is its ability to “foresee” the achievement gains of future students and future earnings of students. But, like my son’s flawed heroes, it also has drawbacks. One key weakness of the student achievement–gain measure is the limited number of grades and subjects for which assessment data are currently available. In many school districts, fewer than one-quarter of teachers work in grades and subjects where student achievement gains are tracked with state assessments.

In addition, student achievement gains provide few clues for what a teacher might do to improve her practice. A performance-evaluation system should support growth and development not just facilitate accountability. Teachers need to be able to see their own strengths and weaknesses clearly and recognize where they need to hone their skills. That is not information a value-added measure can provide.

Classroom Practice

One way to develop such feedback is by means of classroom observation by a trained adult. Over the years, education researchers have proposed a number of instruments for assessing classroom instruction. To test these approaches, the Educational Testing Service trained more than 900 observers to score 7,500 lesson videos using different classroom-observation instruments. Depending on the instrument, observers received 17 to 25 hours of initial training. At the end of the training, observers were required to score a set of prescored videos. If the discrepancy between their scores and the master scores was too large, they were prevented from participating. (Across all the instruments, 23 percent of trained raters were disqualified because they could not apply the standards accurately.)

Every video was rated at least three times: once using the Framework for Teaching, developed by Charlotte Danielson; once using the Classroom Assessment Scoring System (CLASS), developed by Bob Pianta and Bridget Hamre at the University of Virginia; and a third time using a subject-specific instrument. The math lessons were scored using the Mathematical Quality of Instruction (MQI), developed by Heather Hill at Harvard. The ELA videos were scored on the Protocol for Language Arts Teacher Observation (PLATO), developed by Pam Grossman at Stanford. Finally, the National Math and Science Initiative scored a set of 1,000 math lessons, using the Uteach Observation Protocol.

I’m often asked, “Do you really think you can quantify the ‘art’ of teaching?” I argue that is not the right question. Of course, it is impossible to codify all the nuances that go into great teaching. But an instrument need not capture all the dimensions of great teaching in order to be useful. Each of the classroom-observation instruments proposes an incomplete but discrete set of competencies for effective teaching and provides a description of differing performance levels for each competency. The instruments’ usefulness depends not on their completeness but on the demonstrated association between the few discrete competencies and student outcomes.

For example, one of the competencies highlighted by the Framework for Teaching is questioning skill. A teacher would receive an “unsatisfactory” score if she asked a series of yes/no questions, posed in rapid succession, to the same small group of students. A teacher would receive an “advanced” score on questioning skill if she asked students to explain their thinking, if the questions involved many students in class, and if the students began asking questions of each other. Depending on the instrument, observers tracked 6 to 22 different competencies, including “behavior management,” “time management,” and “engaging students in learning.”

The goal of classroom observations is to help teachers improve practice, and thereby improve student outcomes. A classroom-observation system that bears no relationship to student outcomes will be of no use in improving them. As a result, we tested the relationship between classroom observations and a teacher’s average student-achievement gains. All five of the instruments yielded scores that were related to student achievement gains, in the classroom of students where the teacher was observed as well as in other classrooms of students taught by the same teacher.

In theory, classroom observations allow teachers to be more discerning about their own practice, and their improved practice will yield improved student outcomes. This is as yet a “potential superpower” of classroom observations, since there’s not a lot of evidence that providing such feedback leads to improved student outcomes.

The poor track record of professional-development interventions provides ample reason for caution. Yet there is some reason for optimism. Eric Taylor and John Tyler report that midcareer teachers in Cincinnati saw significant improvements in student outcomes in the years during and after intensive observations (see “Can Teacher Evaluation Improve Teaching?” research, forthcoming Fall 2012). In fact, the gains in student outcomes were similar in magnitude to those seen during the first three years of teaching. It may be that professional growth must begin with an individualized (and honest) assessment of a teacher’s strengths and weaknesses. We need better evidence in the coming years on the types of feedback and support that lead to improved student outcomes.

There are some downsides to classroom observations. First, if they are the sole basis for a teacher evaluation (as is true in many systems now), they may stifle innovation, forcing teachers to conform to particular notions of “effective practice.” Second, each of the instruments requires judgment on the part of observers. Even with trained raters, we saw considerable differences in rater scores on any given lesson. Moreover, possibly because different content requires teachers to exhibit different skills, a teacher’s practice seems to vary from lesson to lesson. Even with trained raters, we had to score four lessons, each by a different observer, and average those scores to get a reliable measure of a teacher’s practice. Given the high opportunity cost of a principal’s time, or the salaries of professional peer observers, classroom observations are the costliest source of feedback.

Student Surveys

Student evaluations are ubiquitous in higher education, where they are often the only form of feedback on instruction. (Student achievement gains and classroom observations are rarely used at the college level.) The MET project investigated the usefulness of student evaluations in 4th-grade through 9th-grade classrooms.

To collect student feedback, the project administered the Tripod survey, developed by Ronald Ferguson at Harvard’s Kennedy School of Government. Rather than being a popularity contest, the Tripod survey asks students to provide feedback on specific aspects of their classroom experiences. For example, students report their level of agreement to statements such as, “In this class, we learn to correct our mistakes,” “Our class stays busy and does not waste time,” and “Everybody knows what they should be doing and learning in this class.” While administering the survey, we took steps to protect students’ confidentiality, such as providing students with thick paper envelopes for submitting paper-based surveys or secure passwords to submit web-based surveys.

We learned several important lessons: First, students perceive clear differences among teachers. For example, in a quarter of classrooms, less than 36 percent of students agreed with the statement, “Our class stays busy and does not waste time.” In another quarter of classrooms, more than 69 percent of students agreed.

Second, when teachers taught multiple sections of students, student feedback was often consistent. The between-classroom correlation in Tripod scores was .66. This is higher than we saw with the achievement gains measure. Attaining a comparable level of consistency with classroom observations required scoring four different lessons, each by a different observer. We had to average over multiple observations by multiple observers to generate reliable scores. Even if the typical student is less discerning than a trained adult, the ability to average over many students (rather than one or two adults), and having students experience 180 days of instruction (rather than observe two or three lessons), obviously improves reliability.

Third, the student responses were more correlated with teachers’ student-achievement gains in math and ELA than the observation scores were. (Just as we did with classroom observations, to avoid generating a spurious correlation between student survey responses and achievement scores for the same group of students, we estimated the correlation across different classrooms of students taught by the same teacher.) In other words, student responses were not only consistent across classrooms, they were predictive of student achievement gains across classrooms.

For those many states and districts that are struggling to find ways to measure performance in non-tested grades and subjects, well-designed student surveys should be an attractive option for supplementing classroom observations. They are also among the least costly of the measures.

The Case for Multiple Measures

As with superheroes, all the measures are flawed in some way. Test-based student-achievement gains have predictive power but provide little insight into a teacher’s particular strengths and weaknesses. Classroom observations require multiple observations by multiple observers in order to provide a reliable image of a teacher’s practice. The student surveys, while being the most consistent of the three across different classrooms taught by the same teacher, were less predictive of student achievement gains than the achievement-gain measures themselves.

Fortunately, the evaluation methods are stronger as a team than as individuals. First, combining them generates less volatility from course section to section or year to year, and greater predictive power. Figure 1 compares the three different methods (classroom observations, student surveys, and student achievement gains) on reliability and predictive power. On the horizontal axis is the reliability of each method. (We report reliability as the correlation in scores from classroom to classroom taught by the same teacher.) On the vertical axis is predictive power, or correlation with a teacher’s average student-achievement gain working with a different group of students in 2009–10. Both predictive power and reliability are desirable traits, so values in the upper-right-hand corner of the graph are more desirable. The student achievement–gain measure is most highly correlated with student achievement gains but has lower reliability than student surveys. Student surveys have the highest reliability but are less correlated with student achievement gains. Classroom observations, based on the Framework for Teaching, are less reliable and less correlated with achievement gains.

Figure 1 also reports two different combinations of the three measures: an “equally weighted” combination (standardizing each of the measures to have equal means and variances and then applying a weight of .33 to each) and a “criterion-weighted” combination. (To generate the weights, we regressed a teacher’s average student-achievement gain in one class against the three different measures from another class, resulting in weights of .758, .200, and .042 on value-added, student survey, and classroom observation, respectively). The “criterion-weighted” measure offers more of the two desirable properties—predictive power and reliability—than any of the measures alone. (Even though classroom observations do not add much predictive power, it is hoped that classroom observations excel on a third dimension, not captured in the graph: the ability to diagnose specific strengths and weaknesses.) The next MET project report will explore weighting strategies in depth (see sidebar, page 40).

A second reason to combine the measures is to reduce the risk of unintended consequences, to lessen the likelihood of manipulation or “gaming.” Whenever one places all the stakes on any single measure, the risk of distortion and abuse goes up. For instance, if all the weight were placed on student test scores, then the risk of narrowing of the curriculum or cheating would rise. If all the weight were placed on student surveys (as happens in higher education), then instructors would be tempted to pander to students and students might be more drawn to play pranks on their teachers. If all the weight were placed on classroom observations, then instructors would be tempted to go through the motions of effective practice on the day of an observation but not on other days.

The use of multiple measures not only spreads the risk but also provides opportunities to detect manipulation or gaming. For example, if a teacher is spending a disproportionate amount of class time drilling children for the state assessments, a school system can protect itself by adding a question on test-preparation activities to the student survey. If a teacher behaves unusually on the day of the observation, then the student surveys and achievement gains may tell a different story.

There is a third reason to collect multiple measures: conflicting messages from the multiple sources of information send a signal to supervisors that they should take a close look at what’s going on in the classroom. Suppose a teacher is employing unconventional teaching methods that don’t correspond to the classroom-observation instrument being used in a state or district. If the teacher is getting exemplary student-achievement gains and student survey reports, a school leader should give the teacher the leeway to use a different instructional style. Likewise, if a teacher is performing well on the classroom observations and student surveys but had lower-than-expected student-achievement gains, a school leader might give the teacher the benefit of the doubt for another year and hope that student achievement gains will rise.

Implication for Practice

The MET findings have a number of implications for ongoing efforts to provide more meaningful feedback to teachers:

The main reason to conduct classroom observations is to generate actionable feedback for improving practice. Therefore, the standards need to be clear and the observers should not only be trained, they should demonstrate their understanding of the standards by replicating the ratings given by master scorers. School systems could certify raters using prescored lesson videos, such as we did in our project. They should also conduct multiple observations by more than one rater, and audit a subset of observations to track reliability.

Student surveys are an inexpensive way to add predictive power and reliability to evaluation systems. They could be particularly useful to supplement classroom observations in the grades and subjects where student achievement gains are not available. Although our results suggested such measures could be reliable and predictive, even with students as young as 4th grade, more work needs to be done to evaluate their usefulness in younger grades. To reduce the risk of pressure from teachers or peer pressure from fellow students, it is important that schools take steps to ensure the anonymity of individual student responses.

When it comes to measuring teachers’ effectiveness, the state ELA assessments are less reliable and less related to other measures of practice than state math assessments (or the assessment of students’ short-answer writing responses we used to supplement the state tests). The implementation of new literacy assessments in line with the Common Core state standards may help. In the interim, schools might adapt their classroom observations and student surveys to look for evidence of student writing or add questions to the student survey asking students to describe the quality of feedback they receive on their writing.

None of the data collected for MET were used for high-stakes personnel decisions. It may be that the measurement properties of student surveys, or classroom observations, or achievement gains could be distorted when stakes are attached. If principals inflate (or lower) their scores, or if students use the student surveys to play pranks, such changes should become evident in changing relationships among and between the measures. As a result, school systems should monitor those relationships as such systems are implemented.

Finally, we need many more studies evaluating the ways in which better feedback can be paired with targeted development investments to raise teachers’ effectiveness in improving student outcomes.

No information is perfect. But better information on teaching effectiveness should allow for improved personnel decisions and faster professional growth. We need to keep in mind the rudimentary indicators used for high-stakes decisions today: teaching experience and educational attainment. When compared with such crude indicators, the combination of student achievement gains on state tests, student surveys, and classroom observations identified teachers with better outcomes on every measure we tested: state tests and supplemental tests as well as more subjective measures, such as student-reported effort and enjoyment in class.

Thomas Kane is professor of education and economics at the Harvard Graduate School of Education. He was formerly deputy director within the U.S. education group at the Bill & Melinda Gates Foundation, where he led the Measures of Effective Teaching project. This essay draws from research done jointly with Douglas O. Staiger from Dartmouth College.

Over the past 10 years I have spent countless hours in the classrooms of aspiring champions – new teachers, veteran teachers, teachers working in district schools, charter schools, and private schools. When a teacher can effectively utilize all 49 of Lemov’s techniques in perfect harmony, it is feat at which to marvel. Much more commonly observed, however, is the teacher trying heroically – yet unsuccessfully – to fully engage each of his 30 students in the lesson he stayed up half the night planning. Lemov has provided us with an essential framework for instruction and classroom management. Yet the much more pressing question is how we can create the conditions that make “achieving the championship” more achievable and sustainable for all teachers. I believe that blended learning holds that promise.

Lemov organizes his 49 techniques into seven overarching themes. Because seven is easier to tackle than 49, I thought it might be an interesting exercise to examine the role that blended learning can and cannot play in helping to create more champions within each of these themes:

1. Setting High Academic Expectations. No matter how good the technology, students will always need to be surrounded by adults who set high expectations and hold them accountable for achieving at the highest possible level. Teacher need to do so, as Lemov points out, “without apology” and particularly for those students who have been led to believe either by themselves or others that success is out of reach. There are ways, however, that a blended-learning environment can make the task of setting high expectations for every student more feasible. First, online content can be structured in such a way that a student will be forced not to “opt out” until they demonstrate mastery of a concept. Second, and perhaps more critically, a teacher can spend focused time with an individual student while the rest of the class is working online. Without the pressures of engaging an entire classroom, the teacher can provide undivided attention where needed to ensure all students are meeting the high expectations that have been set.

2. Planning that Ensures Academic Achievement. The need for strong planning remains absolutely critical in a blended-learning environment. What becomes possible, however, is the shift from classroom level to individual student level planning. “Beginning with the end in mind” can focus on a particular student’s learning path; setting “manageable and measurable objectives” can be informed by real-time, student level data. The reality today is that our best teachers, particularly early in their career, spend much of their nights and weekends lesson planning. Although it may be feasible to do this through brute force at the classroom level, attempting to do so at the individual student level becomes effectively impossible. In order to create plans that differentiate for each individual student, teachers will need to depend on the growing number of resources available through online content, learning management systems, and data analysis tools. Harnessing these tools will in turn make these essential planning skills more accessible for a larger subset of teachers, allowing them to focus their time on how to best personalize the curriculum for each student rather than building each lesson from scratch.

3. Structuring and Delivering Your Lesson. One of Lemov’s techniques that’s gotten the most traction – and for good reason – is the notion of “Ratio”, the proportion of the cognitive work students do versus teachers. Although many teachers understand the importance of students doing the work, few are able to create the conditions for success within a classroom of 30 students. Most revert to the “sage on a stage” method of teaching. Making an increased “ratio” more accessible to the average teacher is perhaps one of the greatest promises of a blended-learning environment. Students are required to do their own cognitive work while engaging with the online content. Yet equally, if not more, promising are the opportunities created by the individual and small group attention that teachers can now provide to a subset of students while the others are working online. With a smaller number of students at any one time, teachers can focus on pushing the ratio of higher order thinking through Socratic seminars and other targeted strategies of instruction only possible in a small group setting.

4. Engaging Students. If they’re not engaged, students won’t learn. Many of the techniques Lemov describes depend on a heroic teacher with the personality of a “Vegas” performer. In order to keep the engagement of an entire classroom at once, teachers in a traditional setting must strike the perfect balance between being a stand up comedian and a drill sergeant. It’s a feat that few can accomplish. In contrast, a blended-learning environment can more feasibly harness student’s intrinsic motivation through online content that is differentiated and contains immediate feedback. That said, there are several critical roles that teachers must still play regardless of what can be provided online. Similar to setting high academic expectations, most students will depend on adult support to keep their engagement level high. When the going gets tough, nearly every student will be tempted at times to disengage. Teachers need to ensure that students stay focused and engaged. But again, the blended-learning environment allows teachers to target and differentiate their support in a much more manageable manner.

5. Strong Classroom Culture. Ensuring a strong classroom culture is critically important in a blended-learning environment. Tight transitions are essential and yet perhaps even harder to accomplish because of all the technology to be navigated. Many of Lemov’s techniques, such as having a strong “entry routine” remain important for teachers to master. Many of his techniques, however, are necessary only in an environment where 30 students must be streamlined into one cohesive entity; they therefore become obsolete in a blended-learning classroom. For example, “Do Now”s (a commonly used technique for keeping students on task while waiting for whole-group instruction to start) can become “Start Now”s, which allow each student to dive into their individualized content as soon as they enter the classroom. Similar to the notion of engagement, a strong classroom culture becomes that much more obtainable in an environment where students are intrinsically motivated. Rather than drilling students into compliance around the critical components of SLANT (Sit Up, Listen, Ask and Answer Questions, Nod your head, Track the Speaker), a blended-learning environment will hopefully create the conditions where students will want to be more fully present through individualized learning and targeted small-group engagement.

6. Setting and Maintaining High Behavior Expectations. Similar to academic expectations, high behavior expectations are critical regardless of the learning environment. Teachers will always need to be the owners of this within their classrooms.  A blended-learning environment, however, may make it more feasible for the average teacher to find her “strong voice” when more students are actively engaged and less interventions are required. Additionally, it’s important to reevaluate what teachers are striving for “100% compliance” around within their classrooms. For example, in a blended-learning classroom where students are engaged deeply in their individualized learning plan, the occasional side bar conversation or student listening to music while working may not be something that runs against a positive classroom culture.

7. Building Character and Trust. Together with setting high expectations, ensuring each student feels cared for and supported on his learning journey is perhaps the most critical role that a teacher needs to play. Unfortunately, however, this role often gets relegated to the bottom of the list because there is so much whole-group direct instruction and content delivery that needs to occur. Additionally, with 49 techniques to master each and every day, it becomes virtually impossible for the average teacher to preserve the “emotional constancy” that Lemov describes to be so critical. A blended environment affords teachers the opportunity to connect individually with each student and differentiate to both their academic but also, when necessary, personal and emotional needs.

Skeptics of technology-enhanced instruction are quick to assert that teaching is a craft that can never be replaced. I couldn’t agree more. The strength of Lemov’s framework is that it articulates for us exactly what that craft looks like when effectively mastered. The fact of the matter is, we simply don’t have enough – or even remotely close to enough – champion teachers in our schools today. To be clear, it’s not for lack of effort, motivation, intelligence, or passion. It’s because the job we’re expecting our teachers to accomplish is superhuman. Far from replacing our teachers, a blended-learning environment holds the potential of making the job more accessible for more individuals. It provides the opportunity to create more champions.

Allison Akhnoukh has been working in education reform for over 10 years, most recently with the KIPP Foundation supporting the growth and sustainability of the network of charter schools. While studying at Harvard Business School, she worked with Clayton Christensen on early research that led to the creation of Disrupting Class.

This article first appeared on the blog of the Innosight Institute.

Hostetter might just as surely have been talking about Relay, which aims to transform teacher education to fit the needs of urban schools. The amazing—or at least attention-getting—improvement on the wheel is that New York–based Relay is linking the success of its students to the success of their students.

During their second year in Relay’s two-year masters-degree program, elementary-school teachers are asked to show that their own students averaged a full year’s reading growth during the school year. They must also set a reading goal for each child, perhaps two years’ growth for a child who is three years behind, for example. Students can earn credit toward an honors degree if 80 percent of the children they teach meet their individual reading goals.

To earn their degrees, elementary-school teachers are also asked to show that their students earned, on average, 70 percent mastery on a year’s worth of state or Common Core Standards in another subject, usually math. In other words, a math class would meet the goal if students’ individual mastery scores, when averaged, were 70 percent or better. Middle-school teachers use the same yardstick, but only in their specialized subject.

Relay’s cofounder and president, Norman Atkins, talks movingly about the crisis in inner-city teaching and the need to “grow a pipeline of effective teachers who can make an immediate difference.” But the true value of Relay’s model may go beyond potentially improving the teaching in the classrooms where Relay’s graduates work. Robert Pianta, dean of the University of Virginia’s Curry School of Education, explained that Relay is creating a “feedback loop,” using child-level data to measure the outcomes of its teacher-training program, and using those measures to make decisions about program design. “This is how systems get better,” he told me.

Spreading accountability from the teacher back to the education school is an idea the Obama administration is also promoting in its efforts to remake teacher training. This spring, a federal panel looking at teacher-preparation programs debated, among other things, rating ed schools based on how much their teachers add to student learning. That possibility riles ed school deans, among others, but “individual accountability is coming down the pike,” says Kate Walsh, president of the National Council on Teacher Quality, a research and advocacy group.

Even Relay’s admirers concede that it’s too soon to tell whether the model works. It’s operating in just two cities: New York, where it’s offering a master’s degree to 206 students this year, and Newark, New Jersey, where so far it has state approval only to offer a one-year teaching certificate and has enrolled 64. Relay’s first class won’t graduate until 2013. Philanthropies are still footing much of the bill.

Relay has hired a research director, but Atkins says it may not open itself to independent researchers for another four years. Its students—with undergraduate degrees from the likes of the University of Virginia, Lafayette, and Georgetown—are atypical for an ed school, which could complicate comparisons with other teacher programs. Above all, trying to measure student achievement and a teacher’s role in improving it is hard to do.

Still, says Arthur Levine, former president of Columbia University’s Teachers College and a member of Relay’s board, Relay is helping to reinvent teacher education. “Relay is the model,” he told me. “It is the future.”

Nuts and Bolts

During their second year in Relay’s two-year masters-degree program, elementary-school teachers are asked to show that their own students averaged a full year’s reading growth during the school year. They must also set a reading goal for each child.

If there were ever a system in need of reinvention, it would be teacher education. Decades of studies, reports, and blue-ribbon commissions have criticized ed schools for low entrance requirements, mediocre standards, an emphasis on theory over practice, and outdated curricula. “It’s an accepted truth that the field is broken,” Walsh told me. The problem with fixing it, she added, is that “nobody has known what to do.”

What Relay is doing largely breaks the mold. Its students are full-time elementary- and middle-school teachers, almost all of them fresh out of college, almost none of them with a traditional teaching degree. The program is heavy on practice and nuts-and-bolts technique. It is competency-based: students can be waived out of Designing Assessments, for example, if they can show they are already adept at writing tests.

Relay’s method flips the classroom, with an online lesson at the start of every module or teaching unit (about 40 percent of instruction is online) and in-class discussions and exercises afterward. Twice-monthly night classes, once-monthly Saturday classes, and two summer terms are taught by master teachers and charter school heavyweights. Online instructors include Lee Canter, author of Assertive Discipline, charter school founders and principals, and Relay professors and deans.

Modules vary in duration and range from the nitty-gritty of classroom management—how to arrange furniture, how to grade papers, how to deal with families, how to open and close a lesson—to big-picture subjects, including literacy instruction, writing development, learning disabilities, unit planning, and character development. For a class called Benchmarking and Tracking Progress, scheduled to last 11¼ hours, the catalog says Relay students will create a spreadsheet to track their own students’ progress against year-end goals, and use the data to customize their teaching. In a class called Behavior Management Plans, Relay students will write a set of classroom rules and learn “how to engage in the very necessary practice of correcting students when they misbehave.”

Everybody Engaged

I logged onto an online lesson for a module titled Engaging Everybody, taught by Doug Lemov, managing director of Uncommon Schools. In the 3¾-hour lesson, Lemov lectured for three or four minutes on each of four techniques that he promotes to keep youngsters involved in class, techniques he labels “wait time,” “everybody writes,” “cold call,” and “call and response.” Each of Lemov’s minilectures was followed by a few pages of online reading from his book Teach Like a Champion, and an essay question or two that students answer online (see “Tools for Teachers,” book reviews, Spring 2011). Then came several short videos showing teachers using each technique in the classroom, with Lemov noting the teachers’ use of an apt pause or effective gesture.

Ed schools typically separate lectures from experience, Arthur Levine pointed out to me; by putting lessons online, Relay can blend them. Next came practice scenarios—what do you do if only three children raise their hands to a question about angles?—online group exercises, and instructions to prepare a lesson plan that incorporates the techniques.

Evening classes are on pedagogy—how to teach—and Saturday classes are on subject matter—what to teach. At the second Engaging Everybody evening class, Relay students are expected to present a 10-minute video of themselves using the techniques in their own classrooms. A complex “rubric” describes how students will be assessed on each: on the wait-time technique, students are evaluated on whether they wait at least three seconds between posing a question and calling on a child for an answer, and whether they “strategically narrate” the wait with encouraging comments.

The classroom lessons are heavily scripted. During the first three minutes of the Engaging Everybody class, for example, the Relay students are to report on how often they’re using the four techniques. The script then lists four paragraphs of narrative and questions for the Relay professor to pose over the next four minutes. For five minutes after that, there’s a review, with 10 questions for the professor to ask, and then a suggested transition: “All right, our minds are fresh on today’s content and we’re ready to move.” Then there’s a guided 7-minute “table discussion,” 5 minutes of class discussion, 11 minutes of partner feedback, and so on.

“It is the most self-consciously designed program I’ve ever seen at a university level. Everything was thought out,” said David Steiner, dean of the Hunter College School of Education, which hosted Relay’s predecessor, Teacher U, beginning in 2008. (Teacher U’s last class of about 147 students will graduate this summer.)

Relay students “model” the kind of behavior they hope to see in their own classrooms, so their hands fly up at questions, they rush to stack chairs and pass out papers, they snap their fingers or waggle their hands to show approval. Relay’s scripts do the same kind of modeling by showing students how to effectively use their limited class time, Steiner explained.

Feedback Loop

When asked a question about reworking a question from a teachers’ guide to fit the current lesson, Mayme Hostetter, Relay’s dean, says, “Sure. No need to totally invent the wheel. Just make the wheel amazing.”

Relay’s class of first-year students had moved to the 6¾-hour Designing Assessments module when I visited in the spring. At the first of the module’s two evening classes, they had practiced writing “exit tickets,” quick quizzes to measure kids’ understanding of that day’s lesson. At the second evening class, students were to write an end-of-the-week test. Again, the script divided the evening into increments, with 45 minutes for students to practice writing test questions and the final 10 minutes for “team building.”

The Newark class, held at North Star Academy Charter School, was looser than the script suggested—and heavier on inspiration. James Verrilli, director of the Newark program and founding principal of North Star, opened with a clip from a Hollywood film about an innocent man’s decades-in-the-making escape from prison, and asked how it related to urban teaching.

The communal answer was that the escape seemed doomed—“like some people look at our kids,” one young woman said—but that perseverance and vision will yield success. “Shout-outs” followed that, with Verrilli singling out students, and students singling out each other, for exemplary work in their classrooms that week. The evening closed with an animated call-and-response reading of Relay’s creed, which ends with the lines, “We touch lives daily. We are teachers.”

In between, the discussion ranged from how to align test questions with the state standards to the layout of a test paper. There was agreement that some questions on the New Jersey tests included extraneous information that obscured the lesson. But “if the state is doing it, we don’t want our kids walking in blind,” said another young woman, who suggested that everyone write a few wordy questions for their own students to practice.

A few days earlier, I attended a Saturday class at Baruch College Campus High School. It gathered most of the Relay students working in New York City schools (mostly in charters, but a few in district schools where Teach For America has assigned them), divided them into subject specialties, and then again by elementary- and middle-school levels. Relay says 61 percent of the New York class is working toward a master’s degree in childhood education, 9 percent in middle-school math education, 5 percent in middle-school science, 3 percent in social studies, 12 percent in English, and 10 percent in general middle-school education.

In a class called Geometry, Fractions, and Measurements, Nicole Chalfoun—a former Bronx 5th-grade teacher—asked Relay students to design a “remediation strategy” for a child whose answer to the equation 2/5 + 3/8 is 5/13. “Where would you start?” she asked, as her students discussed which manipulatives would best convey to the child why uncommon denominators can’t be combined.

In Teaching Middle School Social Studies III, Ali Brown—director of history achievement for the Achievement First schools—asked her class to write an “essential question” that would frame a unit they were soon to teach on the American Revolution. “What is best going to make your kids think hard?” she asked. In an Elementary School Literacy class—the last of 10 sessions in the module—students were critiquing videos they had made of themselves teaching a reading lesson. In Teaching Middle School Math III, the morning began with a game called Buzz and a discussion of ways to modify it to include higher-level math, including calculus.

Every class meeting ends with a survey—was the lesson helpful, how could it be improved?—with comments fed back to course designers, says Hostetter.

The students I talked with—almost all of them first-year teachers—told me that Relay’s lessons were helping them plan their classes, practice their presentations, keep their kids engaged. “Everything I learn here I can use the next day,” said Milan Reed, who graduated from the University of Virginia with a major in political and social thought and now is teaching at Newark’s Spark Academy charter school.

“I get ideas, I get practice, I get feedback,” added Adam Feiler, a 2008 Georgetown graduate and Teach For America volunteer who’s teaching 4th grade at North Star’s Vailsburg Campus elementary.

Many also told me that Relay’s lessons have changed their classroom culture. “The culture went from being compliant to being invested,” said Max Silverstein, a Penn State business major now teaching in an early-childhood classroom at Newark Legacy Charter School. I heard the same thing from Alonte Johnson, a Morehouse College English major who is teaching middle-school English at Kings Collegiate Charter School in Brooklyn. A few days earlier, his students designed a seating chart that paired the better and slower readers. “The environment is more interdependent instead of everyone working for me,” he said.

On a Mission

What Relay is doing largely breaks the mold. Its students are full-time elementary- and middleschool teachers, almost none of them with a traditional teaching degree.

Norman Atkins, who founded Uncommon Schools and its North Star Academy as well, said he, David Levin, cofounder of the KIPP charter schools, and Dacia Toll, founder of Achievement First schools, began talking about an education school when they found themselves competing for the same teachers. “Rather than fight over a shallow pool of talent, we were interested in what it would take to build a generation of teachers,” he told me in Relay’s spare Manhattan offices above a public library. (Levin and Toll are on the Relay board, but don’t hold executive positions in the nonprofit.)

They approached 10 college presidents looking for a partner institution, Atkins said, and Hunter’s David Steiner, himself a critic of teacher training, “was waiting for us with open arms.” Last year, seeking more autonomy, Atkins launched Relay and began phasing out Teacher U. The new school takes its name from research suggesting that a “relay” of three years of good teachers can erase the average educational disadvantage of low-income children.

The idea of holding Relay students—and before them, students at Teacher U—accountable for their students’ progress was “one of the very first things we talked about,” Atkins said. The school settled on the 70 percent average mastery floor after looking at the New York math and language tests, where proficiency generally is defined as a score of 70 percent correct answers. The 80 percent stretch goal (for the honors degree) was less data-based but is “at the nexus of ambition and feasibility,” said Brent Maddin, Relay’s provost.

Some 95 percent of Teacher U’s 2010 graduates and 98 percent of its 2011 graduates met the 70 percent targets, he said, although the graduation rate over the two-year master’s program is lower, between 70 percent and 80 percent because of attrition, Hostetter noted.

Reading progress can be assessed with any of six tests, including Fountas and Pinnell Benchmarks and Pearson’s DRA2. But subject tests don’t exist for all subjects and grades. Relay’s handbook says its students instead can use tests they acquire elsewhere or even write themselves, if the assessments show mastery of state or Common Core standards, or of standards set by charter networks or individual schools.

I asked Verrilli, head of the Newark program, how Relay could analyze achievement among youngsters taking so many different tests (a half dozen of his Newark students who are teachers in district schools are writing their own year-end assessments). It wasn’t an apples-and-oranges comparison, he said, but one between “McIntosh and Golden Delicious.” The comparison isn’t among tests, but about mastery levels, he said.

I put that to Scott Marion, associate director of the National Center for the Improvement of Educational Assessment, who told me that Relay was “dreaming” if it hoped to compare performance across schools, but he otherwise sounded supportive. “I care about the end determination: Is the teacher effective or not,” Marion said, “not ‘did these kids achieve a number’?”

Atkins is a serial social entrepreneur who also started the Robin Hood Foundation, which invests in schools and antipoverty programs in New York. His plans for Relay set a breakneck pace: Next year, he expects Relay to enroll 500 to 550 students in New York and New Jersey. It will add classes for high school teachers, including chemistry, biology, and physics. It has applied to New Jersey to begin a master’s program. And Relay expects to extend its reach further into district schools under an agreement to train up to 60 NYC Teaching Fellows in the Bronx.

Atkins said he expects Relay to be fully supported by tuition and client-school fees in three or four years. For now, philanthropies are footing about $13,000 of the $35,000 two-year tuition bill. Students pay about $4,500, with charter schools and federal grants and subsidies making up the rest. Arthur Levine, the board member, agrees with Atkins’s aim. “For innovation to survive, it has to be self-sustaining. If something’s not self-sustaining, it’s not serious,” he told me. And Relay is nothing if not serious.

“What calls us every day is the sad and tragic circumstance” of urban education, Atkins told me, and in one phrase or another, everyone at Relay says the same thing. The other thing they all told me turns on its head the notion of what makes a great teacher.

“We’re saying great teachers are made,” James Verrilli told me, “not born.”

June Kronholz is a contributing editor of Education Next and a former foreign correspondent, bureau chief, and education reporter for the Wall Street Journal.

Teacher Evaluations Found to Improve Midcareer Effectiveness

When teachers in Cincinnati were evaluated rigorously, student performance on math tests improve

CAMBRIDGE, MA –A new study shows that Cincinnati’s rigorous Teacher Evaluation System (TES) has had a direct and lasting effect on midcareer teachers’ performance.  Students taught by a teacher in the years after she had been through the evaluation program scored 0.11 standard deviations higher in math, on average, than the students she taught in the years before her evaluation (as measured by end-of-year 4^th through 8^th grade state tests).  This difference is equivalent to about 3 – 4 months of additional instruction or a gain of about 4.5 percentile points for the average student.  The Cincinnati evaluation is a yearlong process and a teacher’s students also scored 0.05 standard deviations higher in the year their teacher was being evaluated, a difference of 1.5 – 2 months of additional instruction.

Researchers Eric S. Taylor and John H. Tyler note that to the best of their knowledge, their study is the first to test the hypothesis that practice-based teacher evaluation programs can help to improve teacher performance, in addition to their value in identifying teachers’ strengths or weaknesses.  Well-designed performance evaluation “can be an effective form of teacher professional development,” the authors observe.  Their analysis, “Can Teacher Evaluation Improve Teaching?” will appear in the Fall issue of Education Next and is available at www.educationnext.org.

In 2000, Cincinnati launched TES, a thorough process by which teachers’ performance is assessed through classroom observations and a review of work products.  Teachers are observed and scored four times:  three times by a peer evaluator (an experienced teacher) and once by a school administrator.  Both peer evaluators and administrators complete an intensive training course, learning to accurately score videotaped teaching examples according to a specific rubric.  TES is costly (about $7,500 per teacher evaluated, primarily for evaluators’ salaries) and its approach “contrasts starkly with status quo ‘principal walk-through’ styles of class observation,” note the authors.

Currently, all teachers newly hired by the Cincinnati school district, regardless of prior experience, take part in a TES evaluation during their first year.  In their fourth year they are evaluated again, prior to receiving tenure (assuming successful evaluations), and once every five years after achieving tenure.  For tenured teachers, evaluation scores determine eligibility for some promotions or additional tenure protection, or, in the case of very low scores, placement in a peer assistance program with a small risk of termination.

The researchers’ analysis includes only teachers hired before TES was introduced in the 2000-2001 school year and who were teaching 4^th through 8^th grade in the years 2003-04 through 2009-10.  The group evaluated included 105 experienced teachers hired by the district in the school years from 1993-94 through 1999-2000.  Evaluating this sample of teachers allowed the authors to measure the effect of evaluation on performance separate from any gains that come from increased experience, and permitted them to make comparisons of the achievement levels of a given teacher’s students both before and after the TES assessment.

The authors observe that the teachers in their study experienced their first rigorous evaluation after 8 to 17 years on the job, and may have been particularly receptive to comments from peer evaluators, rather than solely from administrators.  The TES impact was found to be largest for teachers who were the weakest prior to evaluation.  The researchers note that their findings suggest well-structured evaluation systems can be cost-effective expenditures that “not only serve (a) sorting purpose but also enhance education through improvements in teacher effectiveness.”

About the Authors

Eric S. Taylor is a doctoral student at Stanford University.  John H. Tyler is professor of education, economics, and public policy at Brown University.  This article is based in part on a forthcoming study in The American Economic Review.  The authors are available for interviews.

About Education Next

Education Next is a scholarly journal published by the Hoover Institution that is committed to looking at hard facts about school reform.  Other sponsoring institutions are the Harvard Program on Education Policy and Governance, part of the Taubman Center for State and Local Government at the Harvard Kennedy School, and the Thomas B. Fordham Foundation.  For more information about Education Next, please visit:  www.educationnext.org.

For more information on the Program on Education Policy and Governance contact Antonio Wendland at 617-495-7976, pepg_administrator@hks.harvard.edu, or visit www.hks.harvard.edu/pepg.

Sure, low morale might simply reflect tough economic times; when (or if) state and local coffers finally recover, higher morale might too. But let’s be honest: The message we reformers are sending isn’t all peace, love, and happiness, and that’s probably having an impact, and not for the better.

We think many teachers are dumb (look at those SAT scores!); greedy (look at those gold-plated healthcare and pension plans!); racist (look at those achievement gaps!); lazy (look at those summers off!); ill-prepared (look at those crappy ed schools!); uncaring (look at all that bullying!); unnecessary (look at what computers can do!); and incompetent (look at those low value-added scores!). Or at least that’s how many teachers hear it, I suspect. We love teachers—we just hate everything about them.

One option, according to union leaders, Diane Ravitch, and others, is to stop pressing for reform. Stop complaining about unaffordable pensions or healthcare plans. Stop worrying about across-the-board raises. Stop measuring teachers’ contributions to student achievement gains. Stop pressing for LIFO and tenure and collective bargaining changes. Stop obsessing about online learning.

That might get us happier teachers but it won’t get us dramatically better schools.

So what’s the other option? How can we continue to make the case for reform without alienating teachers, without turning them into the enemy, the problem, the object of our disdain?

One way is to put teachers in charge of their own schools. That’s the argument Ted Kolderie and his colleagues at EducationEvolving have been making. (See this great Education Next article for an overview of teacher-led schools.) If we want teachers to feel respected and motivated, we should treat them as true professionals. Let them call the shots. Set the budget. Hire new teachers. Deal with management concerns. In all likelihood, these teacher-leaders will come to some of the same conclusions as reformers. (Such as: low performers need to go; there are trade-offs between small class sizes and more generous salaries and benefits; all teachers need their craft to be regularly evaluated against some clear and common expectations around good practice; etc.)

Another way is to champion reforms that teachers do support. For instance, make it easier for educators to discipline unruly students, or to use “ability grouping” in their classrooms instead of mandating the nearly-impossible strategy of “differentiating instruction.”  In other words, remove the obstacles (often ideological in nature) that are getting in the way of teachers achieving success in their classrooms. If we don’t want to put teachers in charge of their own schools, at least give them more control over their work, as Richard Ingersoll argues. And get their backs when they are faced with ridiculous demands from parents or others.

Another possibility: find smart ways to give teachers a “voice” that doesn’t entail subjugating them to union bosses. That’s part of the idea behind Teach Plus, the Association of American Educators, and Educators for Excellence. The other side of that coin is to get better information to rank-and-file teachers in the first place, so they aren’t learning about reform solely through the filter of union rhetoric.

None of these are perfect solutions. As long as reformers are talking about curtailing teachers’ benefits, or making their jobs less secure, or evaluating their instructional practices, there is going to be some anger and resentment. And talk about those reforms we must. Let’s just try to make some effort to heed teachers’ concerns, and inspire them to excellence, too.

-Mike Petrilli

Special thanks to Ty Eberhardt, Joanne Jacobs, Steve Farkas, Ted Kolderie, and Amber Winkler for seeding several of the ideas mentioned above.

This blog entry originally appeared in the Fordham Institute’s Education Gadfly Weekly.

Public Impact has published new career paths stemming from our school models that use job redesign and technology to reach more students with excellent teaching. These models enable excellent teachers to expand their positive impact on students, and many allow additional time for planning, collaboration, and development—so all teachers can improve.

Prior decades are littered with abandoned efforts to create teacher career paths. Well-intended new efforts risk falling into the same traps. To understand how our career paths are different, it helps to understand what hasn’t worked previously. The fatal flaws of most teaching career paths include:

• No pay increases. Career advancement often does not come with more pay. Or, add-on pay disappears when special money runs out. Instead of being a compliment, promotions become a reminder to our best teachers of how little their profession values their excellence.
• Financially unsustainable roles. When schools pay for advanced roles with temporary, special funding, and when new roles do not produce a sustainable financial benefit to schools, even schools that value excellence dearly cannot pay for it.
• Too few options. Narrow choices have typically been limited to instructional specialists who provide differentiation outside the classroom and “master” or “mentor” roles responsible for coaching other teachers. Coaching is an important role, as is differentiating instruction. But today, these are the only school-level advancement options in the profession, other than becoming a principal.
• Limited authority. Mentors and specialists typically have no authority. Their advice is useful only if a classroom teacher chooses to adopt their successful methods and techniques.
• Limited accountability. Mentors typically have no accountability for mentees’ success with students. Data about students whom specialists help are not tracked formally as they are for classroom teachers. So, essentially, many of the best teachers are removed from responsibility for students, rather than having enhanced responsibility.

Public Impact’s career paths, built from school models with input from many partners, address all of these flaws. What makes our career options different?

We give educators at least 15 career paths to choose from. These options allow teachers who achieve excellence to advance, earn more within regular budgets, enhance their authority within schools, and keep clear responsibility and credit for helping more students learn.

The paths include:

• Teacher-leader roles, in which excellent teachers extend their reach by leading multiple classrooms and a team of teachers—allowing teachers to develop leadership skills earlier in their careers, while helping peers improve their performance immediately.
• Specialist jobs that let elementary teachers focus on their best subjects and roles.
• Blended-learning roles that enable teachers to extend their reach by swapping enough teaching time with digital instruction to teach more students, focus on higher-order thinking skills, and increase planning and collaboration time.
• Remotely located roles that let excellent teachers teach and take responsibility for students anywhere, using new technologies like webcams that allow teachers to connect with students.
• Boundless teaching roles that let teachers create video lessons, design software, and develop curricula and assessments for limitless numbers of students.
• Team-teaching roles that allow solid and developing teachers to learn from excellent teachers, focus on their strengths, and pursue advancement, too.

New paraprofessional roles, such as tutors, learning coaches, and lab monitors, make it possible for teachers to save time and reach more students. When people in these jobs enable excellent teachers to help more students, they too can earn more.

Instead of the mixed signals of most education career paths, these send a clear, sustainable message that schools value teaching excellence and their great teachers’ positive impact on students, peers, and their profession. This is just one aspect—but a critical one—of building an Opportunity Culture for teachers.

-Emily Ayscue Hassel and Bryan Hassel

Fourteen of the 4th graders at Washington, D.C.’s Hope Community Charter School had chosen the right answer—1/3 and 5/15—on a test written for the school by Boston-based Achievement Network (ANet). But 20 chose the wrong answer, and two didn’t answer at all.

So on a bright November afternoon three weeks after the test, Hope’s math specialist, Christine Madison, and two of the school’s 4th-grade teachers huddled over five pages of test-score data assembled for them by ANet. Hope’s Tolson campus serves 420 youngsters in grades PreK–8, almost all of them African American and two-thirds of them from low-income families. It is one of three D.C. charters that are operated by Virginia-based Imagine Schools and are working with ANet. The city’s charter board calls Hope “mid-performing”—about 40 percent of its elementary-school children and 60 percent of its middle schoolers are considered proficient in math and English.

ANet coach, Amrutha Nagarajan, coaches 14 schools for the organization.

The ANet data showed that the children generally understood fractions. But they also showed that many youngsters—including some with otherwise good scores—were unsteady at fractional models, or word problems, which are among the 15 math standards that Washington schools are expected to teach their 4th graders.

The fraction lesson, drawn from the class textbook, apparently didn’t work when the teachers first taught it. So at this half-day data-analysis exercise scripted by ANet and overseen by an ANet coach, Madison and the teachers debated why it failed and plotted how to reteach it. How about using an art project, fraction charts, flip-books, team competitions, they mused. How about reteaching the lesson to youngsters grouped by ability? How about reteaching boys and girls differently?

Think about how you taught the lesson the first time, and then do something different, urged Madison, who grew more exuberant with each new idea. “I think I may not have used enough visual aids,” one teacher finally conceded as Madison beamed.

Learning Curve

John Maycock is the founder of Achievement Network, a nonprofit organization that provides data-analysis training and coaching for school leaders and teachers.

Data-driven instruction began its spread across the country about a decade ago, in the footsteps of the No Child Left Behind requirement that schools administer yearly achievement tests. Those tests didn’t help teachers spot and backfill learning gaps, though. Scores came back after everyone had moved on to the next grade, and anyway, the tests were designed to hold schools accountable for the performance of groups: Did enough English-learners pass, enough African Americans? They were not intended to show which students didn’t understand decimals.

By most accounts, a few charter schools began testing their youngsters more frequently, with the idea that teachers could use those interim results to inform their teaching. “If you pay attention to what students learn and what they don’t, you learn how to teach more effectively,” says Paul Bambrick-Santoyo, whose book Driven by Data is a primer on data-driven instruction.

But on the ground, data-driven instruction has encountered problems. Schools complain that interim assessments produced by publishers aren’t always aligned with curricula, pacing guides, or year-end state tests. The assessments are often too easy, handing schools an unhappy surprise when state test results are posted.

Some districts have taken over the job of producing interim tests, but their data offices have the reputation of taking so long to return results that the information is too old to be of much use. (Ben Fenton of New Leaders for New Schools says he has encountered schools that sidestep their districts by photocopying their kids’ answer sheets and grading the assessments themselves.)

Schools that have tried to develop their own assessments have found the job overwhelming. Jermall Wright, principal of southeast Washington’s Leckie Elementary, told me that his leadership team tried it when they decided that the district’s assessments were inadequate. But writing, scoring, and analyzing the tests took so much time that they quickly abandoned the effort.

In any event, few teacher-education schools include data-analysis training, so many teachers don’t know how to read the data, or don’t have the time to use the information to rethink their lesson plans.

By the mid-2000s, “data was starting to become a hot topic,” says John Maycock, who at the time was completing a master’s degree in the school-leadership program at Harvard’s Graduate School of Education. But “teachers were saying they wanted help” understanding and using it, he adds.

“We started to see that just having access to better data was not enough to drive improvement,” says Joe Siedlecki, a program officer at the Michael and Susan Dell Foundation, which has given $1.7 million to ANet.

Maycock’s solution was to found a nonprofit organization that combines rigorous, standards-aligned assessments; data-analysis training and coaching for school leaders and teachers; guided peer review; and networking across schools. Schools join ANet, pay a fee for its services, and commit their teachers and principals to a four-times-a-year cycle of testing and data review. The model goes beyond traditional professional-development models by linking ANet’s work to each school’s data feedback loop: student achievement results inform the guidance ANet provides.

Coaching the Team

School leaders also agree to carve out time for teachers to look at the data together, and to take part in the cycle of meetings and reviews themselves.

Two days after Hope’s data-analysis meeting, I returned to the charter school to listen as its leadership team reviewed the session with ANet coach Amrutha Nagarajan, a 28-year-old Wellesley- and Harvard-educated former banker. Nagarajan came to Washington as a D.C. Teaching Fellow, resisting pressure from her Indian-immigrant parents to pursue a business career, she says, and now coaches 14 schools for ANet.

Hope had administered its second cycle of interim assessments in math and English-language arts on November 8 and 9 after downloading the tests from ANet’s web site. The untimed tests are given every six to eight weeks and typically take youngsters about an hour, Nagarajan told me. The 4th-grade math test asked 34 questions; the 3rd-grade language-arts test included three readings—a folk tale, a poem, and a nonfiction passage—and 20 questions.

The school’s leadership team had the option to view the year’s assessments well beforehand to be sure the school’s lesson plans and pacing would prepare kids for the district’s year-end tests. Hope doesn’t factor the ANet interim test scores into youngsters’ overall grades, and in their contract with ANet, network schools agree not to use the scores to rate their teachers, a move designed to dampen teacher resistance. School leaders also agree to carve out time for teachers to look at the data together, and to take part in the cycle of meetings and reviews themselves.

After the early-November tests, Hope shipped its completed answer sheets to ANet’s Boston office. Within 48 hours of receiving them, ANet posted the results online, and Hope printed out a set for every teacher. The data tell teachers how their students answered each question, of course, but also how each youngster, the class, and the grade scored on questions aligned to each standard, like dividing whole numbers or identifying details in a reading passage.

The data showed that among Hope’s 5th graders, for example, 88 percent appeared to understand how to find the area and perimeter of rectangles and triangles, but only 26 percent could do the same with circles. Among 8th graders, 65 percent could analyze details and draw conclusions from two reading passages—they did better at nonfiction than fiction—but just 52 percent could identify the author’s main purpose in writing the piece.

In their contract with ANet, network schools agree not to use the scores to rate their teachers, a move designed to dampen teacher resistance.

ANet’s coaching script next called for Nagarajan and the leadership team to go over the results—in ANet parlance, this is a pre-data meeting—and set priorities for a professional development day, or data meeting, two days later. They agreed that Hope’s 8th-grade language-arts teachers would concentrate on how better to teach “author’s purpose,” a D.C. learning standard. Its 6th-grade teachers would focus on “drawing conclusions,” its 3rd-grade teachers on “analyzing details,” and so on, through each grade and subject.

The idea, Nagarajan told me, is for teachers to “go deep on one or two standards” by dissecting four or five test questions each at the data meeting. The goal, she added, is for that kind of item analysis to become part of each teacher’s routine as she becomes more comfortable with data.

Nagarajan—whose teaching experience includes a year in Chennai, India, after the 2004 Indian Ocean tsunami—remained in the background on data meeting day as Hope’s teachers worked on their reteaching plans. But she and ANet provided a clear structure to keep the school’s improvement plans on track.

During the data meeting, teachers pored over a form called an “item analysis template”—downloaded from the ANet web site—that forced them to think through the test questions that had given their kids the most grief. “What were the misconceptions” that led so many students to choose the wrong answer, the form asked them to consider. What groups of students missed the answer? What did students need to know to get it right?

Next, they worked through a “reteach action plan,” also downloaded from ANet. How was the lesson taught originally, the form asked. How and when would it be retaught, and to whom—the whole class, a small group, individual children?

Nagarajan, meanwhile, pressed Hope’s leadership team to meet deadlines and create what she called “follow-up structures.” When Dr. Chloé Marshall, Hope’s high-energy principal, said her teachers would file their reteaching plans that Friday, Nagarajan asked, “By the end of Friday or the beginning of Friday?” When would they do the reteaching, the next step on the ANet agenda, she asked. Those “reteaches” are supposed to be slipped into a compatible lesson so they don’t derail a teacher’s lesson plans and pacing, and target just those kids who need them.

Nagarajan continued: When would Hope retest—a quick two- or three-question quiz in each class—to make sure the new lesson was effective? When would teachers hold their “reflection meeting,” the last step in the assessment cycle, to look at the new results? “Does that make sense? What do you think?” she pressed the leadership team.

At the postdata-day debrief—more ANet parlance—Nagarajan and the school’s leadership team conceded that the English teachers were still learning how to use the ANet data to break down the broad standards into smaller skills, and to figure out which skills their students were lacking. But they also saw progress: teachers were talking more, sharing strategies, and acknowledging the need to teach differently.

“Some teachers were still challenging the test” by laying the blame on bad questions, Nagarajan said. But many more were “owning the data,” insisted Marshall, making the shift from the-kids-aren’t-learning-it to I’m-not-teaching-it. And with that, the discussion moved on to new teaching strategies, new delivery strategies, resources for new lesson plans, and the team’s goals for Hope’s students.

“The object isn’t to teach kids a process” that leads them to the right answer on a test, “but to visualize a problem and solve it,” Madison said to general agreement. “That’s what will help them in real life.”

Meeting a Need

Many teachers were "owning the data," making the shift from the-kids-aren't-learning-it to I'm-not-teaching-it.

John Maycock, who is now 37 and calls himself ANet’s “chief growth officer,” had managed afterschool centers in San Francisco, where he says he became “hooked forever” on education. But his real interest was “to be part of something entrepreneurial. I wanted to start something that was an expressed need from the schools,” he adds.

In 2004, Maycock and his mentor, Marci Cornell-Feist, assembled leaders from 10 Boston charter schools around the idea for Achievement Network. Cornell-Feist is the founder of the High Bar, which helps charter boards with management and governance issues.

The Boston charters had begun using interim assessments to prepare their kids for the year-end Massachusetts Comprehensive Assessment System, or MCAS. But the interim tests from outside vendors weren’t as rigorous as, or even aligned with, the MCAS. “They weren’t setting up the school leaders and teachers for success,” Maycock says.

The charters told him they needed better assessments, better data, and help understanding how to use the information, he says. They wanted a common assessment so they could compare results among themselves and use the data to identify best practices. And they wanted assessments that would serve as an instructional tool and not another gotcha mechanism to punish teachers.

Maycock raised $200,000 in seed money from a Massachusetts foundation, but also asked the schools each to pitch in $5,000 “to make it count,” he says. Schools now pay on a sliding scale: those like Hope that are in their first year and need intensive coaching pay $30,000. That declines to $14,000 a year once schools have been in the network for a few years and need less coaching.

Seven charter middle schools signed up with ANet in the 2005–06 school year, its first. Massachusetts had released the MCAS questions for the first time, and Maycock separated them by standard and skill, dissected them for rigor, and wrote his own interim assessments that mirrored the state exam.

James Peyser, a partner in NewSchools Venture Fund, which has invested $1.4 million in ANet and holds a seat on its board, says ANet’s assessments are remarkable for their rigor, which he adds are aimed at readying kids for college, not just for the state tests.

Three Boston district schools joined in ANet’s second year after catching wind of it. Maycock formed a second network of charter schools in Washington in 2008, and nine D.C. district schools joined the next year with help from the Dell grant. There are now 74 schools in the D.C. network.

New Orleans, Newark, Chicago, New York City, and Nashville-Memphis have since launched networks. There’s a network of three virtual schools, and a Baltimore network is planned for 2012. ANet says that 250 schools with some 70,000 kids were members of its networks in the 2011–12 school year. The organization has revenues of $9 million this school year, including $6 million in school fees.

Testing has expanded from the initial grades 6 and 7 to cover grades 3 through 8; ANet is piloting interim assessments for 2nd graders and a set of science tests. High school interims are more complicated because of wider course offerings, but they are “on our radar to consider—very much so,” Maycock says.

In 2010, ANet won a competitive $5 million Investing in Innovation (i3) grant from the U.S. Department of Education, which it is using, in part, to fund a large randomized study of its impact.

In its own analysis, ANet says the number of its youngsters who scored proficient or above on state tests last year increased by 7 percentage points in English and 4 percentage points in math in Chicago, and by 5 points in English and 3 points in math in New Orleans. Of the six cities for which it reported scores last year, ANet said four made twice the gains in English as the rest of their respective states, and three made double the state gains in math.

In D.C., about 6,600 youngsters in ANet’s charter and district schools took year-end tests in 2011. ANet says those scoring proficient in English increased by 4.5 percent and in math by 9 percent from the year earlier. That translates into 319 more kids passing the language exam and 662 more passing math, numbers Maycock calls “huge.” In just the D.C. district ANet schools, the increases were smaller—4 percent in English and 6.6 percent in math—but still better than the improvement of less than 2 percent posted by district schools that didn’t partner with ANet.

Network Strength

A learning walk explores peer-group feedback, or how to get teachers to help one another figure out how to reteach a troublesome lesson.

The schools in ANet’s original network were a lot alike: urban with high-need populations. Maycock has recently convinced stronger schools to join each network; in D.C., Janney and Horace Mann Elementary Schools, which are among the district’s highest-performing, white-majority schools, joined a network that is generally minority and struggling. The idea is to get charters and district schools, and stronger and weaker schools—schools that don’t generally cross paths—to share ideas and goad each other to improve.

Network schools have access to each other’s grade-level data, they share ANet coaches, and they’re invited to regular “learning walks,” where one network school models a practice for other network members.

A few days after the data-day review, I visited Powell Elementary, a district school in northeast D.C., for a learning walk on peer-group feedback, or how to get teachers to help one another figure out how to reteach a troublesome lesson. Teachers, data and instructional coaches, and a principal from eight widely different schools attended.

The practice Powell was showing off involved having its teachers present their reteaching plans—developed on data day—to a handful of teachers from other grades and specialties. These “critical friends” ask “clarifying questions” about the plan, and then talk it over among themselves. The presenting teachers can take or leave the suggestions without having to defend their lesson plans.

As I listened, a Powell math teacher modeled the process while the visitors leaned in close and tossed out their own ideas. Consider a math competition, said the dean of an all-boys, entirely African American charter school that seemed to have little in common with Powell: “Kids respond well to that.” Identify the 10 words most commonly used in word problems, said a math specialist from a district school that seemed to mirror Powell’s English-learner enrollment.

“I hadn’t thought about using manipulatives” in the lesson, conceded the Powell teacher as the ideas rolled in—and his kids would benefit from a hands-on lesson that burned up some of their energy, he added. After two hours, with the learning walk long ended, a dozen teachers from around the network were still huddled together, still talking lesson plans.

Powell keeps an ANet data wall in its front lobby and records how many youngsters in each class score proficient or advanced in math and in language arts for each ANet assessment cycle. Powell’s parents attend a data meeting when the results come out each cycle, and “all but three or four” regularly attend, principal Janeece Docal told me.

Powell’s highly public use of the data contrasts with that of Hyde-Addison Elementary, a third-year ANet school in D.C.’s swank Georgetown neighborhood, which uses the ANet data only internally. “We see what you know and what you don’t know. We see what we’ve taught you,” principal Dana Nerenberg told me.

Powell links the data discussion to the kids’ future, Docal explained: good ANet scores translate into good scores on the year-end test, which will land the youngsters in the high school and then the college and then the job of their choice. “Education equals freedom,” she said a dozen times over the afternoon.

How schools use the data “depends on the school’s culture,” says Justin Jones, a former Teach For America corps member and recruiter who heads the D.C. network.

Peyser, at NewSchools Venture Fund, says the goal is to help “change and strengthen school culture toward data” until “it becomes the way they do business.”

June Kronholz is an Education Next contributing editor.

Raj Chetty, John Friedman, and Jonah Rockoff have carried out a remarkable study, but I suspect it will be misinterpreted.

The main contribution of their research is quantifying the importance of teaching. Specifically, the authors conclude that students taught by a more effective teacher will collectively earn hundreds of thousands of dollars more over their lifetimes, and that good teachers similarly influence college going and teenage pregnancy. Because each teacher influences thousands of students over a career, this suggests that one excellent teacher could generate enormous social and economic benefits.

I find these results plausible, though there are some real limitations. The researchers present convincing evidence that their estimates of teacher contributions to student achievement are valid and do not simply reflect differences in student background. But this type of “selection bias” could influence effects on earnings and other long-term outcomes. So, the most intriguing findings here are also still somewhat tenuous. Given the small size of the effects for each individual student, even a slight bit of selection bias could dramatically alter the estimated benefits of an individual teacher.

Perhaps the more important question is, what do the results mean for policy? Policymakers had already concluded that we need to do more to improve teaching. As a result, schools and districts around the country are now experimenting with a wide range of policies to improve teacher performance measures and use these to make high-stakes decisions such as dismissing low-performing teachers.

And here is the rub. The authors, recognizing the interest in dismissing low performers, conduct a simulation of such a policy and emphasize these results in their summary. But it would be a mistake to interpret even these careful simulation results as evidence about actual policies. The effects of actual policies never play out the way simulations suggest, because policies are rarely implemented as intended and the inevitable secondary effects are hard to predict.

There are substantial legal, political, and organizational problems associated with dismissing low performers. For example, in a simple system, many teachers would be fired unjustifiably as a result of imprecision in the performance measures—a lawsuit waiting to happen. High stakes associated with the tests will inevitably distort student scores and the assignment of students to teachers, worsening the measurement problem. A more elaborate evaluation system can address this measurement problem, but such systems are costly, and those costs are not considered here. Such an approach could also change the makeup of the profession, in both positive and negative ways.

There is good reason to think that dismissing more low-performing teachers would improve student outcomes, but the Chetty study is not designed to tell us much about that, or about any of the various policy alternatives. What it does provide is the best evidence yet that teachers matter a great deal and that we should continue looking hard for ways to improve teaching and learning in schools.

Douglas Harris is associate professor of educational policy studies at the University of Wisconsin-Madison.

Return to “Great Teaching” (Summer 2012)

Over the last decade, research in public education has led us to three conclusions about the teaching profession: teachers are the most important in-school factor in determining student achievement; there is wide variation in teacher effectiveness; and those differences really matter for kids.

These findings should have profound implications for policymakers and practitioners. Now that we have evidence attesting to the enormous contributions of the most effective educators, if we are truly serious about improving student learning and closing the achievement gap, we must think anew about teacher recruitment, placement, evaluation, professional development, retention, and separation.

Raj Chetty, John Friedman, and Jonah Rockoff have helped advance the conversation through their longitudinal study of 2.5 million students over a 20-year span. The correlation between teacher effectiveness (as demonstrated by value-added student growth measures) and student life outcomes (higher salaries, advanced degrees, neighborhoods of residence, and retirement savings) is staggering; it’s not an exaggeration to say that great teachers substantially improve students’ future quality of life and those students’ contributions to the common good. Conversely, traditional education output measures like student course completion, grades, and diplomas have a substantial degree of subjectivity across schools and districts and can potentially provide a misleading account of a student’s college and career readiness.

In New Jersey, we are assessing where our finite resources are best invested. The Chetty study contrasts the opportunity cost of providing retention incentives to effective teachers with that of investments to attract new teachers. Similar cost/benefit questions arise in relation to shaping teacher-placement strategies, developing career ladders, and providing meaningful professional development. To make informed decisions in these areas, we first need to be able to differentiate among our teachers and, ideally, identify strengths to build on and weaknesses to address. That’s why the foundation of our human-capital efforts is a new educator-evaluation framework that’s substantially based on student learning outcomes. If we are able to assess an educator’s effectiveness accurately, we can improve the array of policies and practices that influence our teachers and school leaders. The hallmark of these efforts in our state will not be based on separating ineffective teachers but rather on using evaluation results to target resources toward improving teaching practice.

New Jersey is still in the early innings of this work. Eleven districts, through a pilot initiative, have joined with the state to create the new teacher-evaluation system. This collaboration has helped jump-start this work across the state and shed light on the many significant challenges associated with overhauling the hoary systems in place, such as measuring student achievement in “untested” grades and subjects, ensuring inter-rater agreement and accuracy of teacher practice observations, and ending the long-standing culture of “The Widget Effect.”

The primary takeaway from this critically important research, as the study authors note, is that “finding policies to raise the quality of teaching… is likely to have substantial economic and social benefits in the long run.” We agree with this conclusion, and New Jersey, like other states, must develop such policies over time through a confluence of national and local research, lessons learned from our classrooms, and an unwavering resolve to provide our students with high-quality teachers.

Chris Cerf is acting commissioner of education for the State of New Jersey. Peter Shulman is chief talent officer for the New Jersey Department of Education.

Return to “Great Teaching” (Summer 2012)

The new study by Raj Chetty, John Friedman, and Jonah Rockoff  asks whether high-value-added teachers (i.e., teachers who raise student test scores) also have positive longer-term impacts on students, as reflected in college attendance, earnings, avoiding teenage pregnancy, and the quality of the neighborhood in which they reside as adults. As a step on the way, the researchers investigate whether such teachers have been properly identified, that is, are the teachers who are producing larger achievement gains from year to year, according to value-added models, actually responsible for those gains? The paper contains valuable evidence indicating that the answer is yes. First, the authors obtain data on family background from federal tax returns not normally available to researchers. This allows them to measure family characteristics (such as parental income) not typically controlled for when teacher value-added is estimated. If introducing such factors reduces the explanatory power of teacher value-added, it is an indication that the value-added estimate was inflated, and that part of what had been attributed to the teacher was in fact due to favorable family circumstances. The study authors find that including such controls does not detract from the explanatory power of estimated value-added.

The authors also investigate whether high-value-added teachers have benefited by being assigned students who would have made greater gains on standardized tests for unobserved reasons (such as family factors that cannot be gleaned even from tax returns). This is normally difficult to do, given the possible influences on the way students are assigned to teachers. The report succeeds by focusing on average test gains in grades within schools where mean value-added within a grade has been affected by the movement of teachers in and out of the grade. What matters for this analysis is not which student was assigned to which teacher within the grade, but how the movement of teachers has altered the quality of teaching in that grade as a whole. It turns out that subsequent gains within these grades are close to those what would be expected from the change in mean teacher value-added. Provided the movement of teachers in and out of a grade has not changed the makeup of students enrolled in that grade, this finding supports the conclusion that measured value-added of teachers is an unbiased predictor of future test-score gains, as there appears to be no other explanation for the resulting improvement in test scores.

When the authors examine the association between teacher value-added and outcomes in young adulthood, however, for the most part they do not undertake the same tests to ensure that these associations are not artifacts of the way students are sorted among teachers. They do not introduce controls from tax returns to see whether the explanatory power of teacher value-added for later earnings, college attendance, and other factors, falls. Nor, with the exception of college attendance, do they test for the influence of unobservable factors in the manner just described.

The omission of such tests undercuts their claim to have demonstrated that high-value-added teachers contribute to better long-term outcomes. Without the same rigorous tests, we cannot be sure that the observed association between teacher value-added and long-term outcomes was not the result of other factors (for example, efforts made by parents with the strongest parenting skills to ensure their children were assigned to the most effective instructors). It is not enough to show that omitted family characteristics have not been confounded with value-added as a predictor of future test-score gains. The factors that shape test performance are not necessarily those that influence future earnings or the avoidance of a teenage pregnancy. Character education and the values parents impart to their offspring are likely to matter for the latter in ways that they do not for cognitive functioning.

In short, the authors provide a persuasive answer to the question: does a high-value-added teacher actually raise subsequent test scores? They have not so far provided equally persuasive evidence answering the question: does a high-value-added teacher improve subsequent life outcomes?

Dale Ballou is associate professor of public policy and education at Vanderbilt University and associate director of the National Center on Performance Incentives.

Return to “Great Teaching” (Summer 2012)

The movie Waiting for Superman chronicles the role of chance in determining the fate of a relatively small number of families trying to enroll their children in oversubscribed charter schools. Raj Chetty, John Friedman, and Jonah Rockoff document the much larger problem of ineffective teachers scattered about a multitude of schools. From the viewpoint of the student, this latter issue may appear to be chance when class assignments are made, and when some get good teachers and others get ineffective ones. From the standpoint of the system, however, it is not chance but mismanagement that allows ineffective teachers to continue harming students.

Chetty et al. have produced new and elegant estimates of how teacher effectiveness relates to long-run student outcomes. As economists are prone to do, they have produced a paper that deals with a long list of technical questions that have absorbed the scientific literature on teacher effectiveness. Their work is thorough, convincing, and scientifically innovative.

The overarching idea of the paper is linking gains from having a high-value-added teacher in grades 4–8 to subsequent long-run outcomes, including college attendance, earnings, and family creation. But, from the outset, they must deal with the two primary challenges leveled at teacher value-added measures based on student test scores. First, are these  estimates biased measures of effectiveness? The answer is no. The wealth of information that Chetty et al. have about families from tax records and some clever analyses effectively rule out the possibility that conventional estimates of value-added based only on school administrative data are misleading. Second, do the effects of good teachers (or bad teachers) quickly fade away? Again, the answer is no. Even as these students leave school and enter into adult careers in their late 20s, the significant trace of their early schooling is quite discernible.

But the warranted attention to this work derives not from its technical aspects but from the policy implications of the results. The fundamental finding is that good teachers have an extraordinarily powerful impact on the future lives of their students. Symmetrically, the researchers show the lasting damage that poor teachers have on the lives of their students. This work sweeps away a variety of attempts to deflect questions about the importance of teacher quality and our ability to identify it. It also brings us back to the question of informed policy.

As the evidence on the importance of teacher quality has grown, policy discussions have actually moved. In the beginning, there were doubts about the impact of teacher quality relative to families, curriculum, or a host of other influences. Those doubts have largely receded and been replaced by questions of how policy should proceed. And here is where the additional evidence presented in the Chetty study comes into play.

Much of the discussion has centered on the political difficulties of reforming the schools by dealing with the problem of the most ineffective teachers. The unions have dug in their heels, resisting any change that does not ensure perfect identification of the worst teachers. Their resistance has resulted in many policymakers simply asserting that it is too politically costly to make active decisions about teacher effectiveness and instead looking to alternatives such as more professional development, better mentoring, or heightened requirements of certification.

Chetty et al.’s evidence shows that bad teachers cost hundreds of thousands of dollars in lost income and productivity each year that they remain in the classroom. These costs are large enough that failing to address them is simply inexcusable. It is time that we develop policies that truly are designed to help our children and not just the adults in schools today.

We have recently seen a number of brave states step out and legislate better evaluations of teachers including, when possible, the use of value-added measures. Coupled with both pay and tenure reforms, these movements show real promise and should be encouraged on a wider scale.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University.

Return to “Great Teaching” (Summer 2012)

Birdette Hughey is the 2011 Mississippi Teacher of the Year.

In February 2012, the New York Times took the unusual step of publishing performance ratings for nearly 18,000 New York City teachers based on their students’ test-score gains, commonly called value-added (VA) measures. This action, which followed a similar release of ratings in Los Angeles last year, drew new attention to the growing use of VA analysis as a tool for teacher evaluation. After decades of relying on often-perfunctory classroom observations to assess teacher performance, districts from Washington, D.C., to Los Angeles now evaluate many of their teachers based in part on VA measures and, in some cases, use these measures as a basis for differences in compensation.

Newspapers that publish value added measures no doubt relish the attention they generate, but the bigger question in our view is whether VA should play any role in the evaluation of teachers. Advocates argue that the use of VA measures in decisions regarding teacher selection, retraining, and dismissal will boost student achievement, while critics contend that the measures are a poor indicator of teacher quality and should play little if any role in high-stakes decisions. The Obama administration has thrown its weight squarely behind the advocates, launching a series of programs that encourage states to develop evaluation systems based substantially on VA measures.

The debate over the merits of using value added to evaluate teachers stems primarily from two questions. First, do VA measures work? In other words, do they accurately capture the effects teachers have on their students’ test scores? One concern is that VA measures will incorrectly reward or penalize teachers for the mix of students they get if students are assigned to teachers based on characteristics that VA analysis typically ignores.

Second, do VA measures matter in the long run? For example, do teachers who raise test scores also improve their students’ outcomes in adulthood or are they simply better at teaching to the test? Recent research has shown that high-quality early-childhood education has large impacts on outcomes such as college completion and adult earnings, but no study has identified the long-term impacts of teacher quality as measured by value added.

We address these two questions by analyzing school-district data from grades 3–8 for 2.5 million children, linked to information on their outcomes as young adults and the characteristics of their parents. We find that teacher VA measures both work and matter. First, we find that VA measures accurately predict teachers’ impacts on test scores once we control for the student characteristics that are typically accounted for when creating VA measures. Second, we find that students assigned to high-VA teachers are more likely to attend college, attend higher-quality colleges, earn more, live in higher socioeconomic status (SES) neighborhoods, and save more for retirement. They are also less likely to have children during their teenage years.

Teachers in all grades from 4 to 8 have large impacts on their students’ adult lives. On average, a 1-standard-deviation improvement in teacher value added (equivalent to having a teacher in the 84th percentile rather than one at the median) in a single grade raises a student’s earnings at age 28 by about 1 percent. Replacing a teacher whose value added is in the bottom 5 percent with an average teacher would increase students’ total lifetime incomes by more than $1.4 million for a typical classroom (equivalent to $250,000 in present value). In short, good teachers create substantial economic value, and VA measures are useful in identifying them.

Our findings address the three main critiques of VA measures raised in a recent Phi Delta Kappan article by Stanford education professor Linda Darling-Hammond and her colleagues. We show directly using quasi-experimental tests that standard VA measures are not biased by the students assigned to each teacher. Hence, value-added metrics successfully disentangle teachers’ impacts from the many other influences on student progress. We also show that although VA measures fluctuate across years, they are sufficiently stable that selecting teachers even based on a few years of data would have substantial impacts on student outcomes such as earnings.

Students assigned to high-VA teachers are more likely to attend college, earn more, live in higher socioeconomic status neighborhoods, and save more for retirement.

Data

We draw information from two sources: school-district records on students and teachers, and information on the same students and their parents from administrative data sources such as tax records. The school-district data contain student enrollment history, test scores, and teacher assignments from the administrative records of a large urban school district. These data span the school years 1988–89 through 2008–09 and cover roughly 2.5 million children in grades 3 through 8.

The school-district data include approximately 18 million test scores. Test scores are available for English language arts and math for students in grades 3–8 from the spring of 1989 to 2009. In the early part of the sample period, these tests were specific to the district, but by 2005–06 all tests were statewide, as required under the No Child Left Behind law. In order to calculate results that combine scores from different tests, we standardize test scores by subject, year, and grade. The district data also contain other information on students, such as race or ethnicity, gender, and eligibility for free or reduced-price lunch (a standard measure of poverty).

Our data on students’ adult outcomes include earnings, college attendance, college quality (measured by the earnings of previous graduates of the same college), neighborhood quality (measured by the percentage of college graduates in their zip code), teenage birth rates for females (measured by claiming a dependent born when the woman was still a teenager), and retirement savings (measured by contributions to 401[k] plans). Parent characteristics include household income, marital status, home ownership, 401(k) savings, and mother’s age at child’s birth.

Having spent a single year in the classroom of a teacher with value added that is 1 standard deviation higher increases earnings at age 28 by about 1 percent. If that 1 percent advantage were to remain stable throughout an individual's career, it would add up to about $25,000 in total earnings.

Do Value-Added Measures Work?

Value-added analysis aims to isolate the causal effects teachers have on student achievement by comparing how well their students perform on end-of-year tests relative to similar students taught by other teachers. These comparisons take into account students’ test scores in the prior year as well as their race or ethnicity, gender, age, suspensions and absences in the previous year, whether they repeated a grade, special education status, and limited English status. We also control for teacher experience as well as for class and school characteristics, including class size and the academic performance and demographic characteristics of all students in the relevant classroom and school.

Many other researchers use methods for measuring teacher value added that are similar to ours, so it is not surprising that we obtain similar results. For example, we find that a 1-standard-deviation increase in teacher value added corresponds to increases in student math and English scores of 12 and 8 percent of a standard deviation, respectively. In both subjects, this difference is equivalent to approximately three months of additional instruction.

Can we take this as evidence of teachers’ causal impact on student test scores? Recent studies by economists Thomas Kane, Doug Staiger, and Jesse Rothstein, among others, have reached divergent conclusions about whether VA measures should be interpreted in this way. In particular, critics contend that VA measures are likely to be biased as a result of the way that students are assigned to teachers. For example, some teachers might be consistently assigned students with higher-income parents (which typically cannot be accounted for by school districts when generating VA measures because they do not collect precise data on family income). We implement two new tests to determine whether VA estimates are biased.

Our first test examines whether in fact high-VA teachers tend to be assigned students from more-advantaged families. We calculate an overall measure of parents’ socioeconomic status, combining the parental characteristics listed above. Not surprisingly, parent socioeconomic status is strongly predictive of student test scores, and, looking at simple correlations, we find that less-advantaged students do tend to be assigned to teachers with lower VA measures. However, controlling for the limited set of student characteristics available in school-district databases, such as test scores in the previous grade, is sufficient to account for the assignment of students to teachers based on parent characteristics. That is, if we take two students who have the same 4th-grade test scores, demographics, classroom characteristics, and so forth, the student assigned to a teacher with higher VA in grade 5 does not systematically have different parental income or other characteristics.

This first test shows that any bias in VA estimates due to the omission of parent characteristics that we are able to observe is minimal. The possibility remains, however, that students are assigned to teachers based on unmeasured characteristics unrelated to parent socioeconomic status. For example, principals may consistently assign their most-disruptive students to teachers whom they believe are up to the challenge. Alternatively, principals might assign these same students to their least-effective teachers, whom they are not worried about losing. Our second test seeks to determine the amount of bias introduced by this kind of sorting.

To do so, we exploit the fact that adjacent grades of students within the same school are frequently assigned to teachers with very different levels of value added because of idiosyncrasies in teacher assignments and turnover. During our analysis period, roughly 15 percent of teachers in our data switched to a different grade within the same school from one year to the next, 6 percent of teachers moved to a different school within the same district, and another 6 percent left the district entirely. These year-to-year changes in the teaching staff at a given school generate differences in value added that are unlikely to be related to student characteristics.

To illustrate, suppose a high-VA 4th-grade teacher enters a school at the beginning of a school year. If VA estimates capture teachers’ true impact on their students, students entering grade 4 in that school should have higher year-end test scores than those of the previous cohort. And the size of the change in test scores across these consecutive cohorts should correspond to the change in the average value added across all teachers in the grade. For example, in a school with three equal-sized 4th-grade classrooms, the replacement of a teacher with a VA estimate of 0.05 standard deviations with one with a VA estimate of 0.35 standard deviations should increase average test scores among 4th-grade students by 0.1 standard deviations.

In fact, that is exactly what we find, as shown in Figure 1. To construct this figure, we first define the top 5 percent of teachers as “high VA” and the bottom 5 percent as “low VA.” Figure 1 displays average test scores for cohorts of students in the years before and after a high-VA teacher arrives. We see that end-of-year test scores in the subject and grade taught by that teacher rise immediately by about 4 percent of a standard deviation. This impact on average test scores is commensurate in magnitude with what we would have predicted given the increase in average teacher value added for the students in that grade.

We obtain parallel findings when we examine the departure of high-VA teachers and the entry and exit of low-VA teachers. When a high-VA teacher leaves a given subject-grade-school combination, test scores of subsequent students in that subject, grade, and school fall. Likewise, students benefit from the departure of a low-VA teacher and are harmed by the arrival of a low-VA teacher.

Together, these results provide direct evidence that removing low-VA teachers (bottom 5 percent) and retaining high-VA teachers (top 5 percent) improves the academic achievement of students. But what about the remaining 90 percent of teachers? When we perform a similar analysis for all teachers, we again find that changes in the quality of the teaching staff strongly predict changes in test scores across consecutive cohorts of students in the same school, grade, and subject. Moreover, in middle schools, where students usually learn math and English from different teachers, we confirm that the arrival or departure of math teachers affects math scores but not English scores (and vice versa).

Using these techniques, we can calculate the amount of bias in our VA estimates. We find that the degree of bias is, on average, less than 2 percent. We therefore conclude that standard VA estimates accurately capture the impact that teachers have on their students’ test scores. Although the results could differ in other settings, our method of using natural teacher turnover to evaluate bias in VA estimates can be easily implemented by school districts to evaluate the accuracy of their VA models.

Do Value-Added Measures Matter?

Even though value-added measures accurately gauge teachers’ impacts on test scores, it could still be the case that high-VA teachers simply “teach to the test,” either by narrowing the subject matter in the curriculum or by having students learn test-taking strategies that consistently increase test scores but do not benefit students later in their lives. To address this issue, we measure the relationship between teachers’ value added and their students’ outcomes in adulthood. We compare students who were assigned high-VA vs. low-VA teachers in grades 4–8 and study their outcomes in adulthood.

We find that high-VA teachers raise students’ chances of attending college at age 20 (see Figure 2a). A student assigned to a teacher with a VA 1 standard deviation higher is 0.5 percentage points more likely to attend college at age 20 (an increase of 1.3 percent). Students of higher-VA teachers also attend higher-quality colleges, as measured by the average earnings of previous graduates of those colleges.

A person’s income doesn’t begin to stabilize until their late twenties, so our analysis of earnings focuses on the year when students were 28, the oldest age at which we observe a sufficiently large number of students. We find that having spent a single year in the classroom of a teacher with value added that is 1 standard deviation higher increases earnings at age 28 by $182, or about 1 percent (see Figure 2b). If that 1 percent advantage were to remain stable throughout an individual’s career, it would add up to about $25,000 in total earnings.

In addition to improved earnings, we also find that improvements in teacher value added significantly reduce the likelihood that female students will have a child during their teenage years, increase the socioeconomic status of the neighborhoods in which students live in adulthood, and raise 401(k) retirement savings rates. Moreover, it is likely that improved education would yield benefits that we are not able to measure but have been shown by other studies, such as reduced crime and improved citizenship.

To sum up, our evidence confirms that the students of high-VA teachers benefit not just by scoring higher on math and reading tests at the end of the school year, but also through improved outcomes later in life. The size of these effects may seem small, but recall that they reflect the impact of a higher-VA teacher for a single year and could compound over time to the extent that students are exposed to multiple high-VA teachers. As important, a single high-VA teacher has this effect not only on a single student but rather on an entire classroom—and often on many classrooms of students over the course of a career.

Wanda Booth, Florida's 2011 Charter School Teacher of the Year, works with students. Teachers in all grades have large impacts on their students' adult lives.

Policy Implications

In a recent article (see “Valuing Teachers,” features, Summer 2011), Eric Hanushek argues in favor of dismissing the bottom 5 percent of teachers based on their VA scores. While such a policy would have many costs and benefits that are beyond the scope of our study, we can illustrate the magnitudes implied by our analysis by calculating its impacts on students’ earnings. Our estimates imply that replacing a teacher whose value added is in the bottom 5 percent with an average teacher would increase students’ cumulative lifetime income by a total of $1.4 million per classroom taught. This gain is equivalent to $267,000 in present value at age 12, discounting at a 5 percent interest rate. However, it is important to realize there is uncertainty in VA measures, which are estimates that may be based on only a few classrooms of students, so the gains from removing teachers identified as ineffective based on a limited number of years of data are smaller. We estimate the gains from “deselecting” the bottom 5 percent of teachers to be approximately $135,000 in present value based on one year of data and $190,000 based on three years of data. These benefits, while still large, would have to be weighed against any costs associated with the policy, such as teachers demanding higher pay to compensate them for the risk of dismissal.

We also measure the expected gains from policies that pay higher salaries or bonuses to high-VA teachers in order to increase retention rates. The gains from such policies appear to be only somewhat larger than their costs. Although the benefit from retaining a teacher whose value added is at the 95th percentile after three years is nearly $200,000 per year, most bonus payments end up going to high-VA teachers who would have stayed even without the additional payment. Replacing low-VA teachers is therefore likely to be a more cost-effective strategy to increase teacher quality in the short run than paying to retain high-VA teachers. In the long run, higher salaries could attract more high-VA teachers to the teaching profession, a potentially important benefit that we do not measure here.

While these calculations illustrate the magnitudes of teachers’ impacts on students, they do not by themselves offer a blueprint for the design of optimal teacher evaluations, salaries, or merit-pay policies. Teachers were not evaluated based on test scores in the school district and time period we study. VA measures may not be as useful for identifying teachers with positive long-term impacts on their students if teachers respond to their use in evaluation systems by engaging in practices such as teaching to the test or even outright cheating. In addition, our analysis does not compare value added with other measures of teacher quality, like evaluations based on classroom observation, which might be even better predictors of teachers’ long-term impacts than VA scores.

In summary, our research demonstrates that good teachers are of great value to their students, and that VA measures are a potentially valuable tool for measuring teacher performance. The most important lesson we draw is that finding policies to raise the quality of teaching is likely to yield substantial economic and social benefits.

Raj Chetty is professor of economics at Harvard University. John N. Friedman is assistant professor of public policy at Harvard Kennedy School. Jonah E. Rockoff is associate professor of business at Columbia University’s Graduate School of Business. For further information on the study, see http://obs.rc.fas.harvard.edu/chetty/value_added.html.

Commentary

In light of the widespread attention given to the Chetty, Friedman, and Rockoff research, Education Next asked four experts to comment on the study’s implications for teacher policy.

Implications for Policy Are Not So Clear - By Douglas Harris
Profound Implications for State Policy - By Chris Cerf and Peter Shulman
More Evidence Would Be Welcome - By Dale Ballou
Low-Performing Teachers Have High Costs - By Eric A. Hanushek

He has more to say about a larger strategy for boosting teacher quality in “An Effective Teacher in Every Classroom,” which appeared in the Summer 2010 issue of Ed Next.

He also authored “Valuing Teachers: How Much is a Good Teacher Worth?” which appeared in the Summer 2011 issue of Ed Next.

The Chetty study shows that over a ten year period, the payoff for the students of a very effective teacher amounts to a total of $2.5 million. The harm done by a very ineffective teacher is the same. So if we could replace a terrible teacher with a great one, it would be worth $5 million total for all those kids affected by the switch.  And losing a great teacher, only to hire a bad one, would cost the same.   That’s convincing evidence for those who want to limit the tenure of non-performing teachers while giving the excellent ones their just reward.

But unions want to protect teacher tenure and pay all teachers the same, regardless of effectiveness.  So denying the Chetty study is absolutely crucial.

Though he lacks the necessary econometric skills, Michael Winerip takes up the assignment, claiming the data on teacher effectiveness, which comes from student testing during the 1990s, is too old to tell us anything.

But to ascertain the impact of teaching on student earnings that occur much later in life, it is of course necessary to look at those educated in the 1990s.   Those students have now finished high school (or not), gone to college (or not), and entered the work force (or not).  For today’s students, no one has that information–for the obvious reason that they are still too young.

Aha! says Mr. Winerip. That is the fatal flaw. Back in the 1990s, when students took standardized tests, No Child Left Behind did not exist, so “whether those results are applicable to our post-2004 high-stakes world, we cannot tell.”

If we are to buy this argument, the data will always be too old to tell us anything.  To learn what works we have to wait twenty years, and when that data is available, it will be just too old.

But is it?  Why should we assume that the tests taken back in the 1990s were more accurate than the post-NCLB tests given in 2005, when both teachers and students took them more seriously.  Student performance is more accurately measured when students take a test seriously and when teachers make sure the students understand the testing procedures to be followed. All that is more likely when tests count for something.

So if Chetty and his colleagues could identify large impacts of effective teaching using data from the 1990s, his successors will probably find even larger impacts from more accurate information gathered in the first decade of the 21^st century.

Of course, I cannot prove that, but it is certainly more likely than Winerip’s counter-hypothesis.  While he admits the 1990s tests were accurate, he claims tests today no longer are.  Only if Winerip is willing to make the astounding claim that most teachers today are cheating deliberately and systematically does that assertion hold. Otherwise, we can characterize his argument in one word:  Silly.

- Paul Peterson

Kipp shares her experience teaching 11th and 12th grade English online while she resides with her family in rural Colorado. Though not physically in a classroom, Kipp has been able to successfully engage students through live class sessions, emails, instant messaging, and texting. Kipp used to teach in a traditional classroom setting but says that despite some of the unique challenges teaching virtually presents, she finds the online teaching experience more rewarding and in many instances more effective.

My take is threefold. (An earlier version of this originally appeared as an Ed Week commentary, but I thought it worth sharing a tweaked version here.)

First, claims that teachers are, in Duncan’s words, “desperately underpaid,” are a familiar refrain. Yet, given that we’ve steadily boosted staffing and after-inflation spending in recent decades to little obvious effect, and that states and districts are wrestling with structural shortfalls, it’s healthy to question such orthodoxies. Biggs and Richwine remind us that the costs of teacher benefits dramatically inflate the cost of compensation, even if the results aren’t always obvious when scanning a paycheck. Recall, for example, that University of Arkansas economist Bob Costrell pointed out during the Wisconsin collective bargaining fight earlier this year that the average Milwaukee teacher earned a salary of $56,500 but, due to benefits, actually cost the district $100,005 in total compensation. This ought to be of particular concern to educators eager to see more of their compensation show up in their pay stubs. In light of that, I’m disappointed (if not surprised) that most of the responses I’ve seen to Biggs and Richwine have been ad hominem, with Duncan declaring in the Huffington Post that the study “insults teachers and demeans the profession.”

Second, their analysis is intriguing, but it rests upon assumptions and data which deserve to be carefully scrutinized. For instance, Biggs and Richwine rely upon SAT and GRE scores to measure cognitive ability. It’s fair to ask both how good those metrics are and how much they may say about teaching ability. And it’s worth noting that their cognition data are nearly two decades old; if the makeup of the teaching force has changed significantly in that time, it would obviously change the outcomes. Similarly, the job-security and benefits data don’t reflect more recent developments or the fact that teaching positions may be less secure going forward; it will be interesting to see how such changes might impact the underlying data. At the same time, it’s important to note that Biggs and Richwine penned for the HuffPo what I thought was a pretty compelling response to the two methodological criticisms that Duncan had raised.

Third, I ultimately think the are-teachers-overpaid-or-underpaid question is just not that interesting or helpful to those of us in the fields of schooling and education. It’s a useful question for policymakers who must decide how to allocate dollars for highways, health care, and schooling, but for those of us working in the K-12 arena, the more relevant question is: How do we most wisely spend the dollars we have?

For what it’s worth, I’m firmly convinced that, today, some teachers are underpaid and others are overpaid. When I am asked the long-standing question about whether teachers are underpaid or overpaid, my consistent refrain is, “Yes.” I’m much more interested in the broader issue of how we can rethink the profession, make fuller use of talented teachers, and wisely spend the dollars we do have than in debating what the “right” wage level should be.

Under today’s step-and-lane pay scales, the primary way we determine how much teachers are worth is how long they’ve taught and how many graduate credits they’ve accumulated. Now, there’s nothing innately wrong with step-and-lane compensation. Indeed, when introduced in the early 20th century, it was a sensible response to reflexive, sweeping discrimination under which women were routinely paid half as much as their male counterparts. When a captive market of women had few options except to teach, the benefits of this more equitable system outweighed its defects.

Today, however, the world has changed. Whereas limited professional options meant that more than half of women graduating from college became teachers in mid-20th-century America, the figure today is closer to 15 percent. At the start of the 21st century, new college graduates–both men and women–are much less likely to stick to a job for long stretches, the competition for college-educated talent has intensified, and we are becoming better able to track educational outlays and outcomes. All this adds up to a new environment in which step-and-lane industrial-era pay is ill-suited to attracting and retaining talent. The consequence of treating different employees similarly, despite their varying work ethics and skills, has become a growing burden.

As school systems wrestle with tough fiscal decisions, it’s vital to understand that one-size-fits-all pay is insensitive to questions of productivity. Although the term “productivity” is typically regarded as a four-letter word in K-12 conversations, teacher productivity means nothing more than how much good a given teacher can do. If one teacher is regarded by colleagues as a far more valued mentor than another, or helps students master skills much more rapidly than another, it’s axiomatic that one teacher is more productive than the other. Yet, step-and-lane pay makes no allowance for such differences.

Today, we’re paying the most productive employees too little, paying their less productive colleagues too much, or, most times, a little of each. In a world of scarce talent and limited resources, this is a problem. School systems casually operate on the implicit assumption that most teachers are similarly adept at everything. In a routine day, a 4th grade teacher who is a terrific English language arts instructor might teach reading for just 90 minutes. This is an extravagant waste of talent, especially when one can stroll down the hallway and see a less adept colleague offering 90 minutes of pedestrian reading instruction.

One approach to using talent more wisely might entail overhauling teacher schedules and student assignment so that an exceptional 4th grade English language arts instructor would teach many more students. Colleagues, in turn, would shoulder that teacher’s other instructional responsibilities. An essential component of such rethinking is to adjust compensation to recognize the importance of their various roles.

After all, we pay thoracic surgeons much more than we do pediatric nurses–not because we think they’re better people or because they have lower patient-mortality rates, but because their positions require more sophisticated skills and more intensive training and because surgeons are harder to replace. Salary should be a tool for solving problems by finding smarter ways to attract, nurture, and use talent; it should not be an obstacle to doing so.

Almost any effort to really rethink staffing and pay entails some educators earning more–probably, a lot more–and other educators earning less. That sounds about right. The real question isn’t whether we should pay all teachers more or less; it’s how to pay the right teachers more, in a way that serves students and maximizes the bang we get for the educational buck.

-Frederick Hess

This post also appears on Rick Hess Straight Up.

Schools will not need as many conventional teachers as they did yesterday, but those they need will be able to tap top-notch technology and instructional support teams to achieve excellence at scale. They’ll get paid more, too, potentially a lot more. And all this can be done within tight budgets so long as education systems judiciously blend technology and people.

Digital learning has the potential to transform the teaching profession in three major ways:

• Extending the reach of excellent teachers to more students.
• Attracting and retaining more excellent teachers.
• Boosting effectiveness and job options for average teachers.

Extending the reach of the best. In the digital future, teacher effectiveness will matter even more than it does today. As digital learning spreads, students worldwide will gain access to core knowledge and skills instruction. What will increasingly differentiate outcomes for schools, states, and nations is how well responsible adults carry out the more complex instructional tasks: motivating students to go the extra mile, teaching them time management, addressing social and emotional issues that affect their learning, and diagnosing problems and making the right changes when learning stalls.

The top 20 or 25 percent of teachers already meet these challenges. But in traditional classrooms, they only reach 20 to 25 percent of students. That’s where digital learning can help.

Digital technology, along with changes in teacher roles and schedules, should make it possible for top teachers to assume responsibility for all students, not just 20 or 25 percent of them

For example, by replacing 25 – 50 percent of teaching in some subjects, digital instruction can free excellent teachers’ time, enabling them to take responsibility for more students – keeping similar class sizes and gaining planning time. These “time-technology swaps” are already used in top-performing schools that combine digital learning with excellent teachers to boost results.

Digital tools can also connect excellent teachers working live with students across the hall, state, or nation – using web cameras and email. Shy instructional masters can help design smart software to personalize learning. Star performing content masters can go viral on digital video, and someday holograms, to millions of students anywhere, who with excellent teachers can convert that access into stellar learning.

Attracting and retaining the best. Digital learning will also transform career opportunities for excellent teachers. As they reach more students, they should earn more out of the per-pupil funds generated by the expanded number of students. The chance of enhanced advancement and pay will, in turn, make the profession a more attractive long-term career for top performers, wooing unfulfilled engineers and lawyers into a better life.

-Bryan Hassel and Emily Asycue Hassel

This blog entry also appears on Flypaper. It is based on “Teachers the Age of Digital Instruction,” a paper published this week by the Fordham Institute as part of its Creating Sound Policy for Digital Learning series.

As a “practitioner” of sorts, I’ve wondered the same thing for 15 years. The K–12 school sector generates little empirical research of any sort. And of this small amount, most is targeted to policymakers and superintendents, and concerns such matters as the effects of class size reduction, charter school attendance, or a merit-pay program for teachers. Why is there virtually no empirical education research meant to be consumed by the nation’s 3 million teachers, answering their questions?

Those 3 million teachers generate about 2 billion hour-long classes per year. We do not know empirically which “teacher moves,” actions that are decided by individual teachers in their classrooms, are most effective at getting students to learn. Why doesn’t this kind of research get done?

Mr. Gates has part of the answer. Money. For 2011, the Microsoft R&D budget is $9.6 billion, out of total revenue in the $60 billion range. The U.S. Department of Education’s Institute of Education Sciences (IES) represents only a fraction of total education research, but its budget gives some perspective: IES spends about $200 million on research compared to more than $600 billion of total K–12 spending. So, 15 percent to upgrade Microsoft, 0.03 percent to upgrade our nation’s schools. And while Microsoft’s research is targeted to the bottom line ($8.6 billion is on cloud computing, the profit center of the future), IES spends almost nothing examining the most important aspect of schools: the decisions and actions that individual teachers control or make.

One IES project is the What Works Clearinghouse (WWC), established in 2002 to provide “a central and trusted source of scientific evidence for what works in education.” The WWC web site lists topic areas like beginning reading, adolescent literacy, high school math, and the like. For each topic, WWC researchers summarize and evaluate the rigor of published studies of products and interventions. One might find on the WWC site evidence on the relative effectiveness of middle-school math curricula or of strategies to encourage girls in science, for example.

But there is almost nothing examining the thousands of moves teachers must decide on and execute every school day. Should I ask for raised hands, or cold-call? Should I give a warning or a detention? Do I require this student to attend my afterschool help session, or make it optional? Should I spend 10 minutes grading each five-paragraph essay, 20 minutes, or just not pay attention to time and work on each until it “feels” done?

And the WWC’s few reviews of research on teacher moves aren’t particularly helpful. A 63-page brief on the best teaching techniques identifies precisely two with “strong evidence”: giving lots of quizzes and asking deep questions. An 87-page guide on reducing misbehavior has five areas of general advice that “research supports,” but no concrete moves for teachers to implement. It reads, “[Teachers should] consider parents, school personnel, and behavioral experts as allies who can provide new insights, strategies, and support.” What does not exist are experiments with results like this: “A randomized trial found that a home visit prior to the beginning of a school year, combined with phone calls to parents within 5 hours of an infraction, results in a 15 percent drop in the same misbehavior on the next day.” If that existed, perhaps teachers would be more amenable to proposals like home visits.

By contrast, a fair number of medical journals get delivered to my house. They’re for my wife, an oncologist. They’re practical. In each issue, she learns something along these lines: “When a patient has this type of breast cancer, I currently do X. This study suggests I should do Y.” There is a bit on medical policy, but most of the information is meant for individual doctors in their day-to-day work.

That’s not to say that we shouldn’t conduct research on education policy. My own work has certainly benefited from it. For example, the quasi-experimental study by economists Tom Kane and Josh Angrist on Boston charter schools, which compared the winners and losers of charter admission lotteries, helped change the Massachusetts law that had blocked the creation of new charters. The change enabled me to help launch a new charter school, MATCH Community Day. My point is simply that relative to education policy research, there is very, very little rigorous research on teacher moves. Why? Gates knows it’s more than a lack of raw cash; it’s also about someone taking responsibility for this work. “Who thinks of it [empirical research on teachers] as their business?” he asked. “The 50 states don’t think of it that way, and schools of education are not about [this type of] research.”

I agree, but I contend there are a number of other barriers. The first is a lack of demand.

The Demand Side

Why aren’t teachers clamoring for published research? One reason is that researchers generally examine the wrong dependent variable. Researchers care about next August (when test scores come in, because they can show achievement gains). Teachers care about that, too, but they care more about solving today’s problems (see sidebar, page 26).

A second issue is that researchers don’t worry about teacher time. Education researchers often put forward strategies that make teachers’ lives harder, not easier. Have you ever tried to “differentiate instruction”? When policy experts give a lecture or speak publicly, do they create five different iterations for their varied audience? Probably not.

The return on investment for teacher time and the opportunity cost of spending it one way rather than another is rarely taken into account. In what other, valuable ways could teachers be spending the time taken up with building “differentiation” into a lesson plan? They could phone parents, tutor kids after school, grade papers, or analyze data. Much research implies that teachers should spend more time doing X while not indicating where they should spend less time.

Teachers don’t trust research, and understandably so. There’s a lot of shoddy research that supports fads. Experienced teachers remember that “this year’s method” directly contradicts the approach from three years ago. So they’d rather go it alone. Newer teachers pick up on the skepticism about research from the veterans.

Unlike medical research, teacher research rarely examines possible side effects, and whether they are short-term aggravations or can be expected to persist. Imagine that a teacher reads an article arguing that students benefit from being asked “higher-order questions.” She begins doing that. Some students, surprised at this new rigor, are frustrated. Some students throw up their hands and give up. Misbehavior ensues.

Student frustration is probably a fairly predictable short-term side effect of asking higher-order questions. If she isn’t being properly warned, a teacher might quickly abandon this technique.

For all these reasons, the 3 million teachers aren’t forming picket lines to demand research.

Do We Know What Works?

Neither policy camp, reformers nor traditionalists, care much about research into teacher moves, either. Some traditionalists see teaching as an art, one that cannot be subjugated to quantitative analysis (“every teacher is different”). Others aren’t averse to research; they simply don’t see it as a priority. They’d prefer that limited resources be used to fight poverty, not to improve students’ day-to-day classroom experiences.

Meanwhile, some reformers argue “we already know what works,” and we just need to scale it.

As part of the “reformer” community, I find this troubling. From charter opponents like Diane Ravitch to supporters like education secretary Arne Duncan, there’s agreement that “some charter schools work.” Furthermore, there’s strong evidence that the charters that succeed tend to be “No Excuses” schools. So do we know what works?

I’m the founder of one of those charter schools; our high-school students have the highest value-added gains of all 340 public high schools in Massachusetts. I’m also the founder of a small teacher residency program that supplies teachers to schools like KIPP (Knowledge Is Power Program). Many of us would agree to a very different proposition: We know teacher moves “that work” to some extent, enough to create very large achievement gains, but we don’t know teacher moves well enough to get our college graduation rate near where we’d like it to be. Nor do we know how to help teachers do these moves more efficiently, so that their jobs are sustainable.

Without a massive uptick in our knowledge of teacher moves, we’ll continue on the current reform path. That path is a limited replication of No Excuses schools that rely on a very unusual labor pool (young, often work 60+ hours per week, often from top universities); the creation of many more charters that, on average, aren’t different in performance from district schools; districts adopting “lite” versions of No Excuses models while pruning small numbers of very low performing teachers; and some amount of shift to online learning. Peering into that future, I don’t see how we’ll generate a breakthrough.

Bridging the Divide

The final barrier to research on teacher moves is the divide between practitioners and researchers. My analogy is a 5th-grade dance. Boys stand on one side. Girls stand on the other. There is very little actual dancing. In this case, teachers are off to one side, and quantitatively oriented researchers are on the other.

After a while, the boys go into the hallway and talk about video games. Similarly, quantitative researchers find the transaction costs of setting up experiments are too high and give up on doing research about teacher actions. They take their problem-solving marbles and find other data sets to crunch.

Girls see that the boys aren’t around anymore. So they dance with each other. Teachers and school leaders, if they like to learn, do so through observation of and conversation regarding perceived “best practices.” There aren’t many practitioners who care about rigorous empirical research.

With all these barriers, is there much hope? There’s not going to be a pot of gold in this funding environment. If research on teacher moves matters, we need to be more creative about catalyzing the low-hanging fruit. That would mean identifying practitioners who are unusually interested in randomized research, and connecting them with doctoral students who are unusually interested in teachers and teaching.

What does it look like when practitioners and researchers dance together? Here is one example.

In July 2010, I asked Harvard economist Roland Fryer for some help. My research question was fairly simple: Do teacher phone calls to parents “work”?

In our school, teachers proactively phone parents. Typically, the parents have not been heavily involved in their children’s previous schools. We believe that phone calls to parents help teachers generate improved decorum, effort, and ultimately learning from students. (Sometimes the calls to parents are supplemented with teacher calls to students) These parent relationships seem to be linked to very high parent-satisfaction ratings, and in turn we have thought those were related to our high test-score growth. Truth be told, however, we just don’t know whether this is a productive use of teachers’ time.

Fryer enlisted two doctoral students, Shaun Dougherty and Matt Kraft, from the Quantitative Policy Analysis in Education program at the Harvard Graduate School of Education. These two did an amazing job, operating skillfully within our school to do the randomized study. From their findings:

“On average, teacher-family communication increased homework completion rates by 6 percentage points and decreased instances in which teachers had to redirect students’ attention to the task at hand by 32%.”

This collaboration worked for several reasons. First, we have a teacher residency embedded in our charter school, so I had 24 student teachers who could be fairly easily randomized during the summer school session. Second, a professor I trusted chose the graduate students who would conduct the research. These guys were, in my view, dispassionate. I’ve tried to work before with grad students who have strong preexisting beliefs about what they’ll find (typically with a “progressive” lens), and it was difficult to gain real knowledge. (Researchers often feel the same way about practitioners, that we’re searching for marketing, not truth). Also, Fryer paid them a stipend; in my experience, graduate students working for free, and only for credit of some sort, don’t always follow through.

The cost of the two graduate students was not the only expense. In our experiment, at any given time, there were 16 classrooms in action. The researchers needed to hire 16 observers to carefully code student behavior for a few weeks. The total bill was around $10,000. Kraft and Dougherty found a Harvard grant of $1,000. The rest I needed to pay.

Once we’d designed the experiment, I needed to explain it to my team: the principals of our high school and middle school, and the student teachers who were involved. These are people I know well, and they generally trust me. Still, this buy-in phase required expending both time and “relationship capital,” a resource that gets spent down and must be built back up over time. Using student teachers was also of benefit. It would have been tough to randomize our regular teachers. Their belief in the efficacy of parent communication is so strong I suspect many would have doubted the value of changing their normal routines.

There were other costs to the experiment. The head of our teacher-prep program spent many hours handling the experiment’s complex logistics, including a permission slip for parent consent. He could have spent those hours coaching these student teachers, which is the main task I was paying him to do.

All of these issues reflect transaction costs: finding the right people and then doing the right study well takes time, effort, and money.

Researching Teacher Moves

Think of the Human Genome Project. When the project started, scientists didn’t know how many genes there were; now they believe the number is 20,000 to 25,000.

We don’t know how many teacher moves there are. The number is certainly high but not infinite, maybe 200, 2,000, nobody knows. Presumably, there are some unusually high-yield teacher moves across all contexts, some moves that are high yield but only in specific situations or contexts, and other less powerful moves. There is undoubtedly lots of interaction effect among many moves. Mapping all of this might be called the Teaching Move Genome Project, and at the beginning it would be a scary undertaking.

Absent this work, what do we have? Perceived best practices, often buttressed by observation or nonrandomized studies. In his best-selling book Teach Like a Champion, Doug Lemov describes 49 teaching moves he has observed in the nation’s top charter schools. At the University of Michigan, Deborah Ball and her colleagues are close to unveiling a list of 88 math teacher moves. Lee Canter’s Assertive Discipline and Jon Saphier’s Skillful Teacher discuss scores of moves, like the “10-2” rule (have kids summarize for 2 minutes in small groups after 10 minutes of teacher-led instruction), much of it supported by nonrandomized research. On the basis of its observations of effective teachers, Teach For America (TFA) promotes 6 teacher behaviors and 28 component parts, like “plan purposefully” or “set big goals”; none are specific moves.

What would a series of randomized trials look like? Let’s apply it to Lemov’s 49. Imagine a group of trials that would ask the questions, Do all of the moves work? Are any particularly successful? How does the degree of teacher buy-in interact with effectiveness? What are the “costs” of these moves?

An example from Lemov is “Right Is Right.” The idea is that when a kid gives an answer that is mostly right, the teacher should hold out until it’s 100 percent correct. Lemov describes various tactics the teacher can use to elicit the 100 percent right answer from the student (or first from another student, before having the original student repeat or extend the correct answer).

The obvious cost of implementing this move is time. These back-and-forths add up to lost minutes each period when other topics are not being discussed. A less skillful teacher might be drawn into a protracted discussion, when her next best alternative (simply announce the 100 percent right answer, and move on) might work better. We just don’t know.

Back in 2003, education researchers David Cohen, Stephen Raudenbush, and Deborah Ball argued that “one could make accurate causal inference about instructional effects only by reconceiving and then redesigning instruction as a regime, or system, and comparing it with different systems.” That suggests “a narrower role for survey research than has recently been the case in education, and a larger role for experimental and quasi-experimental research. But if such studies offer a better grip on causality, they are more difficult to design, instrument, and carry out, and more costly.”

Still, we need a better grip on causality. So who would undertake this cost?

A Proposal

Once again borrowing some terminology from medicine, I propose a typology of trials, delineating phases in a continuum.

Phase 1 trials would be small, nongeneralizable empirical studies of teacher moves. These could be randomized, single-subject, or regression discontinuity, but the dependent variable would not be year-end test scores. Instead, we’d look for next-day or next-week outcomes: measurable effects on student behavior, effort, or short-term learning.

Who would decide what moves to test? Some would be proposed by established authors and thinkers in the teaching field. Some would come from the nation’s 3 million schoolteachers, possibly with crowd sourcing to identify the most-promising ideas. Some would come from academic researchers, particularly those from other fields, like psychology, who may offer unusual insights. But for the next level, testing competing ideas, I’d suggest we draw heavily on teacher opinion, particularly a group of teachers selected for their stated willingness to try new methods (if they are supported by research).

Phase 2 trials would test promising teacher practice from Phase 1 on a larger, more varied teacher pool to see if the next-day outcomes held up, probably across different types of schools. Again, the dependent variable is short-term student response.

Phase 3 trials would be randomized trials in which teachers combine multiple moves that emerge from Phase 2. In the end, our bottom line is student learning, and Phase 3 trials are combinations of moves that are measured to see if they bolster year-end student learning gains.

Medical researchers have found that treating some illnesses requires a drug “cocktail,” that is, no one medicine by itself works as well as the combination of several. The same approach might work in education: it could be that individual teacher moves by themselves cannot create measurable year-end achievement gains in students, but combining many together can.

My proposal is that each of the nation’s 1,200-plus schools of education and teacher prep programs conduct one randomized trial on a teacher move each year: Phase 1, Phase 2, or Phase 3. They’d do that by recruiting alumni into a network of experienced teachers willing to participate. The advantage is that once you pay the one-time transaction costs of finding these teachers, the ongoing expenditures related to persuading them to participate, and securing permission from families and principals, decline.

Once that network existed, it would function like a laboratory. Various Phase 1 experiments could be run through it, with small numbers of teachers at first, so that many experiments could be run concurrently. Larger numbers of teachers would be included in more promising Phase 2 validation experiments. Of course, there would be selection bias in terms of which teachers are willing to be participate in this sort of work, and other imperfections. But in the end, experiments could build on proven results from previous ones. Multiple ed schools would combine their networks for Phase 3 trials.

By itself, no single experiment would be that important. Instead, it would be like cancer research: thousands of people each trying to answer small questions in a very rigorous way…which would add up to promising treatments.

The goal is an affordable system for conducting teacher research that teachers would actually consume, that would address both the implementation challenges and the high transaction costs for researchers and practitioners in creating such research. Until that exists, I’ll see you at the 5th-grade dance.

Michael Goldstein is the founder of MATCH Charter School and MATCH Teacher Residency, in Boston.

On student/teacher engagement:

The part that is often missed when discussing these concepts is that it’s a strategy for learning that humanizes the classroom. Building and growing teacher-student relationships is essential to improving student learning outcomes. When a teacher has the opportunity to speak to each student and assess their progress every day, students feel that learning matters. They feel challenged and supported. Again, it’s not about the technology or the devices, it about shifting our pedagogy to put each student first.

I use this tool for all the reasons stated and more, particularly the opportunity to spend more class time addressing the higher order thinking skills. Could it be that this is the point that critics are missing? The term ‘ flipped classroom’ places too much emphasis on a tool used by students to prepare for class and clouds the fact that teachers are developing fuller, richer learning cycles with their new time. Let’s call it the ‘flip-tool’ and start to write more about the consequences that is the rich learning cycles we have been able to develop for our classrooms.

On technology and ensuring equal access:

I have vids on flash drives and DVDs for kids w/ no internet access/digital tools.

[Teachers] don’t need the internet to do this! They can create videos, and save them on students’ machines quickly and easily. That way students just watch them from the computer without having to worry about connecting to the internet. We’ve also been able to repurpose old laptops for just this use. Since all the computers need to do is play a few videos, old laptops are perfect for this task.

On managing time and motivating students to do at-home activities:

The flipped class allows for learners to customize when and where they learn. I have plenty that spend class time learning and practicing, and at home, they don’t worry about chemistry. There is no such thing as “homework” anymore. If they work, they can use class time to front-load and account for their work life. We went through how to budget time and use the resources so they don’t overwhelm themselves through the year…. Most of what my kids (and many flipped kids) do is use the videos as A) remediation if they need it, or B) pre-learning for use in class the next day. The videos I put out are less than 10-minutes in length, so the time at home is considerably LESS than a “normal” homework assignment. Plus, they aren’t sitting at home struggling with a worksheet or book assignment, so their mental stress is also alleviated to a degree with a flip.

one of the most surprising things I learned when a colleague and I went to using videos to deliver most of the “content” of our class was that when forced to boil down the content to the most important concepts in order to create the videos, we ended up with a total of 8 videos of about 10-15 minutes each for our 10 week course in microbiology. In the past, we wold have spent FAR more time delivering the same content in class. Now, class time is spent exploring the content in context, the students are in the lab more often and the class time is a far more collaborative endeavor for the students. We have been able to do more higher-order thinking projects with the “found” time. Also, the students really like being able to control the pace of the delivery of the content in the videos. We provide them with sheets to take notes on while watching the videos so it is not simply a passive activity. Flipping has definitely resulted in more engaging and enjoyable class time for the students and the teachers.

Finally, as chemistry teacher Aaron Sams explains, it’s important to emphasize that there’s no single model, with most teachers figuring this out, adapting, and improving their practice as they go.

Is there a bolder alternative that might actually induce our nation to achieve widespread learning excellence?

Here’s a simple idea: put excellent teachers, the top 20 to 25 percent who achieve well over today’s “year of learning progress,” in charge of every child’s learning—consistently. Even with solid teachers who achieve a full year of progress, students who enter school behind stay behind, and those in the middle do not leap ahead. Moreover, the current teacher pool feeds the anemic principal pipeline, meaning excellent teachers are often pulled from instruction—or forced to work under inadequate leaders.

In our recent report, Seizing Opportunity at the Top, we suggest three major ways to generate the significant will needed to put excellent teachers in charge of every child’s learning. We must, at the federal or state level:

• Limit who can teach to top high school graduates, with further screening for behavioral competencies of excellent teachers;
• Offer large financial incentives from regular funding streams, for districts, schools, and teachers when they produce high-growth learning, up to and far beyond standards; and/or
• Create a new civil right to excellent teachers, one that parents and students can enforce legally when a child is behind standards, not making a full year of progress annually, or has not had an excellent teacher in a subject for two years running.

The only way to implement any of these reforms successfully, within budget and at scale, is to help excellent teachers increase their productivity: swap portions of excellent teachers’ time with digital instruction so they can teach more classes with similar or even smaller group sizes; let them delegate nonessential tasks to other adults; use digital tools to save time on instructional monitoring and planning; put them in charge of other teachers; and let the willing have more students to nurture under their strong wings. Find more discussion of these options in Opportunity at the Top and 3X for All. This is not new: Other excellent professionals, whose jobs and pay aren’t frozen into molds, started making these changes for themselves a half-century ago, developing differentiated teams and using technology to save time and eliminate error from routine work, leaving the best free to do the most complex tasks.

Public Impact, with help from teachers and others, will soon begin releasing designs that clarify how to make these changes in schools, within budget, and pay excellent teachers more for the additional children they reach. “How to” models will help, but without major policy changes to induce the will, all evidence is that schools simply won’t budge—not even the ones that already can (e.g., charter schools).

ESEA could help. At a minimum, it could:

• Require states to identify excellent teachers immediately (even if full-blown evaluation systems take longer to develop);
• Require reporting of the percentage of students reached by teachers at each effectiveness level, not just the percentage of teachers at different effectiveness levels—rewarding places that put excellent teachers in charge of more students, directly or through managed teams; and
• Make federal funding contingent on clearing barriers that keep excellent teachers from reaching more students, such as limits on their pay, class sizes, and non-teaching staff who could monitor digital instruction.

Seizing Opportunity at the Top lists the basic policies that states must change. Absent will-inducing provisions, though, even ESEA and basic state policy changes combined will be inadequate.

Policy and political leaders at all levels: We must stop haggling over how to pack our saddlebags while other nations board helicopters. Our nation needs us to step up. We need major action to induce the will to put excellent teachers in charge of every child’s learning.

- Emily Ayscue Hassel and Bryan C. Hassel, Public Impact

My classmates complained that her class was too hard.

One of my other instructors spent class sessions badly summarizing the readings, instigating awkward and often one-sided class discussions, or trying to explain the homework assignments and projects she thought up. When she assigned one of her own articles for us to read, it became clear that despite having completed a doctorate at our university, she could not write a coherent academic article.

Desperate for a more challenging academic experience, I increased my course load for the second semester and handpicked my instructors. I actually enjoyed most of my classes that semester, but it was at this point that I began to deeply question the university’s approach to preparing future teachers.

It baffled me, for example, that I could get a master’s degree in teaching English to speakers of other languages (TESOL) after having completed only one rudimentary course in linguistics and one in English grammar. Almost all of my classmates struggled greatly in these two courses, leading me to wonder whether perhaps the admission requirements might also need refining. A class in adolescent development was useful, but the program offered no course in child development, despite the fact that my certification would be for grades K–12. It seemed that they were skimming over the important topics while bogging me down with courses in “theory and practice,” which did little to make me feel prepared to begin teaching on my own.

The focus of the third and fourth semesters was student teaching. My first placement was in high-school foreign language, for which I was also receiving certification. I was fortunate to work with a relatively strong supervising teacher; the infuriating aspect of this first placement was how I was evaluated. A supervisor from the university observed me during three lessons over the course of the semester. After each observation, she completed a write-up and made a few minimally helpful suggestions. During the final observation, she leaned over to my supervising teacher and casually asked, “So, what grade would you give her?” No criteria for evaluation, no request for a report on what I needed to work on. Fortunately, I did receive some valuable feedback from my supervising teacher that semester; I cannot say the same about my English as a Second Language student-teaching placement the following semester.

The final task I was asked to complete for the program was an “individualized project,” which sounded to me like a dumbed-down version of a thesis or capstone project. I have to confess that I took the easy way out. I knew I wasn’t going to get the kind of academic support I would need to complete an actual thesis, so I settled for designing a unit based on what I was already working on with my ESL students. After meeting with the professor a few times and receiving some vague suggestions, I handed in a project that earned me the last of a full transcript of easy As, with a friendly note on the cover and not a single comment or suggestion for how the unit could have been improved.

After observing and teaching in a variety of classroom settings over the course of my graduate studies, I have concluded that good teaching depends on three things: mastery of the subject, a keen understanding of how children learn, and an ability to maintain a disciplined yet positive learning environment. It is hard for me to express how disheartening it is to have spent two years and more than $80,000 in student loans on a program that did justice to none of those objectives.

The author earned a masters degree in education at a private university in the Northeast. Julia Harvey is a pseudonym.

The American Public School Teacher is not for the faint of heart.  Rather, it is a comprehensive report on the state of the teaching profession in the United States based on a 5-year study by the National Education Association.  It is steeped in useful data, rational commentary and thoughtful analysis.  Of course, like John Merrow, I found some of the reportage more illuminating than others.  But the authors included in this compendium are all distinguished in their fields, and not one can be dismissed as an educational lightweight.  As an historian, I was particularly pleased to see how the editor saw fit to include an historical review of the period since the 1950s, mentioning, among other things, Arthur Bestor’s great book, Educational Wastelands: The Retreat from Learning in Our Public Schools, the influence of Sputnik, and (a little later) the Great Society legislation, to underscore the national commitment to education for everyone.

What the book does not do, however, is to introduce many new approaches to the challenges the profession faces.  Of course, as with so many other arenas, America finds itself hard put to keep up with some other countries’ success at significantly raising the levels of academic achievement on the part of the high school graduate.  Indeed, the high school graduation rate, we are reminded, remains stuck at about 75%.  Despite the influence of the unions (the NEA should be commended for the thoroughness of the research on which the book is based), teacher remuneration continues to lag behind other more prestigious professions.  While student-teacher ratios are strikingly reduced from past years, the hoped-for improvement in academic achievement associated with more intimate circumstances has not occurred.  The majority of teachers are still given to teaching behind closed doors. Only the few dare to work cooperatively with their peers on any kind of systematic basis. The unintended consequences associated with No Child Left Behind may be ameliorated in the near future, but rational student and teacher evaluation appear to be as elusive as ever.  The standards movement, which appeared to be so promising in its inception in the 1980s, seems to have lost its impetus, resulting in too much attention to English, mathematics and science, and too little to the other subjects in the basic curriculum, a point stressed by two prominent education theorists, Diane Ravitch and Linda Darling-Hammond.  Finally, the concern about outcomes translates into a surfeit of standardized tests, many of which are rather remotely connected with what has been taught.

All these issues and many others are reviewed in the book, which makes the volume an extremely valuable primer when it comes to discussing the status quo.  However, in terms of the future, perhaps inevitably notions are more speculative.  True, the book heralds the recent alliance between NCATE and TEAC.  True, it suggests that there is a growing consensus in the field at large about the need for a radical reappraisal of what should constitute teacher education and teach licensing, Katherine Neville’s comments notwithstanding.  True, it argues that more and more teachers are losing their technophobia.

All in all, this book should serve as a splendid resource for the foreseeable future.  There is little of consequence that is not reviewed.  In other words, this is not bedtime reading.  It is a serious book of reference dealing with an extraordinarily important aspect of school reform.

-A. Graham Down

They’re wrong—fortunately. The new Phi Delta Kappan/Gallup survey makes clear that most adults value their children’s teachers. Seventy-one percent say they “have trust and confidence in the men and women who are teaching children in the public schools” and 67 percent say they would like to have one of their own children become a public-school teacher.

That’s tons more positive than the public’s view of schools in general: Just 17 percent give A or B grades to them (though Americans continue to give high marks to their own children’s schools—and this figure, say the pollsters, is rising).

Respondents were also asked to grade the teachers, principals, and school board in their own community. Here again, teachers fared best: Sixty-nine percent of respondents would award their town’s teachers either an A or a B versus 54 percent for principals, and a meager 37 percent for the school board. (This widening recognition of the governance failings of our public-school systems is, in its way, heartening.) Parents—interestingly—rank the worst: A discouraging 36 percent of respondents would give their communities’ parents top marks for “bringing up their children.”

So whence cometh the perceived public ire?  PDK and Gallup lift the lid a bit: Forty-seven percent of survey respondents feel that unionization (of teachers) has hurt “the quality of public education in the United States” compared with 26 percent who say it has helped. (Are you paying attention, Randi and Dennis? Your organizations don’t have a lot of fans. Even school boards fare better!)

Some aspects of school teaching seem permanent, even eternal, but in many ways teaching today has changed from my own student days and it’s likely to be even more different tomorrow.

In the last half-century, unionization has flooded the schools (and is now slowly starting to ebb—or be pushed back). Possibly more important, though, has been the sheer growth in the number of public-school teachers. In the 1950s, the crude ratio of students to teachers across American K-12 education was 27:1. Today it’s 14:1. That doesn’t mean everybody’s classes are smaller but it does mean that we now employ an enormous number of teachers—in the ballpark of 3.5 million—and essentially all the extra money we’ve put into public education has gone to pay for their salaries and benefits. That’s why teacher pay has simply kept pace with the cost of living and why these levels of compensation in much of the U.S. today aren’t sufficient to attract and keep a great many of our ablest college graduates. (Mercifully, they attract and keep some!) If today’s ratio were still 27:1, today’s school budgets would be sufficient to pay an average teacher salary north of $100,000.

As for what will be different in the teachers’ world tomorrow, five developments need to be noted and taken seriously.

First, technology is going to have a major impact, both on what happens within traditional schools and classrooms and, more broadly, on what we mean by “school” and where and when learning occurs. Most likely, it will mean that we need fewer flesh-and-blood teachers sitting in the classroom with Johnnie and Susie—though we may need more aides and tutors and such to provide face-to-face explanations, pats on the back, and (when needed) stern looks and reminders to remain on task. (Expect a paper soon from our “Creating Sound Policy for Digital Learning” series on the specifics of these shifts.)

Second, school budgets are going to be flat (or falling) for the foreseeable future—and looming deficits in retirement and pension funds almost certainly mean that the take-home pay of practicing  teachers will see no real-dollar growth and could well decline. (The only rational antidote to that is, in fact, employing fewer individuals and paying them better.)

Third, there’s a revolution underway in teacher evaluation and many of the HR practices associated with it, including retention, tenure, compensation, promotions, and layoffs. It’s rocky, to be sure, but we’re gradually coming to gauge teachers more by what their students learn and less by the credentials that they carry. (And this isn’t just a cause trumpeted by wonks and reform junkies. Per yesterday’s poll, 74 percent of adult Americans say that it’s important to incorporate student test-score data into teacher evaluations.)

Fourth, big changes are brewing in teacher preparation and licensure as ed schools come under fire, as “alternate routes” proliferate, as programs like Teach For America get greater traction, and as more attention is paid to what a teacher knows about her subject than to what pedagogy courses she took in college.

Fifth, though the system hasn’t quite made this adjustment yet, we’re seeing that a non-trivial fraction of teachers are people who want to do this work for a time, before or after they do something else, rather than make a lifelong career of it. We’ll likely evolve a set of arrangements that capitalizes on the short-termers as well as the classroom careerists.

As we contemplate this future, it will surely help if teachers themselves, with or (more likely) without their unions’ help, prove willing to experiment, to grow, to listen, and to learn. And it will help if all the rest of us—even the curmudgeonly crew at Fordham—pause to thank today’s hardworking educators for selfless, challenging, and not very well compensated work on which our kids’ future and our country’s prospects depend so heavily.

-Chester E. Finn, Jr.

Teacher benefits, once a sleepy question primarily of interest to actuaries, have become a flash point in the education debate. With individual states on the hook for tens or hundreds of millions in unfunded pension and health insurance obligations, state leaders are trying to determine the severity of the situation and the appropriate response. In this forum, Robert Costrell of the University of Arkansas and Mike Podgursky of the University of Missouri argue that the situation is critical, but offer an opportunity for overdue reform, while Christian Weller of the University of Massachusetts-Boston argues that measured steps will put teacher pensions on sound footing.

Education Next: How bad is the teacher pension crisis?

Christian Weller: The states’ fiscal crisis necessitates that they address pension underfunding. Underfunding means that pension assets are lower than liabilities, or those benefits promised to beneficiaries. The underfunding often seems staggering. The Center for Retirement Research at Boston College, for instance, estimated the gap at more than $700 billion in 2009. The aggregate underfunding reflects the money that states will need to come up with over several decades. But the CRR also estimates that an additional 2 percent of payroll would cover the expected shortfall, making the problem manageable without ruining governments.

Christian Weller

States can take a balanced approach to managing pension underfunding that fits their particular circumstances. Thirty-nine states reduced benefits, increased contributions, or both between 2001 and 2009, according to the Pew Center on the States. The exact combination of benefit and tax changes depends on several factors, including public employees’ Social Security coverage, current benefits and contributions, and states’ human resource needs. States still want to make sure that their benefits allow them to hire the most-effective employees.

Robert Costrell and Michael Podgursky:

Indeed, educator pension systems are becoming increasingly expensive and, in many states, are seriously underfunded (see “Teacher Retirement Benefits,” research, Spring 2009). One major source of this problem is the massive increase in benefits from several decades of legislative enhancements. The key to understanding this is the concept of “pension wealth,” the current dollar value of the expected stream of future benefits, in other words, the cash value of a retiree’s annuity. Pension wealth encompasses both the annual pension payment and, importantly, the number of years it is collected.

The two solid curves in Figure 1 show pension wealth for a typical Missouri teacher in 1975 and today. Each curve is calculated under the current salary schedule for teachers in the state capital, so the growth represents only pension rule changes. The bottom curve shows that under 1975 rules a teacher entering at age 25 would have accrued just under $400,000 in pension wealth by age 55. Today, the same teacher would have accrued pension wealth of just under $900,000 by the same age. Not surprisingly, these enhancements have come at a substantial cost: Combined contributions for teachers and districts increased from 16 to 29 percent of salary over this period. However, even this is inadequate; the portion of salary required to pay for pension wealth accruals of current teachers and to pay off the unfunded liability is 31.3 percent.

EN: What steps should states take to address the crisis?

RC & MP: Given concerns about cost and long-term sustainability, a number of states have cut benefits, usually for new teachers, and others are considering doing so. However, in making these changes, policymakers should carefully consider their labor market effects. Some of the proposed cuts reproduce, and even exacerbate, undesirable features of current systems. These shortcomings stem from a fundamental flaw: the failure to tie benefits to contributions. Thus the fix must expose and eliminate the gaps between the two. Below are three recommendations for reforming teacher pensions:

Robert Costrell

1. Report the gaps between contributions and pension wealth. In many respects, current defined benefit (DB) pension plans for teachers are opaque. Teachers rarely know what their plan is worth. By contrast, holders of 403(b) or 401(k) accounts typically know exactly what their account is worth at any point in time. To provide the same transparency for teachers, plans should not only disclose the projected annual pension payment, they should also report pension wealth. For comparison, the plan should disclose the cumulative value of contributions, both the employee’s and the employer’s, along with accumulated returns. In this way, each educator could see how the value of her accrued benefits compares with the value of the contributions. In the typical teacher pension plan, these are going to be very different numbers. Early in a teacher’s career, the value of the contributions will far exceed pension wealth, whereas for more senior teachers, the reverse is true. The dotted line in Figure 1 illustrates this point. It represents the cumulative value of contributions that is fiscally equivalent to the current pension plan, showing that the cumulative value of pension contributions exceeds pension wealth until age 50. However, between ages 50 and 62 pension wealth is typically well in excess of contributions. Not surprisingly, this is when the vast majority of full-career teachers choose to retire.

Michael Podgursky

2. Close the gaps between contributions and pension wealth. To make pensions more equitable and effective tools for staffing schools, we propose that retirement benefits paid to any teacher should be tied to the lifetime contributions made by or for that teacher. If $300,000 has been contributed on behalf of a teacher (including accumulated returns), then the cash value of an annuity provided to this teacher should also be $300,000.

Unfortunately, as we have seen, this fundamental principle is routinely violated in teacher plans. The gap (positive or negative) between the value of benefits and contributions is rarely considered in plan design. Instead, legislatures tinker with complex and arbitrary pension rules, such as the calculation of final average salary (how many years included, what counts as “salary”), the annual service “multiplier,” and the eligibility rules to receive the pension (“rule of 80,” “25-and-out,” etc.). Since these benefit rules are not tied to contributions, legislatures have, over the years, enhanced them, without regard to equity or efficiency, and often without adequate funding. These complex rules also encourage “gaming” by educators and districts in order to increase the gap between benefits and contributions.

Our analysis shows that current systems typically result in very large implicit transfers from young teachers working short spells to “long termers,” who work full careers in the same system (see “Golden Handcuffs,” research, Winter 2010). In our view, a teacher who works 10 years or 30 years should accrue pension wealth roughly equivalent to total pension contributions (with accumulated returns). Thus, in Figure 1, the pension wealth curve would coincide with the contributions curve depicted, for a fiscally equivalent plan, or with a lower curve if costs are to be reduced.

In addition to improving equity, tying benefits to contributions would have important workforce benefits. First, it would provide rational incentives for retirement versus continued work. Each year, an educator would accrue pension wealth in a smooth and transparent way, providing a steady addition to the annual salary she is earning. This would generate neutral incentives to work or retire based on individual preferences and effectiveness. That is not the case with current systems. In our own work, we have shown sharp “peaks and valleys” in pension wealth accrual, which distort incentives for retirement (see “Peaks, Cliffs, and Valleys,” features, Winter 2008). Some years (e.g., at 25 or 30 years of service) yield increases in pension wealth that are several times the teacher’s salary. This provides a huge incentive to stay on the job until that pension “spike,” regardless of classroom effectiveness. There is no economic rationale for favoring one year of work over another in this way. Nor should an additional year of work reduce pension wealth (net of employee contributions), as is the case in current teacher plans after a certain point, often at relatively young ages. This penalizes good teachers who wish to stay.

Tying benefits to contributions would also eliminate the massive penalties for mobility in current systems. It is well understood in the private sector that in order to recruit and retain talented young employees it is necessary to provide portable retirement benefits. This is accomplished by defined contribution (DC) or cash balance (CB) plans that vest immediately or nearly so. Current teacher plans typically have 5- and even 10-year vesting. Our research finds that even for vested educators, the loss in pension wealth for those who split a teaching career between two traditional plans is massive. In a system where benefits are tied to the cumulative value of contributions, it does not matter whether contributions have all been made in one or many jobs: Penalties for mobility are eliminated.

3. There is more than one way to do it right—and to do it wrong. We favor CB plans. These are a form of defined benefit plans that generate individual retirement accounts in bookkeeping form within the pension fund. They are funded by contributions from employer and employee just like most current teacher plans and carry an investment return guaranteed by the employer. Such plans resemble a DC plan, but without transferring investment risk or asset management to the teacher. They are transparent, offer smooth wealth accrual, and are readily turned into annuities at retirement, like traditional teacher plans. However, no one year of retirement is favored over any other. Large private employers such as IBM have converted to such plans, as have a few public employers. The TIAA guaranteed-return plans that are common in higher education are similar in operation. They have provided retirement security for generations of college professors, who often spread careers over multiple institutions.

By contrast, Illinois is a cautionary example of how not to reform teacher pensions. Illinois recently implemented a two-tiered plan, placing teachers hired after January 1, 2011, in the second tier. Tier 2 teachers make identical contributions (9.4 percent) as their Tier 1 colleagues, but take a drastic cut in pension wealth accrual over their work lives, as shown in Figure 2. The Tier 2 plan retains the same basic structure while raising the retirement age. This exacerbates the back-loading and mobility penalties, and widens the gaps between benefits and contributions. A new teacher entering the Illinois plan at age 25 will accrue no pension wealth, net of employee contributions, until age 51. This is not an attractive offer for young, mobile teachers. Indeed, the Tier 2 package is not actually a net “benefit” for entering teachers, since the teacher contributions are nearly double the cost of the average benefit they accrue; the rest is basically a tax to pay for benefits accrued but not funded, by previous cohorts of teachers.

As states grapple with the current pension crisis, a window of opportunity is open to implement more modern and strategic plans, or to make matters worse. Fundamental reforms are needed to fix these broken systems. Systems should first be required to report the gaps between benefits and contributions for all members. Then, as a matter of equity and efficiency, the plans should be restructured to close these gaps by tying benefits to contributions. This would give young teachers their fair share of the retirement benefit pie and rationalize the retirement incentives for all teachers.

CW: States can take a number of steps to alleviate the pension crisis:

1. Spread the pain of addressing underfunding, if adjustments are unavoidable. Changes to pension plans generally only apply to new hires. State constitutions and courts typically hold already-earned benefits and future not-yet-earned benefits for existing employees and beneficiaries inviolate. This protection is also occasionally applied to employee contributions. Governments cannot reduce benefits and raise contributions for current employees, even if they want to. Hence, adjustments fall disproportionately on new hires.

Private-sector pension benefits also enjoy substantial protections, but to a lesser degree than public-sector benefits. The Employee Retirement Income Security Act of 1974 protects from reductions benefits that have already been earned, but it does not protect future benefits not yet earned. Private-sector employers can thus lower future benefits when a crisis requires a drastic change.

States should change their public benefit protections to permit adjustments to be distributed across a broader range of employees, if such adjustments become necessary. States could guarantee already-earned benefits but not those not yet earned, as the private sector does. States could also ease older employees’ distress about potential benefit changes by allowing future benefit reductions only for employees under a certain age.

There are several advantages to this approach: Current beneficiaries would remain fully protected, already-earned benefits could not be taken away, and older employees would receive the retirement benefits that they had earned. Arbitrary divisions in younger employees’ compensation arising from whether they were hired before or after the benefit change went into effect would also be eliminated.

2. Prevent underfunding in the future. The current underfunding resulted from massive stock and real-estate market declines. Public pensions were prudently managed before the crisis, as Jeff Wenger and Christian Weller have demonstrated elsewhere.

But many governments did not contribute as much as necessary to their pension funds, making them vulnerable in a crisis. The problems of pensions are more a result of low employer contributions than poor pension management. Governments often avoided paying the full amount of what was necessary to cover benefits earned in a given year. Even in 2011, Governor Chris Christie (R-NJ) considers the state’s contributions to its pension plan an optional expense. Governments, as employers, have exacerbated, and continue to exacerbate, their pension plans’ financial challenges.

One solution is to make governments pay the necessary amount to their pension plans. States could set a floor under employer pension contributions. The employer contributions could never fall to zero, commonly known as “taking a contribution holiday,” and employer contributions could never fall below the “floor” rate. DB pensions would receive money more regularly than is currently the case and thus underfunding would become less likely, particularly during a crisis.

If they set a floor for employer pension contributions, states would simultaneously have to change the rules that govern pension funding. Strong financial market performance could easily translate into overfunded pensions, which is desirable, since it means that DB pensions are prepared for a rainy day, such as the recent crisis. But overfunded plans could feed appetites for benefit improvements or contribution cuts, unless the law states that better benefits and lower contributions could only be considered if a DB pension has a minimum buffer for emergencies. Weller and Baker (2005) suggest a buffer of 20 percent of liabilities, which could be even smaller for state DB pension plans, since states cannot go bankrupt.

3. Beware of unintended consequences with alternative benefits. The wrong changes could have serious adverse effects. This would be the case if states also changed their retirement plans from DB pensions to an alternative design, particularly defined contribution (DC) savings accounts such as 403(b) plans, but also a cash balance plan. Cash balance plans look like DC plans to employees but operate like a DB pension for employers. Employers offer a guaranteed rate of return on current and past contributions to a cash balance plan and take the risk of higher contributions if the actual rate of return falls below the promised one.

Alternative benefits are less efficient than DB pensions. First, the average teacher effectiveness will likely decrease, as much higher employee turnover will easily offset any potential effectiveness gains. Second, alternative benefits come with substantial costs.

One unproven assertion about alternative benefits is that they would result in greater teacher effectiveness. Alternative retirement benefits are attractive to their proponents because these benefits would offer more compensation earlier in a teacher’s career and promote turnover later in a teacher’s career relative to a DB pension. Higher compensation earlier would attract to the profession people who could potentially become more-effective teachers, while fewer financial incentives to stay would supposedly lead ineffective teachers to leave earlier than they otherwise would.

The literature on teacher effectiveness and employee turnover associated with benefits shows that average teacher effectiveness will likely decline with alternative benefits. Higher early compensation will offer only a small incentive for promising though untested teachers to enter the profession. And the link between teacher pay and student achievement has been shown to be tentative at best. Since a benefit change would only marginally increase beginning teachers’ compensation, any initial bump in overall instructional effectiveness would be both fleeting and faint, if it exists at all. Any small initial improvement in teacher effectiveness will be quickly offset by higher turnover among more-experienced teachers. Experienced teachers who leave will be replaced by inexperienced teachers, who will need time to build their classroom skills. Small turnover increases can quickly offset small productivity gains to ultimately lower average teacher quality. The literature, in fact, shows that we can expect substantial increases in turnover with a switch to DC and cash balance plans from DB pensions so that higher turnover will eliminate any possible gains from higher initial compensation. We estimate, for instance, that the chance of worsening teacher effectiveness is about 60 percent with a cash balance plan and 70 percent with a DC plan under optimistic assumptions that favor alternative benefit designs based on the existing long-standing literature on pensions and turnover and the much smaller literature on initial compensation and teacher effectiveness.

Teacher turnover can be expected to increase with alternative benefits because employees will understand that their economic security is less well protected with a DC or cash balance plan than with a DB pension. National opinion polls routinely find very strong support for DB pensions, as individuals who do not like risk prefer to have some income guarantees for themselves and their families when they retire, become disabled, or pass away. Fewer income guarantees, or insurance, lead people to leave employment more quickly than they otherwise would. Thus, under these circumstances, teacher turnover would increase and average teacher effectiveness would fall.

Private-sector employers without DB pensions often use other tools to mirror the human resource effects, i.e., long tenure of skilled workers, of DB pensions, exactly because they are worried about turnover. Employers in the field of information technology, especially, offer, for instance, stock options and stock grants to recruit and retain skilled workers for long periods of time. States simply cannot offer these benefits and hence have no way to lower turnover among effective employees.

Alternative benefits also cost more. First, DB pensions would have to operate with a finite investment horizon, increasingly moving money to secure, low-return assets so that lower investment earnings would lend less of a helping hand to pay for benefits. Second, employers may have to cover any underfunding more quickly for closed plans than for ongoing ones, raising employer contributions. Third, higher turnover increases cost due to more recruitment and training of new hires. Fourth, there are substantial transition costs. Older employees will continue to earn DB pensions, they will earn more benefits as they stay longer on the job, and there will be more long-term employees under the DB pension, raising the cost per employee of the DB pension. New employees, in comparison, would be more prevalent in the new plan, earn initially higher benefits than with a DB pension, and thus raise costs relative to a DB pension. These transition costs would last for about four decades and could average 1 percent of payroll for many years, even if the costs of retirement benefits are the same before and after the transition. Fifth, DC plans offer fewer insurance benefits than DB pensions. The insurance exists largely because employees who happen to live through a prolonged period of prosperity share some of their gains with less fortunate employees. Researchers at the National Institute on Retirement Security estimated in 2008 that the loss of insurance features meant that each dollar invested in a DC plan generated 46 percent less in retirement benefits than a dollar invested in a DB pension. Finally, there are higher administrative costs due to a large number of small accounts, especially in DC plans, and increased movement of money between retirement plans.

The two states that have switched from DB pensions to DC plans, West Virginia and Alaska, had severe cases of buyers’ remorse. West Virginia eventually switched back to a DB plan for their teachers in 2008, and Alaska’s policymakers have been investigating the possibility of making a similar reversal.

The lessons from the evidence are clear: States can manage the financial challenges of their pension plans. The proposal to use the current crisis as an opportunity to switch retirement plans, though, will leave states with a much less efficient compensation system. The average effectiveness of teachers will likely drop, and costs will go up substantially. States will be better off managing the financial problems of their DB pensions by putting mechanisms in place that will prevent future underfunding instead of engaging in costly retirement-plan experiments that offer no benefits.

EN: What, then, are the main areas of disagreement?

RC & MP: We disagree on structure. We argue that benefits should be tied to contributions. Professor Weller believes this would have adverse consequences.

Weller assumes a shift to CB or DC would raise annual exits at all ages by a hefty rate, between 22 and 220 percent, according to his recent but as yet unpublished paper on which his efficiency claim is based. Thus, the share of novice teachers in the workforce would rise and average effectiveness would fall. However, the 22 percent estimate is drawn from a 1993 paper by Allen, Clark, and McDermed that compares private-sector workers “covered by a company retirement plan” to those who were not covered by any plan, so there are no implications for CB or DC. The 220 percent assumption is drawn from a 1996 paper by Even and MacPherson that actually shows no difference in quit rates between DB and DC.

Economic theory suggests mixed effects of CB on teacher quit rates, raising them for mid-career teachers who would otherwise hang on for early retirement and lowering them for late-career teachers, otherwise driven out by negative accrual. It might also lower quit rates for young teachers, since they accrue more pension wealth under CB than under current plans. This mixed pattern is supported by Costrell and McGee’s findings, in their 2010 peer-reviewed econometric study of teacher response to pension wealth accrual. Their simulation of a shift to CB, based on their behavioral estimates, found a slight rise in average teacher tenure, not a large fall.

Turning to transition costs, Weller claims that new plans raise costs on old plans by forcing changes in investment strategy or amortization schedules. However, pension plans often introduce new “tiers” without these effects, as new and old funds are commingled. Introducing CB as a new tier would be no different.

Weller’s simulation of transition costs, also from his unpublished paper, makes a different argument. He claims costs will rise for decades because entering cohorts have a different time pattern of pension wealth accrual than previous cohorts. But the time pattern is irrelevant here. Each cohort’s cost is the present value of its lifetime accruals, however they are distributed. Costs cannot rise unless some cohort enjoys higher benefits and, hence, higher lifetime accruals of pension wealth. Yet Weller assumes each cohort accrues the same pension wealth—10.25 percent of the cohort’s lifetime payroll. That is the cohort’s “normal cost,” the contributions required to fund the cohort’s lifetime benefits and accruals. The system’s required contributions are a blend of each cohort’s normal costs, but these are the same, 10.25 percent for each cohort. Thus, the system’s contributions are unchanged, and there are no transition costs.

Costs and contributions would fall if benefits were cut, as Weller recommends. Indeed, they would fall more quickly under his reasonable proposal to cut normal costs of current teachers, as a matter of equity between generations. However, we also favor equity for mobile young teachers, who will continue to receive benefits worth far less than contributions, absent fundamental reform.

CW: The evidence shows that defined benefit pensions work for education. Professors Costrell and Podgursky do not address the fact that both employers and employees prefer defined benefit pensions over other retirement benefits.

The vast majority of states underwent pension reforms in the past decade to address the financial challenges of pension underfunding and none abandoned their defined benefit pensions. And private-sector employers in key growth industries, such as information technology and banking, offer either defined benefit pensions or other forms of deferred compensation, such as stock options, to their employees to mimic the retention benefits of pensions when pensions are absent. A substantial literature both develops the theory and shows the supporting evidence for the efficacy of deferred compensation as a retention and recruitment tool for skilled employees. There is a clear economic rationale for deferred compensation, since it allows employers to recoup the investments made in hiring and training skilled employees, such as teachers.

Teachers equally prefer pensions. Opinion polls routinely show a preference for defined benefit pensions, even among younger employees. And when teachers (and other public employees) have been given a choice between defined benefit pensions and defined contribution plans, the vast majority typically chooses the defined benefit pension plan. The evidence contradicts Professors Costrell and Podgursky’s key assertion that alternative plans that offer more immediate compensation are more attractive to younger teachers.

Finally, transition costs from a defined benefit pension to a cash balance plan would quickly drain public coffers. There would be a growing concentration of more-experienced teachers under the defined benefit pension that favors more-experienced teachers and a high concentration of inexperienced teachers under a cash balance plan that favors inexperienced teachers. A long-standing literature has regularly shown that DB pensions substantially reduce turnover compared to other retirement benefits, suggesting that a benefit switch will increase turnover.

The increase in turnover will raise costs and pose the threat of lower average effectiveness, as my own simulations for a switch from DB pensions to cash balance plans show. The costs are predictable and substantial, while any benefits are highly uncertain and likely nonexistent.

Which is why I can’t understand why the organization continues to be so wrong about one of the most consequential developments in education today: The National Council on Teacher Quality’s review of education schools nationwide.

First there was Chad Adelman (since promoted to the U.S. Department of Education), who complained that NCTQ’s study wasn’t focused enough on outcomes:

Absent some objective outcome measures, NCTQ will only be assessing inputs to teacher quality…. There will be no mechanism to determine if all of the box-checking that NCTQ will be assessing has actually produced effective teachers.

You don’t say! As Chad acknowledges, NCTQ has been at the forefront of the push for states to collect value-added data linking ed schools with their graduates’ results in the classroom. A handful of states are starting to do that. But what about the other 45+ states? Should NCTQ sit on its hands until the data become available? Isn’t Chad’s argument just one for giving the ed schools a pass?

Then, last week, Sarah Rosenberg asked whether “anyone at home” really cares about this “report card.”

NCTQ and its supporters believe that clear standards and transparent evaluation will encourage schools to improve their teacher preparation programs and, in turn, their ratings.  For that theory of change to work, a school’s rating must trigger market response: A school of education that receives a high rating should see more students apply as well as more districts interested in partnering with the school and hiring its graduates.  The extent to which NCTQ’s national ratings matter will depend on whether districts and prospective teachers make decisions based on the ratings. The local nature of teacher labor markets makes it unlikely that this will happen in many parts of the country—will anyone in Weldon, NC really care that their one nearby school of education was rated poorly?

Sarah goes on to acknowledge that the “market response” may work OK in urban areas with multiple ed schools—districts may eschew the lousy ones. But since most suburban and rural districts hire from nearby colleges, this market mechanism won’t mean a thing in much of the country.

Maybe. But NCTQ is after much more than just a market response. The study is often billed as the “Flexner Report” for education—referring to a study of medical schools 100 years ago that led to the shuttering of hundreds of them. Yes, I am sure NCTQ wants the “average” ed school to get better. But more than anything else it wants the abysmal ed schools to go out of business. And that will take action by the states—action that is a whole lot more likely if NCTQ calls them out publicly.

So Education Sector, how about a little less skepticism, and a little more love, for one of the gutsiest projects in education reform history?

-Mike Petrilli

If principals have the authority to dismiss teachers, will they dismiss the less effective ones, or will they instead make perverse decisions by letting the good teachers go? Evidence from low-stakes surveys suggests that principals are able to identify the most and least effective teachers in their schools, as measured by their impact on student achievement (see “When Principals Rate Teachers,” research, Spring 2006). But would that ability influence their dismissal decisions?

On this topic, debate has been vigorous but research almost nil, in good part because teachers with tenure are not easily dismissed and principals take on that task only if they have a strong backbone or face an extremely urgent situation, or both. In some instances, however, principals have considerable latitude when it comes to dismissing teachers who have not been in service long enough to have earned tenure.

One such situation developed in Chicago in July 2004 when the Chicago Public Schools (CPS) and the Chicago Teachers Union signed a new collective-bargaining agreement that gave principals the flexibility to dismiss probationary (nontenured) teachers beginning in the 2004–05 school year for any reason and without the documentation and hearing process that is typically required for dismissals in other districts. Since CPS provided information that allowed me to link information on CPS teacher dismissals to several measures of teacher performance, I was able to study whether principals exercise their authority wisely. The procedures were fairly straightforward. By comparing the characteristics of dismissed versus nondismissed probationary teachers within the same school and year, I was able to determine just how much weight school administrators place on a variety of teacher characteristics, including their performance in the classroom.

I find that principals in Chicago do exercise their authority in  sensible ways. Principals are more likely to dismiss teachers who are frequently absent and who have previously received poor evaluations. They dismiss elementary school teachers who are less effective in raising student achievement. Principals are also less likely to dismiss teachers who attended competitive undergraduate colleges. It is interesting to note that dismissed teachers who were subsequently hired by a different school are much more likely than other first-year teachers in their new school to be dismissed again.

These results suggest that other school districts could possibly improve student achievement if they adopted policies similar to those applied in Chicago. To be clear, however, the analysis presented in this paper does not seek to evaluate the educational impact of this new policy. Instead, it uses the existence of the policy, in conjunction with detailed data on teachers and principals, to provide descriptive evidence on the relationship between the exercise of dismissal authority and teacher effectiveness.

Teacher Dismissals in Chicago

As in many public school districts, teacher layoffs and dismissals in CPS are highly regulated. Prior to 2004, virtually no teachers—not even probationary teachers—were dismissed for cause in CPS. Of course, it is likely that some teachers who switched schools or left CPS entirely were informally “counseled out” by school administrators. But it was impossible to distinguish these “involuntary” separations from truly voluntary attrition.

This situation changed with the signing of a new collective-bargaining agreement in 2004. Each February, principals are able to log into a district computer system that has a list of all of the probationary teachers in their school (i.e., those who have been teaching for fewer than five consecutive years during the period of my analysis). The principal can then check one of two boxes: renew or nonrenew. Although principals are required to provide district officials with at least one reason for the nonrenewal decision, they are not required to justify or explain their decision and they do not need to provide teachers with this reason. If a principal chooses nonrenew, the teacher may reapply to positions in other Chicago public schools. However, nonrenewed teachers are not guaranteed another job in CPS. The ease with which administrators can dismiss a probationary teacher, with a simple “click” of a button, is noteworthy. This policy change made Chicago the only large school district in the country to provide principals with this degree of flexibility over personnel decisions. Already since the conclusion of the analysis period for this study (2005 through 2007), this flexibility has diminished in several ways. For example, the probationary period has been reduced from 4 to 3 years, and principals who choose to nonrenew a teacher now must have conducted at least one formal observation of the teacher prior to nonrenewal.

Data

The data for my study of this policy change come from several sources. Teacher personnel files provide information on teacher background, current assignment, and, for probationary teachers, whether or not they were renewed. I supplement this with information on school demographics, principal characteristics from personnel files, and student test-score information.

I examine dismissal among probationary teachers in CPS in three consecutive school years: 2004–05, 2005–06, and 2006–07. The sample excludes individuals who were employed by the central office, including speech pathologists, nurses, counselors, and teachers working in administrative or professional development capacities. Moreover, I exclude teachers in a handful of “alternative” schools that serve severely disabled students or other special populations, as well as teachers on leave or who were employed less than half time. For a small number of teachers who taught subjects such as art or music in multiple schools, I include only the observation in the school that is listed as their “primary” appointment. The final sample consists of 16,246 elementary school teachers and 7,764 high school teachers spread across 588 schools.

Measures of Teacher Quality

This analysis incorporates three proxies for teacher performance. First, I use teacher absences because they are well measured, are easy to interpret, and impose substantial nonfinancial and financial costs on the school. The second measure is the formal performance rating that the principal gave the teacher in prior years. Traditionally, principals rate teachers every one to three years (depending on the tenure status of the teacher) on a four-point scale that indicates superior, excellent, satisfactory, or unsatisfactory performance. While there are no high stakes associated with these ratings (virtually no teachers receive an unsatisfactory rating), there is considerable variation across teachers in the top rating categories, and they arguably provide a sense of how the principal views the teacher. The third measure is a value-added estimate of teacher effectiveness. This measure is meant to capture the extent to which each teacher contributes to student achievement growth from one year to the next, as measured by the standardized tests taken by students in CPS. While this is an objective and direct measure of one important dimension of teacher effectiveness, only a fraction of teachers work in grades and subjects in which students take standardized tests. It is not possible to calculate value-added measures for many teachers in our sample, including teachers in grade 2 or below, most teachers in grades 10 or above, and any teacher in a noncore subject. Unlike some school districts, Chicago traditionally has not maintained reliable data linking teachers to classrooms, particularly at the elementary level. Working with CPS officials, however, I was able to obtain such links for a limited sample of teachers and years, thus allowing me to create value-added measures for part of my sample.

Methods

The primary goal of my analysis is to determine which teacher, principal, and school characteristics are associated with the likelihood that a teacher will be dismissed. I first compare the probability that a teacher is dismissed across schools and years in order to discern any differences related to school characteristics. Then to examine the influence of teacher characteristics on the likelihood of dismissal, I compare teachers within the same year and school to account for unobserved school-level factors that might be correlated with teacher characteristics and the probability of dismissal.

A concern with this approach is that if the analysis fails to include a teacher characteristic that a) principals consider in the dismissal decision and b) is correlated with one of the included variables, the estimate for the included characteristic may be biased. One potentially important variant of this concern involves the supply of teachers. If it is more difficult to find qualified teachers in certain subjects or grade levels, then the principal may be less likely to dismiss teachers in these areas. To the extent that teachers in harder-to-staff areas are concentrated among particular demographic groups, or tend to graduate from particular institutions, the results for these teacher characteristics could be misleading. Also, schools fund teachers from a variety of revenue streams, and it may be difficult for principals to reallocate positions across funds. For this reason, if a school experiences a decline in a particular revenue fund, the principal may be more inclined to dismiss teachers funded by this source.

To address these concerns, I account in all analyses for the teacher’s program area (for example, regular education grades 1 to 3, regular education grades 4 to 8, secondary math, secondary science, bilingual education, vocational education, etc.) and for the revenue source from which each teacher position is funded.

Of course, it is still possible that my results concerning specific teacher characteristics suffer from a standard omitted variable bias. For example, it may be the case that high rates of absenteeism are associated with a bad attitude or shirking in other dimensions, and it is these factors, rather than the absences per se, that the principal is reacting to in dismissing teachers with more absences. In this case, one may not be able to say anything definitive about principal views regarding teacher absenteeism itself, but rather about behaviors and characteristics associated with absenteeism, all of which presumably speak to performance in some form or another.

Dismissal Policy Impact

Each year under the new policy, roughly 11 percent of probationary teachers were dismissed, despite the fact that more than one-third of schools did not dismiss any teachers. The numbers of teachers who were nonrenewed in any given year likely overstates the impact of the policy because a number of young teachers would likely have left CPS in the absence of the policy, either voluntarily or due to subtle “encouragement” on the part of the principals. If the dismissal policy merely formalized previously informal dismissals, however, then one would not necessarily expect to find a substantial change in separations.

Comparing dismissal rates before and after implementation of the new policy provides insight on this issue. In the three years prior to the introduction of the policy, roughly 10 to 15 percent of first-year probationary teachers left CPS and an additional 4 percent moved to a different CPS school. In the years after the policy was in place, the corresponding rates were roughly 18 and 10 percent, respectively. Comparing the year immediately prior to establishment of the policy (2004) with the first two years of the policy’s implementation (2005 and 2006), it appears that the separation rate increased by roughly 9 percentage points (see Figure 1). In contrast, there was virtually no change among more-experienced teachers (i.e., those with 6 to 15 years of experience), who were not subject to the policy. The dismissal policy therefore appears to have had at least a modest impact on the number of teacher separations, although the impact is not as large as the overall nonrenewal numbers would suggest.

It is worth noting that more than half of the dismissed teachers were rehired the following year by another school in the district. For example, 50.6 percent and 56.4 percent of first-year probationary elementary and high school teachers, respectively, who were dismissed in spring 2005 were rehired by a CPS school in the fall. At least some of the dismissals under the policy were the result of position cuts, in which case the teacher’s former principal may have provided the teacher with a good recommendation; it is therefore not surprising that some fraction of dismissed teachers were rehired. It is also likely that some fraction of teachers dismissed due to poor performance were also rehired by other CPS schools.

Which school and principal characteristics are related to dismissal? In both elementary and secondary schools, principals in the district’s larger schools dismissed a smaller fraction of probationary teachers. In elementary schools, higher student achievement at the school is associated with a smaller fraction of probationary teachers being dismissed. Among high schools, however, schools with higher-achieving students dismissed a larger fraction of their probationary teachers. Principals who attended more competitive colleges and principals who were older dismissed a smaller proportion of teachers in both elementary and high schools. Male high-school principals dismissed a significantly smaller percentage of their teachers, while principal gender did not play as important a role at the elementary level. Finally, principals new to the building dismissed a substantially larger fraction of teachers in elementary schools, but not in high schools.

Teacher Characteristics

Turning to the characteristics of individual teachers, I find that prior-year principal evaluations and current-year teacher absences both influence the likelihood of dismissal (see Figure 2). Teachers who were rated satisfactory in the prior academic year were 22.1 percentage points more likely to be nonrenewed than teachers in the same school who were rated superior. Teachers rated excellent were 4.3 percentage points more likely to be dismissed than those rated superior. Given an average dismissal rate of roughly 11 percent, these results suggest that teacher performance as reflected in prior evaluations is strongly associated with dismissal. Teachers who were absent 11 to 20 times between September and March of the current year were also 11.3 percentage points more likely to be nonrenewed than their colleagues who were never absent. Teachers absent 6 to 10 days were 3.5 percentage points more likely to be dismissed.

The results also indicate that principals value teachers with stronger educational backgrounds as measured by college quality. For example, a teacher who attended a highly competitive college (with a Barron’s ranking of four) is nearly 3 percentage points (roughly 15 percent) less likely to be dismissed than a teacher who attended a least-competitive (unrated) college. On the other hand, on average, principals do not seem to value certification exam performance or advanced degrees, at least after taking into account the other available measures of teacher performance.

Interestingly, probationary teachers who were dismissed from another school in the prior year, and rehired by the current school, are substantially more likely to be dismissed a second time. For example, elementary school teachers who were dismissed from another school in the prior year were 4.9 percentage points (about 45 percent) more likely to be let go relative to first-year teachers in the school. In high school, previously dismissed teachers were 13.4 percentage points (more than 130 percent) more likely to be dismissed than first-year teachers. These results suggest that many of the initial nonrenewal decisions were not idiosyncratic, stemming from a particularly bad match, or based on temporary difficulties experienced by the teacher. Rather, they suggest that, at least in many cases, the initial nonrenewal decision reflected a concern with the teacher’s general productivity.

These results provide evidence that principals consider some measures of teacher performance and qualifications in making their dismissal decisions. To the extent that one views student achievement as the primary outcome of interest, however, one should directly assess how a teacher’s ability to improve student achievement influences the likelihood of dismissal. I provide some evidence on this issue by focusing on the relationship between teacher value-added and dismissal for the subsample of 803 elementary school and 1,134 high school teachers for which value-added measures are available.

For elementary schools, a one-standard-deviation increase in teacher value-added is associated with a 7.1-percentage-point (over 100 percent) decrease in the likelihood of dismissal (see Figure 3). In contrast, I find that teacher value-added has zero association with dismissal among the sample of 9th-grade core-subject teachers in high schools. One possible reason for the difference across grade levels is that the assessment used for the 9th-grade value-added measure is the PLAN test, which is given in the fall of a student’s 10th-grade year. PLAN is developed by ACT and is not tightly linked to any particular curriculum. Hence, because of both the timing of the exam and its content, the 9th-grade value-added measures may not capture teacher effectiveness as well as the elementary value-added measures.

Do Principals Discriminate?

One potential concern about policies like Chicago’s that provide principals with greater discretion in personnel decisions is that principals would dismiss teachers capriciously or on the basis of criteria unrelated to performance. Indeed, I find that several teacher demographics, including age, gender, and race, are associated with the likelihood of dismissal, even after controlling for the measures of teacher performance and qualifications described above. Principals are 3.8 percentage points more likely to dismiss male teachers than female teachers, an effect of more than 25 percent given the baseline dismissal rate of 10 to 12 percent. Principals are considerably more likely to dismiss older teachers. For example, teachers 36 to 50 years of age are 4 percentage points (33 percent) more likely to be dismissed than teachers age 22 to 28. The relatively small number of probationary teachers over age 50 is 10 percentage points (nearly 100 percent) more likely to face dismissal than their youngest counterparts. And black teachers are 2.1 percentage points less likely to be dismissed than their colleagues.

While these results raise some concerns, it would be incorrect to conclude on the basis of this evidence alone that principals in Chicago were acting in a discriminatory manner. The analysis reported here cannot control for many direct measures of teacher qualities that principals could legitimately consider in making a dismissal decision (e.g., energy, enthusiasm, ability to relate to children, familiarity with the best instructional practices). Moreover, the sample selection introduced by nonrandom hiring may lead to biased estimates of the relationship between dismissal and any easily observable, predetermined teacher characteristic such as age or gender. If, for example, male teachers were less productive on average than female teachers (or even if the principal believed this to be the case), then the marginal male teacher who was hired must be more attractive on some other, likely unobservable, dimension relative to the marginal female teacher hired.

In order to shed light on the issue of principal discrimination, I examine whether principals are more likely to dismiss teachers of a different gender, age, or race from their own. Although principals are no more likely to dismiss a teacher of the opposite gender, they are somewhat more likely to dismiss teachers of a different race. While these patterns could indicate discrimination, it is possible that they are explained by other factors. Given the widespread belief that same-race role models are crucial for low-income students, it would not be surprising if principals took into account the composition of their student body when making dismissal decisions. Indeed, insofar as prior research has demonstrated that, all else equal, students learn more when taught by a teacher of the same race, this might be a legitimate determination on the part of the principal. My results provide support for this hypothesis. I find that as the fraction of students in the school that share the race of the teacher rises, the likelihood that the teacher will be dismissed declines. Specifically, an increase of 50 percentage points in the fraction of students who share the teacher’s race decreases the likelihood that the teacher will be dismissed by slightly more than 1 percentage point, or 10 percent. More importantly, the evidence that principals are more likely to dismiss a teacher of a different race becomes statistically insignificant after controlling for this variable.

Finally, I find evidence that younger principals are more likely to dismiss older teachers than they are to dismiss younger teachers. There are no obvious explanations for this pattern, although one might speculate that younger principals may value different characteristics in a teacher than older principals. Regardless, this pattern does seem to warrant further exploration.

Conclusions

By comparing the characteristics of dismissed versus nondismissed probationary teachers within the same school and year, the analysis presented above provides a unique source of evidence on which teacher characteristics principals value most highly. I find that principals do consider teacher performance in determining which teachers to dismiss. Principals are significantly more likely to dismiss teachers who are frequently absent and who have received unsatisfactory evaluations in the past. Perhaps most telling, elementary school teachers who were dismissed had significantly lower impacts on student achievement in prior years than their peers who were not dismissed.

These results suggest that reforms along the lines of the Chicago policy could improve student achievement by providing principals with the tools to manage the quality of personnel in their classrooms. It should be noted, however, that many principals—including those in some of the worst-performing schools in the district—did not dismiss any teachers despite the new policy. The apparent reluctance of some Chicago principals to utilize the additional flexibility granted under the new contract may indicate that issues such as teacher supply and/or social norms governing employment relations are more important factors than policymakers have realized.

Brian Jacob is professor of education policy and economics at the University of Michigan. This article is based on a study that is forthcoming in Educational Evaluation and Policy Analysis.

Calls to reform teacher layoff policies have begun to appear with regularity in newspaper editorials, policy briefs, and statehouses—and for good reason. A growing body of research confirms that teacher quality is the most influential in-school factor driv­ing student achievement. That being the case, teacher dismissal policies and procedures can have profound implications for how much students learn.

Newly available data on “reduction-in-force” (RIF) notices received by teachers in Washington State shed light on the consequences of existing layoff policies for student achievement as well as the consequences of adopting alternatives. Our analysis of these data provides strong evidence that seniority plays an out­sized role in determining which teachers are targeted for layoffs, likely in part because collective bargaining agreements ordinarily require that the teachers last hired are the first to be fired. Those in subject areas with teacher shortages, such as mathematics and sci­ence, are less likely than other teachers to receive a lay­off notice, suggesting that districts have some degree of flexibility in their dismissal procedures. However, were districts to adopt policies that allowed admin­istrators to dismiss teachers according to their effec­tiveness rather than their seniority, they could lay off fewer teachers, achieve the same budgetary savings, and increase the overall efficacy of their teaching force.

Seniority-Based Layoff Policies

In the overwhelming majority of school-district collective bargaining agreements, “last in, first out” provisions make seniority the determining factor in which teachers are laid off. All of the 75 largest school districts in the nation use seniority as a factor in layoff decisions, and seniority is the sole factor determining the order of layoffs in more than 70 percent of these districts.

The situation in Washington State—the focus of this study—looks similar. A review of the collective bargaining agreements operating in Washington’s 10 largest school districts shows that all use seniority as a basis for determining layoffs, and 8 of these districts use seniority as the only determinant of which teachers get laid off.

There are notable examples of districts that do not rely solely on seniority. In 2004, the Chi­cago Public Schools changed its policies to allow principals’ evaluations of untenured teachers to influence layoff decisions (see “Principled Principals” research). And the Los Angeles Unified School District recently agreed to limit the use of seniority in layoff determinations as part of a settlement in a lawsuit brought by the American Civil Liberties Union (ACLU). Over the past two years, more than a dozen states have sought to change laws that make seniority the determining factor in layoff decisions; so far, Florida, Idaho, Utah, and Ohio have succeeded.

Driving these changes is a belief that seniority-based layoff policies may have negative consequences for student achieve­ment. First, to achieve a targeted budget reduction, school districts need to lay off a greater number of junior teachers than senior teachers (as junior teachers have lower salaries), meaning that a seniority-based layoff policy will cause class sizes to rise more than they would under an alternate arrange­ment. Second, the most-senior teachers may not be the most effective teachers. With a seniority-based layoff policy, school systems may be forced to cut some of their most promising new talent rather than dismiss more-senior teachers, who may not be terribly effective in raising student achievement. A final way in which seniority-based systems may have consequences for student achievement is that strict adherence to seniority would require at least some districts to lay off teachers in subject areas with teacher shortages, such as math and special education.

Beyond the effects of seniority-based layoffs on the teacher workforce as a whole are potential distributional conse­quences. In many districts, schools with high proportions of at-risk students tend to employ the most first- and second-year teachers. Under a seniority-based layoff policy, these schools stand to lose the largest share of their teachers.

Data

This study relies on a unique dataset from Washington State that links teachers to their schools and, in some cases, to their students; the dataset also includes information on those teachers who received RIF notices in the 2008–09 and 2009–10 school years. In the 2008–09 school year, 2,144 employees received a layoff notice and in 2009–10, some 450 employees received a notice.

Employees who received these notices can be linked with administrative records of their credentials, school assignments, academic degrees, and compensation. The administrative database we used provides a record of employees working in Washington State’s school districts and includes information such as their places of employ­ment, experience and degree, gender and race, and annual compensation levels.

We restrict our analysis to employees who were in a teach­ing position the year they received a layoff notice. Our final sample includes 1,717 teachers who received a layoff notice in 2008–09 and 407 teachers who received one in 2009–10, with 130 teachers who received a layoff notice in both school years. Overall, about 2 percent of teachers in the state received a layoff notice in either year. It is important to stress that not all these teachers were ultimately laid off, largely due to the influx of federal stimulus money. Of the 1,717 teach­ers who received a RIF notice in 2008-09, for example, 1,457 returned to the same district in 2009-10. We still focus on all RIF notices because they indicate the teachers who were targeted for layoffs, and thus tell us about the likely effects of the system that governs layoffs.

The database does not include a direct measure of a teach­er’s seniority in the current district, so we estimate seniority based on how many years the teacher has been employed by the same district. The credentials data include where each teacher was trained and in what areas each teacher holds endorsements. We create a measure of the selectivity of each teacher’s college and code each endorsement a teacher holds in any of 10 subject areas.

Information about the schools in which teachers are employed comes from two sources. Washington State Report Card data provide measures of racial composition, student-teacher ratios, the percentages of students enrolled in the free or reduced-price meals program, total enrollment, and the percentage of students who passed the reading and math Washington Assessment of Student Learning exams in each teacher’s school. We use the Common Core of Data to iden­tify teachers in urban areas, the grade level of each teacher’s school, and the per-pupil expenditure on instruction by each teacher’s district.

We can also link a subset of teachers to their students’ test-score performance, which allows us to use value-added models to estimate their teaching effectiveness. Our data on student achievement come from the Washington State Assessment of Student Learning, a statewide test given annually in 3rd through 8th grade as well as in 10th grade. The student database also includes information on race and ethnicity, free or reduced-price meal eligibility, and status in the following programs: Learning Assistance Program reading/math, Title I reading/math, Title I Migrant, Gifted/Highly Capable, State Transitional Bilingual Program, and Special Education.

Methods

We first examine the simple associations between the various teacher and school characteristics listed above and the likelihood of receiving a layoff notice. In order to provide a more detailed picture of the factors that are associated with teacher layoff notices, we then examine the effects of each of these various factors on the prob­ability that a teacher received a layoff notice, while con­trolling for the others. Of course, these relationships are correlations only and in theory may not represent causal relationships. However, we are confident that, despite the nonexperimental nature of this study, its findings none­theless provide an accurate picture of the causal impact of, for instance, a teacher’s credential on the likelihood of receiving a layoff notice.

The teacher characteristics that we examine include senior­ity in district, degree level (master’s or higher vs. bachelor’s), gender, race, college selectivity, and endorsement area. The school characteristics include whether it is in an urban area, grade level (e.g., high school), the number of students enrolled, student-teacher ratio, the percentage of students who are eli­gible for the free or reduced-price lunch program, the percent­age of minority students, and measures of student achievement in reading and math. In addition, we control for district-level characteristics, including total enrollment, per-pupil expendi­tures, and percentage of funding that comes from local, state, and federal sources.

These analyses identify the teacher, school, and district characteristics that are associated with layoff notices, but perhaps of greater interest is the relative effectiveness of teachers who receive layoff notices. For the subset of teach­ers who can be linked to students, we are able to estimate value-added measures of classroom performance for each teacher in each year. These indicate how well a teacher’s students did relative to other teachers’ students, controlling for prior student achievement and for student and fam­ily background characteristics (for example, age, race and ethnicity, disability, free or reduced-price lunch status, and parental education level).

Who Gets RIFed?

Not surprisingly, we find that most of the teachers receiving layoff notices are relatively junior. Approximately 60 percent of teachers receiving layoff notices have two or fewer years of experience, and approximately 80 percent have two or fewer years of seniority within their current district. It is interesting to note, however, that some teachers who receive layoff notices are well into their careers, implying that at least some districts in the state are making judgments about which teachers should be laid off based on criteria other than seniority.

Teachers who received layoff notices are also far less likely to hold an advanced degree. Consequently, there is an aver­age difference of about $15,000 in salary between teachers who did and did not receive notices. Had all 1,717 teachers who received layoff notices in 2008–09 actually been laid off, the salary savings in the state would have been $5,521,238. As noted earlier, one of the prevail­ing critiques of seniority-based layoffs is that it is necessary to lay off more teachers in order to attain a specified budget objective than it would be if districts used alternative criteria. If teach­ers were laid off at random (so that the laid-off teachers made the average salary in their dis­trict), we estimate that it would only be neces­sary to lay off 1,349 teachers in order to attain the same budgetary savings. This is roughly 20 percent less than the actual number of teachers who received layoff notices.

According to the 2006 report “Educator Supply and Demand in Washington State,” there are 14 endorsement areas for which there are “high degrees of shortage,” all of which fall into math, science, or special education. We classify any teacher with an endorsement in one of these areas accordingly. There is some evidence to suggest that school districts are choosing to retain teachers in subject areas with teacher shortages, with 13.3 per­cent of teachers that received layoff notices falling into such a category compared to 15.1 percent of teachers who did not receive a notice.

Teachers receiving a notice tended to be in smaller schools, but were not, in general, more likely to be teaching in schools with high proportions of minority students or lower test-score levels. However, school-level measures can mask a significant degree of teacher sorting across classrooms within schools. For the subset of teachers who can be linked to their students, we find that teachers who received a layoff notice are more likely to be teaching poor, non-white, and lower-scoring students than other teachers.

We next examine our value-added measures of teacher effectiveness and find that teachers who received layoff notices were about 5 percent of a standard deviation less effective on average than the average teacher who did not receive a notice. This result is not surprising given that teach­ers who received layoff notices included many first- and second-year teachers, and numerous studies show that, on average, effectiveness improves substantially over a teacher’s first few years of teaching.

Explaining RIFs

Our analysis of multiple factors indicates that, as expected, seniority plays an important role in determining whether teachers receive a layoff notice. We find additional evi­dence that districts are choosing to retain teachers thought to have advanced or atypical skills. On average, teachers with a master’s degree or an endorsement in a subject area with teacher shortages are about 0.6 percentage points less likely to receive a RIF notice. Conversely, teachers with endorsements in health, physical education, or the arts are far more likely to receive a layoff notice. Finally, we find evidence that school districts behave strategically by retaining teachers who have endorsements in multiple areas and therefore provide flexibility in terms of the classes they can teach. Perhaps surprisingly, controlling for district and school characteristics does not noticeably change the results reported above, and few of the school-level vari­ables identifying student demographics are predictors of which teachers receive layoff notices.

Finally, we ran our analysis including value-added measures of teacher effective­ness for the subset of teachers we are able to link to individual students. It is first worth noting that the inclusion of the teacher effec­tiveness measures does little to change the estimated effects of the teacher, school, and district characteristics discussed above. More importantly, the effects of the value-added measures (based on both math and read­ing scores) are close to zero, suggesting that effectiveness plays little or no role in deter­mining which teachers are targeted for lay­offs. And, these results were robust to a vari­ety of different ways of measuring teacher value added. In other words, the fact that teachers who received layoff notices were, on average, somewhat less effective than their peers is an artifact of the relationship between effectiveness and seniority.

Policy Implications

Our findings largely comport with what one would expect given seniority provisions in collective bargaining agree­ments. The surprise is that factors other than seniority do appear to influence which teachers are targeted for layoffs.

To get a more concrete sense of the extent to which various factors play into the targeting of teachers for layoffs, we ran simulations based on the effects calculated by our statistical model. First, we calculate the expected probability of a teacher with each combination of endorsement area and seniority level receiving a layoff notice. Although a teacher’s endorse­ment area does affect the likelihood of being laid off, the effect is far smaller than the influence of seniority. For instance, we estimate the probability that a first-year special education teacher receives a layoff notice is 6.2 percent, compared to 17 percent for a first-year health/physical education teacher. This difference is statistically significant, but it pales in com­parison to the difference in probability for a first-year teacher compared to a teacher with 12 or more years of seniority: The estimated probability of a teacher with 12 or more years of seniority receiving a layoff notice is less than one-quarter of 1 percent for every endorsement area (see Figure 1).

Next we examine the implications of employing an effec­tiveness-based layoff policy rather than the seniority-driven system currently in place. First, we calculate a value-added measure of effectiveness that com­bines data from all available years and both sub­jects (averaging math and reading). Teachers in each school district are then ranked accord­ing to this value-added score. Finally, starting with the least effective teachers in each district and moving up the effectiveness ladder, enough teachers are assigned to a hypothetical layoff pool to achieve a budgetary savings for each district that is at least as great as the budgetary savings each district would have seen had all the teachers who received a layoff notice in 2008–09 actually been laid off.

The overlap between the subgroup of teach­ers who received a layoff notice and the sub­group of teachers who received one in our simu­lation is relatively small—only 23 teachers (or 16 percent of the teachers for whom we could estimate value-added who received a layoff notice). Moreover, because the teachers who received layoff notices in our simulation were more senior (and had higher salaries) than the teachers who actually received layoff notices, the simulation results in far fewer layoffs. We calcu­late that districts would only have to lay off 132 teachers under an effectiveness-based system in order to achieve the same budgetary savings they would achieve with 145 layoff notices under today’s seniority-driven system, a difference of about 10 percent.

As expected, there are large differences in classroom effec­tiveness between teachers who actually received layoff notices and those who would have received them in our effectiveness-based simulation. The two groups differ by about 20 percent of a standard deviation in students’ math and reading achieve­ment (see Figure 2). The magnitude of the difference is strik­ing, roughly equivalent to having a teacher who is at the 16th percentile of effectiveness rather than at the 50th percentile. This difference corresponds to roughly 2.5 to 3.5 months of student learning.

Since there is little overlap between the samples under these different scenarios, we investigate the likelihood that different types of students might be disproportion­ally affected by one type of layoff system. For the subset of teachers who can be linked to student-level data, we consider the characteristics of the students whose teachers received a layoff notice under the actual system and in our simulation. We find that the probability that students in a particular subgroup have a teacher who received a layoff notice varies considerably from one subgroup to the next. In particular, black students are far more likely than other students to have been in a classroom of a teacher who received a layoff notice. The effectiveness-based layoffs result in fewer layoff notices and are much more equita­bly distributed across student subgroups; black students in particular are only marginally more likely to have been in a classroom with a teacher who received a layoff notice under this system.

Districts across the country are rethinking layoff strate­gies. This is sensible, because although the simplicity and transparency of a seniority-based system certainly has advantages, it is hard to argue that it is in the best interest of students. The effectiveness-based system in our simulation would result in a very different group of teachers targeted for layoffs than does the current system and in layoffs that affect different segments of the student population. Most importantly, the differences in the effectiveness of teach­ers laid off under each type of system have implications for student achievement.

Dan Goldhaber is director of the Center for Education Data and Research at the University of Washington Bothell and a co-editor of Education Finance and Policy. Roddy Theobald is a researcher at the Center for Education Data and Research and doctoral student in statistics at the University of Washington.

The working paper on which this article is based is available here.

So, if we believe these OECD numbers (which the WSJ apparently did in this blog post), U.S. teachers work 15.3% more hours per year than do their colleagues in other developed countries.

But if you believe the OECD comparison I have a lovely bridge to sell to you.  According to the report’s methodological appendix, the method by which the U.S. information was collected was different (and clearly less reliable)  than how it was collected from all of the other countries.  In every country except the U.S. the hours worked was derived from teacher contracts or laws.  But in the U.S. the information was drawn from self-reported responses to a survey of teachers.  (See p. 75 of the appendix).

A valid comparison would require that the information be collected in similar ways across all countries — either we rely upon self-reports in surveys of teachers for all countries or we rely on contractual hours for everyone.  But using self-reports for the U.S. and contractual hours for everyone else produces obvious distortions.  People may be inclined to exaggerate the hours they work in a survey.  And the definition of time worked is ambiguous.  If I think about my students while I am brushing my teeth or running on the treadmill am I working during that time?

We have good reason to suspect that the self-reports from U.S. teachers are over-stated.  If teachers really worked 1,913 hours over 180 days, as the report claims, they would be working 10.63 hours per day.  And the numbers I’ve provided are just for primary school students.  For high schools, the OECD report claims U.S. teachers are working 1,998 hours over 180 days, which works out to 11.1 hours per day.  I know some teachers are very conscientious and work long hours but I simply do not believe that the average high school teacher is working 11.1 hours per day.

I know this might invite the wrath of Diane Ravitch’s Army of Angry Teachers, but I suspect that the average hours worked by U.S. teachers is significantly less than the OECD says (and the WSJ repeats).  And I know that the comparison between U.S. and other countries is flawed by collecting the information from self-reports in the U.S. but from contracts everywhere else.

-Jay P. Greene

An unabridged version of this article is available here.

In recent years, a consensus has emerged among researchers that teacher quality matters enormously for student performance. Students taught by more-effective teachers learn substantially more over the course of the year than students taught by less-effective teachers. Yet little is known about what makes for a more-effective teacher.

Most research on teacher effectiveness has focused on teacher attributes, finding that readily measurable characteristics such as experience, certification, and graduate degrees generally have little impact on student achievement. Relatively few rigorous studies look inside the classroom to see what kinds of teaching styles are the most effective. We tackle this underexplored area by investigating the relative effects of two teacher practices—lecture-style presentations and in-class problem solving—on the achievement of middle-school students in math and science.

Ever since John Dewey explored hands-on learning at the University of Chicago Laboratory School more than a century ago, lecture-style presentations have been criticized as old-fashioned and ineffective. It is said, for example, that lectures presume that all students learn at the same pace and fail to provide instructors with feedback about which aspects of a lesson students have mastered. Students’ attention may wander during lectures, and they may more easily forget information they encountered in this passive manner. Lectures also emphasize learning by listening, which may disadvantage students who favor other learning styles.

Alternative instructional practices based on active and problem-oriented learning presumably do not suffer from these disadvantages. But they may have their own shortcomings. Learning by problem-solving may be less efficient, as discovery and problem-solving often take more time than mastering information received from an authority figure. And incorrect or misleading information may be conveyed in conversations among students in middle schools.

Nonetheless, a number of small-scale studies have identified positive impacts of interactive teaching styles on student learning. As a consequence, prominent organizations such as the National Research Council and the National Council of Teachers of Mathematics, since at least 1980, have called for teachers to engage students in constructing their own new knowledge through more hands-on learning and group work. By the mid-1990s, in a study for the National Institute for Science Education, Iris Weiss could identify “some encouraging signs. The majority of elementary, middle, and high school science and mathematics classes worked in small groups at least once a week, and roughly one in four classes did so every day. Moreover, the use of hands-on activities had increased since the mid-1980s.” Even so, more than a decade later, traditional lecture and textbook methodologies continue to be a significant component of science and mathematics instruction in U.S. middle schools. A rigorous, large-scale study has yet to resolve a question that has divided pedagogical thinking for generations.

In our study, we examine whether student achievement in the United States is affected by the share of teaching time devoted to lecture-style presentations as distinct from problem-solving activities. Employing information on in-class time use provided by a nationally representative sample of U.S. teachers in the 2003 Trends in International Mathematics and Science Study (TIMSS), we estimate the impact of teaching practices on student achievement by looking at the differential effects on the same student of two different teachers, using two different teaching strategies. We find that teaching style matters for student achievement, but in the opposite direction than anticipated by conventional wisdom: an emphasis on lecture-style presentations (rather than problem-solving activities) is associated with an increase—not a decrease—in student achievement. This result implies that a shift to problem-solving instruction is more likely to adversely affect student learning than to improve it.

Data and Methodology

Our research draws on data from the 2003 Trends in International Mathematics and Science Study (TIMSS). The TIMSS data comprise information on students in two grades in a number of countries, but we utilize only information on 8th-grade students in the United States. Our sample includes 6,310 students in 205 schools with 639 teachers (303 math teachers and 355 science teachers, of which 19 teach both subjects). In addition to test scores in math and science, the TIMSS data include background information on students’ home and family life as well as data on teacher characteristics, qualifications, and classroom practices. School principals provide information on school characteristics.

Most important for our purpose, teachers were asked what proportion of time in a typical week students spent on each of eight in-class activities. The overall time in class apportioned to three of these activities—listening to lecture-style presentation, working on problems with the teacher’s guidance, and working on problems without guidance—likely provides a good proxy for the time in class in which students are taught new material. We divided the amount of time spent listening to lecture-style presentations by the total amount of time spent on each of these three activities to generate a single measure of how much time the teacher devoted to lecturing relative to how much time was devoted to problem-solving activities.

A change in our measure of teaching style can be interpreted as a shift from spending time on one practice to spending time on the other, holding constant the total time spent on both practices. For example, an increase of 0.1 indicates that 10 percentage points of total time devoted to teaching new material are shifted from teaching based on problem solving to giving lecture-style presentations. We combined the other teaching activities (besides lecturing and problem solving) into a separate measure of the share of total teaching time devoted to other activities and control for this measure throughout our analysis. We also control for the total number of minutes per week that the teacher reported teaching the math or science class, as more total instructional time could have an independent effect on student learning.

Although it is difficult to determine from the TIMSS data exactly how much time is spent on lecturing as distinct from problem-solving activities, it appears that teachers generally follow the advice given by progressive educators. On average, they allocate twice as much time to problem-solving activities as to direct instruction. Specifically, teachers devote about 40 percent of class time to problem-solving activities (with or without teacher guidance); during roughly 20 percent of class time, students listen to the initial presentation of material to be learned. The remainder of the class time is allocated to such tasks as class management, reviewing homework, re-teaching the material, and clarifying content (see Figure 1).

Teachers who spent more time lecturing were more likely to be male and under age 50. Interestingly, they were also less likely to have the maximum number of years of teacher training registered by the background survey or to have taken pedagogical or content knowledge classes in the prior two years (see Figure 2).

A key challenge in studying the effects of teaching practices is that teachers may adjust their methods in response to the ability or behavior of their students. If teachers tend to rely more on lectures when assigned more capable or attentive students, this would generate a positive relationship between the amount of time spent lecturing and student achievement, even in the absence of a true causal effect. Similarly, there could be unobserved differences between students whose teachers rely more and less heavily on lecturing if, for example, teachers in schools serving low-income students adopt different practices than teachers in other types of schools.

To address these concerns, we exploit the fact that the TIMSS study tested each student in both mathematics and science. This allows us to compare the math and science test scores of individual students whose teacher in one subject tended to emphasize a different teaching style than their teacher in the other subject. In other words, we ask, if a given student’s math teacher spent more (or less) time lecturing than his or her science teacher, does the student perform better or worse on the math test than on the science test?

Results

Contrary to contemporary pedagogical thinking, we find that students score higher on standardized tests in the subject in which their teachers spent more time on lecture-style presentations than in the subject in which the teacher devoted more time to problem-solving activities. For both math and science, a shift of 10 percentage points of time from problem solving to lecture-style presentations (e.g., increasing the share of time spent lecturing from 20 to 30 percent) is associated with an increase in student test scores of 1 percent of a standard deviation. Another way to state the same finding is that students learn less in the classes in which their teachers spend more time on in-class problem solving.

Importantly, the strength of the relationship increases when we restrict our analysis to the roughly one-third of students in the TIMSS sample who had the exact same peers in both their math and science classes. Among this group of students, a shift of 10 percentage points of time from problem solving to lecturing is associated with an increase in test scores of almost 4 percent of a standard deviation—or between one and two months’ worth of learning in a typical school year (see Figure 3). This pattern increases our confidence that the overall result does not reflect differences in the peer composition of students’ math or science classes. In fact, it suggests that peer effects may actually be leading us to understate the strength of the relationship between lecturing and student learning.

Do certain types of students benefit more from lectures than others? We find suggestive evidence that the relationship between lecture-style teaching and achievement is strongest among higher-achieving and more-advantaged students. For example, the positive effect is largest for students who report having more than one bookcase in the home, a rough indicator of the quality of their home environment. There is no evidence, however, that lower-achieving students or students from less-advantaged backgrounds learn less when their teachers emphasize lectures.

These patterns are consistent with the findings of a 1997 study by Dominic Brewer and Dan Goldhaber, which found that more in-class problem solving for American 10th-grade students in math is related to lower test scores on a standardized test. Because our results are based on comparisons of the same student in two different classes, however, they are less subject to the concern that teachers adjust their practices based on the students to which they are assigned. Furthermore, the other commonly investigated teacher characteristics (e.g., gender, experience, and credentials) do not show significant effects on student achievement in our analysis. This is in line with previous findings in the literature and underscores the importance of the statistical relationship between more lecture-style teaching and student achievement.

While the richness of the TIMSS data enables us to control for an unusually large set of teacher characteristics, our results could still be biased if teachers with different effectiveness levels are more likely to choose different teaching styles. For example, if more-effective teachers tend to spend more time lecturing because they are good at it and enjoy it, then our results could show a positive effect of lecture-style presentations, even if those teachers would have been even more effective had they devoted more time on problem-solving activities. Given the pedagogical emphasis on the use of problem-solving activities, it seems unlikely that the very best teachers would be using the less-effective teaching style (the only alternative explanation for our finding).

Still, it is important to keep in mind that our results are limited to student achievement as measured by the 2003 TIMSS test scores in 8th-grade math and science in the United States. Different results might be found for different subjects, grades, or tests. Depending on the teacher, the students, the content taught, or other factors, problem-solving activities could turn out to be the more effective style. Even though lecture-style teaching seems to be a more effective method in middle-school math and science, that does not mean it would be the preferable approach to elementary-school reading.

Also, our findings are based on student performance on the TIMSS math and science exams, which are designed to measure mastery of factual knowledge of the curricula that schools expect students to learn. Other tests intended to measure problem-solving ability and the competence to apply mathematical and scientific concepts in real-world settings (such as the Programme for International Student Assessment [PISA] administered by the Organization of Economic Cooperation and Development) might yield different results. Unfortunately, we are unable to ascertain whether this might be the case, as PISA did not ask teachers about their pedagogical approach.

Finally, our information on teaching practices, which is based on in-class time use reported by teachers, does not allow us to distinguish between different implementations of teaching practices. In other words, a certain teaching technique may be very effective if implemented in the optimal way. But the strength of our approach is that it examines which teaching style turns out to be effective, on average, for teachers in general. Optimal teaching methods that cannot be executed by teachers in general may do more harm than good.

Conclusion

Given the limitations of the data, our finding that spending increased time on lecture-style teaching improves student test scores results should not be translated into a call for more lecture-style teaching in general. But the results do suggest that traditional lecture-style teaching in U.S. middle schools is less of a problem than is often believed.

Newer teaching methods might be beneficial for student achievement if implemented in the proper way, but our findings imply that simply inducing teachers to shift time in class from lecture-style presentations to problem solving without ensuring effective implementation is unlikely to raise overall student achievement in math and science. On the contrary, our results indicate that there might even be an adverse impact on student learning.

Guido Schwerdt is a postdoctoral fellow at the Program on Education Policy and Governance (PEPG) at Harvard University and a researcher at the Ifo Institute for Economic Research in Munich, Germany. Amelie C. Wuppermann is a postdoctoral researcher at the University of Mainz, Germany.

An unabridged version of this article is available here.

While much of the debate around Teach For America (TFA) in recent years has focused on the effectiveness of its nontraditional recruits in the classroom, the real story is the degree to which TFA has succeeded in producing dynamic, impassioned, and entrepreneurial education leaders. From its inception as Wendy Kopp’s senior thesis project at Princeton more than two decades ago, TFA has sought to bring more teaching talent to some of the nation’s most disadvantaged communities and create a corps of change agents like Rhee, Feinberg, Levin, and Johnston. How well has TFA fared on that second score? Here, in a new line of research, we seek to answer that question.

Since its founding in 1989, TFA has placed more than 24,000 high-achieving college graduates in some of America’s neediest schools. This has produced an alumni network populated by impassioned former educators. TFA aims, proclaims the web site, to turn these alumni into “lifelong leaders for fundamental change, regardless of their professional sector.” Its efforts include keeping close connections with alumni and providing a variety of opportunities to volunteer at schools, join education-oriented political campaigns, advocate, and connect with a wide-reaching education network.

To date, the vast majority of research on TFA has focused on the classroom effectiveness of corps members and how long they remain in classrooms. Very little is known about TFA corps members who leave teaching but stay involved in education reform more broadly. In a recent study of TFA alumni, Doug McAdam and Cynthia Brandt (2009) argue that corps members are more likely to remain in education, whether in administration, educational policy work, or charter school management, than those who opt not to enter TFA or drop out of the program. This suggests that TFA has a lasting influence on corps members’ careers, but does not address the question of whether these individuals become the kind of change agents envisioned in TFA’s mission of eliminating “educational inequity by enlisting our nation’s most promising future leaders.”

We pursue that question here, as part of a larger analysis of organizations that successfully “spawn” education entrepreneurs. Examining the work histories of founders and top management team (TMT) members at nationally prominent entrepreneurial education organizations, we find that TFA appears more frequently in the professional backgrounds of these proven entrepreneurial leaders than does any other source in our sample. We don’t know whether it is the TFA experience, the criteria by which TFA selects its corps members, or institutional relationships that account for this. However, the research does find that TFA is producing a large number of entrepreneurial leaders. How and why this is so, and what might be learned from TFA’s success, are questions that deserve careful scrutiny.

Entrepreneurs Needed

The education sector has long struggled to attract and retain high-quality professionals. At the same time, stubborn achievement gaps, increased competition among school providers, and a heightened focus on performance have created an appetite for creative problem solving and scalable, transformational initiatives. In a world of online learning, school turnarounds, Race to the Top, and the Investing in Innovation Fund, there is room for leaders who are able to lever change by creating and expanding organizations of all kinds. Turning these opportunities into results requires people able to create and lead new, high-quality ventures.

With the proliferation of teacher residency and principal leadership programs, education has seen many efforts to recruit, develop, and retain quality teachers and administrators in recent years. However, there are fewer organizations aimed at developing leaders to direct reform initiatives outside the classroom or the schoolhouse. TFA is one among a small cadre of organizations that currently includes New Leaders for New Schools, Education Pioneers, and Teach Plus. TFA is particularly notable for its efforts on this score, as it engages former corps members through “Alumni Summits” and initiatives to advance alumni in positions of leadership as nonprofit board members, public officials, and leaders at the school and classroom level. It also supports alumni through partnerships with graduate schools and employers to help them transition to the next steps in their careers.

Recently, TFA started a new program, the Social Entrepreneurship Initiative, which explicitly promotes innovation and entrepreneurship in the education sector. The program facilitates connections between alumni interested in starting education ventures with established social entrepreneurs. The initiative supports TFA alumni who are applying for fellowships such as Echoing Green and the Mind Trust, provides tools for developing fundraising plans and grant proposals, and publishes a newsletter that includes information about funding opportunities and management strategies.

Today, there is a sizable network of TFA alumni who have become education entrepreneurs. We have already mentioned KIPP Academy cofounders Mike Feinberg and Dave Levin, who started a single charter school in 1994 that has evolved into one of the most well-known charter organizations in the U.S., with 99 schools in 19 states and the District of Columbia. TFA alum Chris Barbic founded YES Prep Public Schools, which has grown to serve 4,200 students at eight campuses throughout Houston. Sarah Usdin began New Schools for New Orleans (NSNO) in 2006, after Hurricane Katrina devastated public schooling in that city. Before heading up the D.C. school system, Michelle Rhee established The New Teacher Project. Accounts sometimes suggest that these individuals are intriguing outliers. Our research suggests that they are evidence of TFA’s success at recruiting and creating change agents.

Research Methods

The methods used in this study mirror those applied in research on entrepreneurial spawning in other sectors, such as biotechnology. We first identified a group of entrepreneurial organizations within the education sector and traced their founders’ and TMT members’ work histories. We then identified organizations that appeared multiple times as previous employers across the sample and, hence, could be considered “spawners” of entrepreneurial leaders.

To create our list of entrepreneurial education organizations, we limited our search to nonprofit and for-profit organizations that were founded after 1989, TFA’s inaugural year; that focused on domestic, K–12 public education reform; and that could be considered nationally prominent. We drew on three distinct sources to identify organizations for our sample. The first was an electronic survey of 14 widely recognized experts in public school innovation. We asked participants, “From your perspective, what are the top 15 U.S. entrepreneurial education organizations that have emerged in the sector since 1989?” All 14 participants responded, and we identified 16 organizations that more than one respondent identified as a top organization. Next, we identified organizations supported by a donor clearinghouse of venture philanthropies and foundations whose mission is to support social entrepreneurship in K–12 public education across the nation. Finally, we conducted publication searches in popular and academic media using Lexis Nexus and Google Scholar. The searches were conducted in November 2009 and used the following search terms: Education, Entrepreneur*, Organization, and K–12.

These methods yielded a comprehensive list of 49 organizations; many are charter management organizations, some recruit and/or train human capital, and others offer supplemental resources to the public education sector, such as software technologies for data management and assessment or afterschool programs (see sidebar).

We then constructed a database of the work histories of the 49 organizations’ leadership members, comprising 71 founders and cofounders and 320 TMT members. We make the distinction between founders and other management team members in the event that there are noteworthy differences between those who start organizations and those hired to manage daily functioning, growth, and stability. Often, the organizations in our sample publicly listed the founders and members of the management team, along with their work and educational histories. When these data were ambiguous or not publicly available, we called the organizations to request the information.

We term the organizations that appear in founders’ and TMT members’ work histories “originating organizations.” To ascertain which originating organizations were the most prolific spawners of entrepreneurs, we identified those that had at one time employed a founding member of at least 2 of the 49 entrepreneurial organizations in our sample.

Entrepreneurship and TFA

Of all the originating organizations that appeared in work histories, TFA appeared the most frequently. Let’s look first at the 71 founders or cofounders of the 49 entrepreneurial organizations. TFA appeared in the work history of at least one founder of seven of these organizations, or about 15 percent. The next most-represented originating organizations—the San Francisco Public Schools, Newark Public Schools, Chicago Public Schools, AmeriCorps, the White House Fellows program, McKinsey & Company, and the United States Department of Education—each appeared in the work history of a founder of two (or about 4 percent) of these organizations. In other words, the drop-off from TFA to these other large and/or esteemed organizations is stark indeed.

To get a sense of whether TFA’s outsized success is simply the result of its size or TFA is indeed punching “above its weight,” it’s worth noting the comparative size of these various ventures. TFA is today an organization with almost 10,000 employees, including 8,200 current corps members. But TFA’s size a decade ago was only about one-quarter of what it is today, meaning that the alumni pipeline is much thinner than its current size suggests. TFA estimates that it has produced more than 20,000 alumni. TFA is clearly smaller than organizations like the Chicago Public Schools, with around 41,000 employees, and McKinsey, with some 17,000 employees. TFA is dwarfed by the approximately 75,000 current AmeriCorps members and some 500,000 alumni (some AmeriCorps volunteers are also TFA corps members), but is far larger than the White House Fellows program, with 13 current fellows and some 600 alumni. In short, TFA has fared impressively for its relative size.

While many founders have participated in TFA, there is little evidence of their having had other work or internship experiences in common. One reason for this homogeneity may be that approximately 23 percent of the founders had only one job prior to starting their own venture. A lack of experience created fewer opportunities to build professional networks, making the large number of TFA alumni among founders all the more salient.

TFA stands out in the work histories of the TMT members at the 49 organizations on our list as well (see Figure 1). Fourteen of the 49 entrepreneurial organizations had at least one TMT member who was once a TFA corps member or employee, and 10 of these organizations had at least one member who had been a TFA corps member and worked for TFA national. Compare this to the next three highest-ranked originating organizations: 10 entrepreneurial organizations had at least one TMT member who had been employed by the New York City Public Schools, nine entrepreneurial organizations employed KIPP alumni, and the work histories of seven entrepreneurial organizations’ TMT members included Andersen Consulting.

A Look at the Spawners

Only two of the originating organizations that spawned at least two founders, the White House Fellows program and McKinsey & Company, operate outside the public education sector. McKinsey, a management consulting firm, is the only private institution on the list. When it came to spawning TMT members, McKinsey was joined by its consulting brethren Andersen and Deloitte. For TMT members, consulting was a common professional experience with about 10 percent of all TMT members having this practice in their backgrounds.

It is interesting to consider why experience in the consulting industry is not unusual in the career histories of TMT members. Members of top management teams, including chief finance officers, chief operating officers, and even those leading growth and marketing divisions, face complex challenges. Consultants are commonly hired to solve problems in these functional units in both the private and public sectors. Perhaps their skills translate well in the entrepreneurial world. Former consultants may be particularly adept at addressing tough management issues in entrepreneurial organizations in the education sector, where challenges arise both internally and externally, due to the complicated political and financial dynamics of meeting public education needs in the U.S. There may be certain functional roles on TMTs for which having a consulting background prepares leaders particularly well.

Additionally, consulting firms such as McKinsey are increasingly offering their services in the education sector. For example, McKinsey’s Social Sector Office supports an education practice that focuses on systems strategy and transformation, talent and performance management, administration and operations, and institutional strategy and innovations. Teams in McKinsey’s education practice regularly publish reports on the education sector, including a recent analysis of the economic impact of the achievement gap and strategies for attracting top undergraduates to and retaining them in the teaching profession. Such work may be exposing their employees to the overwhelming need in the education sector for solutions to challenging problems. That exposure, coupled with entrepreneurial aspects of the organizational culture, employee selection criteria, or institutional relationships may create an environment similar to TFA. Again, we cannot be sure at this stage what factors may be at play, but it is certainly an intriguing finding.

Another leading spawner of team members is KIPP. Nine organizations in the sample had at least one TMT member who had worked for KIPP’s national office or in a KIPP school. Given that KIPP was started by two TFA alumni, maintains close ties with TFA, and recruits many of its teachers from the TFA ranks, it is no surprise that five organizations in the sample had TMT members who had previously worked both for TFA and for KIPP.

Several school districts were also among the organizations that showed up most often in TMT members’ work histories. New York City appeared most often. Other districts were Washington, D.C., San Francisco, Oakland, and Los Angeles.

Looking across all the spawning organizations, one returns to the question of why, given the many sources feeding into the education talent pipeline, TFA seems so prolific. It seems clear that explanatory factors include the criteria by which TFA recruits, the organization’s strong and purposive culture, the skills that corps members develop, and the opportunities provided to alumni. Just to take one example, by providing talented young college grads with classroom experience, TFA confers upon them a degree of credibility that opens doors that might open less readily for others. Sorting out the relevant import of these elements is far beyond the scope of our current effort, but it is an exercise well worth pursuing for those reformers eager to identify, emulate, and amplify TFA’s successes.

TFA’s Influence

Is there cause to suspect that there are any systematic differences between those education entrepreneurs who are TFA alumni and those who are not? Given their classroom experience, for instance, are TFA alumni more likely to wind up in instructional or curricular roles than are TMT members who are not TFA alumni?

To investigate this possibility, we looked at the 30 TFA alumni who are TMT members at organizations in our sample and identified the specific nature of their jobs. As seen in Figure 2, less than one-third of these TFA alumni are in administrative positions like operations or finance. Most are involved in human resources, such as hiring and training teachers or other support staff; academic affairs, such as developing curriculum for instructional programs or schools; or working to develop new schools or expand existing ones. This first cut suggests that entrepreneurial TFA alumni disproportionately take on roles more closely related to instruction and staffing. As mentioned earlier, it is not uncommon for TMT members in operations and finance to have consulting experience in their professional backgrounds.

Looking Ahead

The findings presented here on the leadership pipeline signal many avenues for productive future inquiry. First, TFA specifically sets out to recruit individuals with leadership potential. As TFA explains, it seeks college graduates who have demonstrated “past leadership and achievement…perseverance and sustained focus in the face of challenges, strong critical thinking skills…[an ability to generate] relevant solutions to problems, superior organizational ability…and superior interpersonal skills to motivate and lead others.” The TFA selection process consists of an online application, a phone interview, and a final interview, which includes multiple individual and group activities, plus a personal interview. Sorting out the impact of TFA acculturation and training from its success as a talent identifier will require additional research that examines the alumni’s career expectations and decisions over time, with an eye to their experiences during and after their corps engagement with TFA.

Second, we found that certain of TFA’s geographic regions appeared more likely to generate entrepreneurial behavior. TFA corps members with work experience in New York City and San Francisco seemed especially likely to become top managers in entrepreneurial organizations in education. Perhaps there is something distinctive about the TFA experience in these locales. Maybe, and more likely, there was something about the place at that particular time that worked in concert with the TFA experience to produce entrepreneurial leaders in a particularly effective way.

In the 1990s, when many of the entrepreneurial organizations in our list were being founded, San Francisco, and the Silicon Valley more generally, was a hotbed of entrepreneurial behavior, which included unprecedented levels of capital funding for those wanting to start their own ventures. At the same time, New York City, with its similar culture of entrepreneurialism and capital funding activity, was going through a period of political and educational reform that would lead to the era of mayoral control. This period of fl ux created opportunities for new organizations and programs to enter the education market. The combination of an entrepreneurial culture, access to funding, and openings within the education market may have made these cities particularly conducive to TFA’s mission of creating entrepreneurial leaders; indeed, the two cities were among the first to bring TFA teachers into their schools. Therefore, it may be useful to think about the TFA experience more expansively and with an eye to its place within a larger context of reform and opportunity.

Third, working for TFA at the national level appears to be a more common experience for those who end up working for an entrepreneurial organization, rather than founding one; TFA members who were founders of organizations were more likely to have been TFA corps members. This suggests that different TFA experiences may equip alumni for different roles. It raises a variety of important questions, most notably, what it is about the TFA experience that imparts to individuals the skills and desire to tackle certain challenges.

Certain kinds of organizations, such as TFA and KIPP, and professions like consulting may be especially conducive to producing educational entrepreneurs. It is worth asking whether there are particular jobs, roles, or work environments that contribute to the cultivation of entrepreneurial behavior.

Finally, our research suggests the value of rethinking how TFA and its alumni have been studied in education and also how we think about retention. Rather than assume that it is good or bad when TFA members leave classrooms or school systems, we focused on the role that TFA alumni may play in launching entrepreneurial ventures. While TFA members may not be retained as teachers, the findings suggest they may still have an impact in education, perhaps an outsized impact.

Another intriguing question is how to weigh the impact of a single Mike Feinberg, Mike Johnston, or Michelle Rhee. Is their impact equal to that of having 100 teachers stay another year? Of 1,000 teachers staying another five years? Is it worth having thousands frequently depart classrooms if it increases the likelihood that a single game-changing entrepreneur—a Steve Jobs or a Bill Gates—will emerge? Conventional debates about retention and TFA teacher effects may start to seem trivial when we compare the potentially enormous impact of a few such individuals.

Monica Higgins is professor of education at the Harvard Graduate School of Education and author of Career Imprints: Creating Leaders Across an Industry. Frederick Hess is an executive editor of Education Next and author or editor of several books, including Education Unbound and Educational Entrepreneurship. Jennie Weiner and Wendy Robison are doctoral students in Education Policy, Leadership, and Instructional Practice at the Harvard Graduate School of Education.

The ubiquity of “satisfactory” ratings stands in contrast to a rapidly growing body of research that examines differences in teachers’ effectiveness at raising student achievement. In recent years, school districts and states have compiled datasets that make it possible to track the achievement of individual students from one year to the next, and to compare the progress made by similar students assigned to different teachers. Careful statistical analysis of these new datasets confirms the long-held intuition of most teachers, students, and parents: teachers vary substantially in their ability to promote student achievement growth.

The quantification of differences has generated a flurry of policy proposals to promote teacher quality over the past decade, and the Obama administration’s recent Race to the Top program only accelerated interest. Yet, so far, little has changed in the way that teachers are evaluated, in the content of pre-service training, or in the types of professional development offered. A primary stumbling block has been a lack of agreement on how best to identify and measure effective teaching.

A handful of school districts and states—including Dallas, Houston, Denver, New York, and Washington, D.C.—have begun using student achievement gains as indicated by annual test scores (adjusted for prior achievement and other student characteristics) as a direct measure of individual teacher performance. These student-test-based measures are often referred to as “value-added” measures. Yet even supporters of policies that make use of value-added measures recognize the limitations of those measures. Among the limitations are, first, that these performance measures can only be generated in the handful of grades and subjects in which there is mandated annual testing. Roughly one-quarter of K–12 teachers typically teach in grades and subjects where obtaining such measures is currently possible. Second, test-based measures by themselves offer little guidance for redesigning teacher training or targeting professional development; they allow one to identify particularly effective teachers, but not to determine the specific practices responsible for their success. Third, there is the danger that a reliance on test-based measures will lead teachers to focus narrowly on test-taking skills at the cost of more valuable academic content, especially if administrators do not provide them with clear and proven ways to improve their practice.

Student-test-based measures of teacher performance are receiving increasing attention in part because there are, as yet, few complementary or alternative measures that can provide reliable and valid information on the effectiveness of a teacher’s classroom practice. The approach most commonly in use is to evaluate effectiveness through direct observation of teachers in the act of teaching. But as “The Widget Effect” reports, such evaluations are a largely perfunctory exercise.

In this article, we report a few results from an ongoing study of teacher classroom observation in the Cincinnati Public Schools. The motivating research question was whether classroom observations—when performed by trained professionals external to the school, using an extensive set of standards—could identify teaching practices likely to raise achievement.

We find that evaluations based on well-executed classroom observations do identify effective teachers and teaching practices. Teachers’ scores on the classroom observation components of Cincinnati’s evaluation system reliably predict the achievement gains made by their students in both math and reading. These findings support the idea that teacher evaluation systems need not be based on test scores alone in order to provide useful information about which teachers are most effective in raising student achievement.

The Cincinnati Evaluation System

Jointly developed by the local teachers union and district more than a decade ago, the Cincinnati Public Schools’ Teacher Evaluation System (TES) is often cited as a rare example of a high-quality evaluation program based on classroom observations. At a minimum, it is a system to which the district has devoted considerable resources. During the yearlong TES process, teachers are typically observed and scored four times: three times by a peer evaluator external to the school and once by a local school administrator. The peer evaluators are experienced classroom teachers chosen partly based on their own TES performance. They serve as full-time evaluators for three years before they return to the classroom. Both peer evaluators and administrators must complete an intensive training course and accurately score videotaped teaching examples.

The system requires that all new teachers participate in TES during their first year in the district, again to receive tenure (usually in their fourth year), and every fifth year thereafter. Teachers tenured before 2000–01 were gradually phased into the five-year rotation. Additionally, teachers may volunteer to be evaluated; most volunteers do so to post the high scores necessary to apply for selective positions in the district (for example, lead teacher or TES evaluator).

The TES scoring rubric used by the evaluators, which is based on the work of educator Charlotte Danielson, describes the practices, skills, and characteristics that effective teachers should possess and employ. We focus our analysis on the two (out of four total) domains of TES evaluations that directly address classroom practices: “Creating an Environment for Student Learning” and “Teaching for Student Learning.” (The other two TES domains assess teachers’ planning and professional contributions outside of the classroom; scores in these areas are based on lesson plans and other documents included in a portfolio reviewed by evaluators.) These two domains, with scores based on classroom observations, contain more than two dozen specific elements of practice that are grouped into eight “standards” of teaching. Table 1 provides an example of two elements that comprise one standard. For each element, the rubric provides language describing what performance looks like at each scoring level: Distinguished (a score of 4), Proficient (3), Basic (2), or Unsatisfactory (1).

Data and Methodology

Cincinnati provided us with records of each classroom observation conducted between the 2000–01 and 2008–09 school years, including the scores that evaluators assigned for each specific practice element as a result of that observation. Using these data, we calculated a score for each teacher on the eight TES “standards” by averaging the ratings assigned during the different observations of that teacher in a given year on each element included under the standard. We then collapsed these eight standard-level scores into three summary indexes that measure different aspects of a teacher’s practice:

• The first, which we call Overall Classroom Practices, is simply the teacher’s average score across all eight standards. This index captures the general importance of the full set of teaching practices measured by the evaluation.

• The second, Classroom Management vs. Instructional Practices, measures the difference in a teacher’s rating on standards that evaluate classroom management and that same teacher’s rating on standards that assess instructional practices. A teacher who is more skilled at managing the classroom environment, as compared to her ability to engage in desired instructional activities, will receive a higher score on this index than a teacher who engages in these instructional practices but who is less skilled at managing the classroom.

• The third, Questions/Discussion vs. Standards/Content, measures the difference between a teacher’s rating on a single standard that evaluates the use of questions and classroom discussion as an instructional strategy, and that same teacher’s average rating on three standards that assess teaching practices that focus on classroom management routines, on conveying standards-based instructional objectives to students, and on demonstrating content-specific knowledge in teaching these objectives.

Our main analysis below examines the degree to which these summary indices predict a teacher’s effectiveness in raising student achievement. Note, however, that we did not construct the indices based on any hypotheses of our own about which aspects of teaching practice measured by TES were most likely to influence student achievement. Rather, we used a statistical technique known as principal components analysis, which identifies the smaller number of underlying constructs that the eight different dimensions of practice are trying to capture. As it turns out, scores on these three indices explain 87 percent of the total variation in teacher performance across all eight standards.

For all teachers in our sample, the average score on the Overall Classroom Practices index was 3.21, or between the “Proficient” and “Distinguished” categories. Yet one-quarter of teachers received an overall score higher than 3.53 and one-quarter received a score lower than 2.94. In other words, despite the fact that TES evaluators tended to assign relatively high scores on average, there is a fair amount of variation from teacher to teacher that we can use to examine the relationship between TES ratings and classroom effectiveness.

In addition to TES observation results, Cincinnati provided student data for the 2003–04 through 2008–09 school years, including information on each student’s gender, race/ethnicity, English proficiency status, participation in special education or gifted and talented programs, class and teacher assignments by subject, and state test scores in math and reading. This rich dataset allows us to study students’ math and reading test-score growth from year to year in grades four through eight (where end of year and prior year tests are available), while also taking account of differences in student backgrounds.

Our primary goal was to examine the relationship between teachers’ TES ratings and their assigned students’ test-score growth. This task is complicated, however, by the possibility that factors not measured in our data, such as the level of social cohesion among the students or unmeasured differences in parental engagement, could independently affect both a TES observer’s rating and student achievement. To address this concern, we use observations of student achievement from teachers’ classes in the one or two school years prior to and following TES measurement, but we do not use student achievement gains from the year in which the observations were conducted. (If some teachers are assigned particularly engaged or cohesive classrooms year after year, the results could still be biased; this approach, however, does eliminate bias due to year-to-year differences in unmeasured classroom traits being related to classroom observation scores.)

We restrict our comparisons to teachers and students within the same schools in order to eliminate any potential influence of differences between schools on both TES ratings and student achievement. In other words, we ask whether teachers who receive higher TES ratings than other teachers in their school produce larger gains in student achievement than their same-school colleagues.

Results

We find that teachers’ classroom practices, as measured by TES scores, do predict differences in student achievement growth. Our main results, which are based on a sample of 365 teachers in reading and 200 teachers in math, indicate that improving a teacher’s Overall Classroom Practices score by one point (e.g., moving from an overall rating of “Proficient” [3] to “Distinguished” [4]) is associated with one-seventh of a standard deviation increase in reading achievement, and one-tenth of a standard deviation increase in math (see Figure 1).

The specific point system that TES uses to rate teachers as Proficient and Distinguished is somewhat arbitrary. For a better sense of the magnitude of these estimates, consider a student who begins the year at the 50th percentile and is assigned to a top-quartile teacher as measured by the Overall Classroom Practices score; by the end of the school year, that student, on average, will score about three percentile points higher in reading and about two points higher in math than a peer who began the year at the same achievement level but was assigned to a bottom-quartile teacher.

This difference might not seem large but, of course, a teacher is just one influence on student achievement scores (and classroom observations are only one way to assess the quality of a teacher’s instruction). By way of comparison, we can estimate the total effect a given teacher has on her students’ achievement growth; that total effect includes the practices measured by the TES process along with everything else a teacher does. The difference between being taught by a top-quartile total-effect teacher versus a bottom-quartile total-effect teacher would be about seven percentile points in reading and about six points in math (see Figure 2). This total-effect measure is one example of the kind of “value-added” approach taken in current policy proposals.

From these data, we can also discern relationships between more specific teaching practices and student outcomes across academic subjects (see Figure 1). Among students assigned to different teachers with the same Overall Classroom Practices score, math achievement will grow more for students whose teacher is better than his peers at classroom management (i.e., has a higher score on our Classroom Management vs. Instructional Practices measure). We also find that reading scores increase more among students whose teacher is relatively better than his peers at engaging students in questioning and discussion (i.e., has a high score on Questions/Discussion vs. Standards/Content). This does not mean, however, that students’ math achievement would rise if their teachers were to become worse at a few carefully selected instructional practices. Although this might raise their Classroom Environment vs. Instructional Practices score it would also lower the Overall Classroom Practices score, and any real teacher is the combination of these three scores.

Do these statistics provide any insight that teachers can use to focus their efforts? First, our finding that Overall Classroom Practices is the strongest predictor of student achievement in both subjects indicates that improved practice in any of the areas considered in the TES process should be encouraged. In other words, the practices captured by the TES rubric do predict better outcomes for students. If, however, teachers must choose a smaller number of practices on which to focus their improvement efforts (for example, because of limited time or professional development opportunities), our results suggest that math achievement would likely benefit most from improvements in classroom management skills before turning to instructional issues. Meanwhile, reading achievement would benefit most from time spent improving the practice of asking thought-provoking questions and engaging students in discussion.

Can we be confident that the various elements of practice measured by TES are the reasons that students assigned to highly rated teachers make larger achievement gains? Skeptical readers may worry that better teachers engage in more of the practices encouraged by TES, but that these practices are not what make the teacher more effective. To address this concern, we take advantage of the fact that some teachers were evaluated by TES multiple times. For these teachers, we can test whether improvement over time in the practices measured by TES is related to improvement in the achievement gains made by the teachers’ students. This is exactly what we find. Since this exercise compares each teacher only to his own prior performance, we can be more confident that it is differences in the use of the TES practices themselves that promote student achievement growth, not just the teachers who employ these strategies.

Conclusion

Is TES worth the considerable effort and cost? Does the intensive TES process (with its multiple observations and trained peer evaluators) produce more accurate information on teachers’ effectiveness in raising student achievement gains than do more-subjective evaluations? In fact, studies of informal surveys of principals (see “When Principals Rate Teachers,” research, Spring 2006) and teacher ratings by mentor teachers find that these more-subjective evaluation methods have similar power to detect differences in teacher effectiveness as the TES ratings. These studies may lead some to question the need for the more detailed TES process. We contend, however, that evaluations based on observations of classroom practice are valuable, even if they do not predict student achievement gains considerably better than more subjective methods like principal ratings of teachers.

The additional information the TES system provides can be used in several important ways. First, the data gleaned from the observations allow researchers to connect specific teaching practices with student achievement outcomes, providing evidence of effective teaching practices that can be widely shared.

The TES program also has the advantage of furnishing teachers and administrators with details about the specific practices that contributed to each teacher’s score. The descriptions of practices, and different performance levels for each practice, that comprise the TES rubric can help teachers and administrators map out professional development plans. A school administrator who desires to differentiate the support she provides to individual teachers would benefit from knowing the components of each teacher’s overall scores. A teacher who would like to improve his classroom management skills may find that he has scored relatively low in a particular standard, and then take steps to improve his practice in response to that information.

Finally, scoring individual practices allows for understanding of more fine-grained variations in skill among teachers with similar overall ratings. It is notable, especially given “The Widget Effect” study, that nearly 90 percent of teachers in our sample received an overall “Satisfactory” rating (i.e., “Distinguished” or “Proficient” in Cincinnati’s terms). Still, there are readily discernible differences in mastery of specific skills within that 90 percent, and those differences in skills predict differences in student achievement.

There are other aspects of the Cincinnati system that may or may not account for the results we observed. First, the observers were external to the school and, in most cases, had no personal relationship with the person they were observing. Second, the observers were trained beforehand and were required to demonstrate their ability to score some sample videos in a manner consistent with expert scores. Simply handing principals a checklist with the same set of standards may not lead to a similar outcome.

The results presented here constitute the strongest evidence to date on the relationship between teachers’ observed classroom practices and the achievement gains made by their students. The nature of the relationship between practices and achievement supports teacher evaluation and development systems that make use of multiple measures. Even if one is solely interested in raising student achievement, effectiveness measures based on classroom practice provide critical information to teachers and administrators on what actions they can take to achieve this goal.

Thomas J. Kane is professor of education and economics at the Harvard Graduate School of Education. Eric S. Taylor is a doctoral student at the Stanford University School of Education. John H. Tyler is associate professor of education, economics, and public policy at Brown University. Amy L. Wooten is a doctoral student at the Harvard Graduate School of Education. Reflecting equal contributions to this work, authors are listed alphabetically. This article is based in part on a larger study which is forthcoming in the Journal of Human Resources.

Opinion: In an Ed Week commentary, Eric Hanushek discusses some policy implications of his findings about the impact of good and bad teachers.

For some time, we have recognized that the academic achievement of schoolchildren in this country threatens, to borrow President Barack Obama’s words, “the U.S.’s role as an engine of scientific discovery” and ultimately its success in the global economy. The low achievement of American students, as reflected in the Program for International Student Assessment (PISA) (see “Teaching Math to the Talented,” features, Winter 2011), will prevent them from accessing good, high-paying jobs. And, as demonstrated in another article in Education Next (see “Education and Economic Growth,” research, Spring 2008), lower achievement means slower growth in the economy. From studying the historical relationship, we can estimate that closing just half of the performance gap with Finland, one of the top international performers in terms of student achievement, could add more than $50 trillion to our gross domestic product between 2010 and 2090. By way of comparison, the drop in economic output over the course of the last recession is believed to be less than $3 trillion. Thus the achievement gap between the U.S. and the world’s top-performing countries can be said to be causing the equivalent of a permanent recession.

According to the president in this year’s State of the Union address, this is “our generation’s Sputnik moment,” the time when we realize the urgent need to step up the performance of our education system. Only today, unlike in the 1950s, we have a clear idea of what it takes to improve achievement. The quality of the teachers in our schools is paramount: no other measured aspect of schools is nearly as important in determining student achievement. The initiatives we have emphasized in policy discussions—class-size reduction, curriculum revamping, reorganization of school schedule, investment in technology—all fall far short of the impact that good teachers can have in the classroom. Moreover, many of these interventions can be very costly.

Indeed, the magnitude of variation in the quality of teachers, even within each school, is startling. Teachers who work in a given school, and therefore teach students with similar demographic characteristics, can be responsible for increases in math and reading levels that range from a low of one-half year to a high of one and a half years of learning each academic year.

But while most parents are able to distinguish a good teacher from a bad one, few have any idea what difference it makes in the lives of their children. And researchers do not help, tending to talk in terms of standard deviations of achievement and effect sizes, phrases that simply have no meaning outside of the rarefied world of research. Here, I translate the researchers’ shorthand into concepts that might be more readily understood: the impact of teachers on the earnings of individuals and on the future of the economy as a whole.

Measuring Teachers’ Impact

Many of us have had at some point in our lives a wonderful teacher, one whose value, in retrospect, seems inestimable. We do not pretend here to know how to calculate the life-transforming effects that such teachers can have with particular students. But we can calculate more prosaic economic values related to effective teaching, by drawing on a research literature that provides surprisingly precise estimates of the impact of student achievement levels on their lifetime earnings and by combining this with estimated impacts of more-effective teachers on student achievement.

Let’s start with the researcher’s point of view. With a normal distribution of performance (the classic bell curve), a standard deviation is simply a more precise measure of how spread out the distribution is. Somebody who is one standard deviation above average would be at the 84th percentile of the distribution. If we then turn to the labor market, a student with achievement (as measured by test performance in high school) that is one standard deviation above average can later in life expect to take in 10 to 15 percent higher earnings per year.

That estimate may be deemed conservative for two reasons. First, it does not account for increases in years of education that may result from having a higher level of performance early on. Also, the estimate is based on information from people’s wages and salaries early in their careers, before they have reached their full earnings potential. Other calculations that take into account earnings throughout entire careers estimate 20 percent increases over the course of a lifetime.

Does 10 to 15 percent amount to much? For the average American entering the labor force, the value of lifetime earnings for full-time work is currently $1.16 million. Thus, an increase in the level of achievement in high school of a standard deviation yields an average increase of between $110,000 and $230,000 in lifetime earnings.

How do increases in teacher effectiveness relate to this? Obviously, teacher quality is not the only factor that affects student achievement. The student’s own motivations and support from family and peers play crucial roles as well. But researchers have worked hard to isolate the impact of teachers from these other influences. Rigorous studies consistently show that the impact of a more-effective teacher is substantial A high-performing teacher, one at the 84th percentile of all teachers, when compared with just an average teacher, produces students whose level of achievement is at least 0.2 standard deviations higher by the end of the school year. In fact, the impact of having such a teacher could plausibly be as large as 0.3 standard deviations.

Those impacts attenuate somewhat over time, however. The literature, though less than definitive, suggests that perhaps 70 percent of the gains achieved that year are retained in the long run by the student. The persistence of achievement gains is important, because the more sustained that these increases are, the greater the positive impact teachers will have on the lifetime skills and therefore the earnings of students. Put together, this evidence suggests that a teacher in the top 16 percent of effectiveness will have a positive impact (as compared to an average teacher) on longer-term student achievement that is 70 percent of the immediate gain, which as noted is at least 0.2 standard deviations.  That lower bound of the estimated effect is what we will use as we calculate the economic worth of a teacher by combining a teacher’s impact on achievement with the associated labor market returns.

Let’s start with some conservative estimates of the impact on an individual student. Take a good but not great teacher, one at the 69th percentile of all teachers rather than at the 50th percentile (that is, a teacher who is half a standard deviation above the average). She produces an increase of $10,600 on each student’s lifetime earnings. Even a modestly better than average teacher (60th percentile) raises individual earnings by $5,300, compared to what would otherwise be expected.

While those numbers are not trivial, they burgeon dramatically once we recognize that every student in the class can expect such increases in earnings. Consider, for example, a teacher with a class of 20 students. Under such circumstances, the teacher at the 60th percentile will—each year—raise students’ aggregate earnings by a total of $106,000. The impact of one at the 69th percentile (as compared to the average) is $212,000, and one at the 84th percentile will shift earnings up by more than $400,000.

But there is also symmetry to these calculations. A very low performing teacher (at the 16th percentile of effectiveness) will have a negative impact of $400,000 compared to an average teacher.

Moreover, the economic value of an effective teacher grows with larger classes, as do the economic losses of an ineffective teacher. Figure 1 illustrates the aggregate impact on students’ lifetime earnings for higher- and lower-performing teachers. As we will discuss below, these results are all very large compared with, for instance, the $52,000 annual salary U.S. teachers were paid on average in 2008.

An Alternate Thought Experiment

We can also approach this valuation calculation from the perspective of the impact of teacher effectiveness on the U.S. economy as a whole, rather than just on the future earnings of students. As noted above, student achievement, which provides a direct measure of later quality of the labor force, is strongly related to economic growth. Improving achievement leads to a better prepared workforce and to greater growth, and this growth translates into higher levels of national income.

Starting again with the estimates of the difference in effectiveness of teachers, it is possible to calculate the long-term economic impact of policies that would focus attention on the lowest-quality teachers from U.S. classrooms. Let us propose the following thought experiment: What would happen if the very lowest performing teachers could be replaced by just average teachers? Based on the estimates of variation in teacher quality identified above, Figure 2 shows the overall achievement impact through a cycle of K–12 instruction. Assuming the upper-bound estimate of teachers’ impact, U.S achievement could reach that in Canada and Finland if we replaced with average teachers the least effective 5 to 7 percent of teachers, respectively. Assuming the lower-bound estimate of teachers’ impact, U.S achievement could reach that in Canada and Finland if we replaced with average teachers the least effective 8 to 12 percent of teachers, respectively.

Here the estimated value almost loses any meaning. Closing the achievement gap with Finland would, according to historical experience, have astounding benefits, increasing the annual growth rate of the United States by 1 percent of GDP. Accumulated over the lifetime of somebody born today, this improvement in achievement would amount to nothing less than an increase in total U.S. economic output of $112 trillion in present value. (That was not a typo—$112 trillion, not billion.)

Admittedly, these estimates are subject to some uncertainty. So if you think those that are given here are too high, even though they are based on the best of contemporary research, then just cut them in half. You will still have effects on growth of one-half of 1 percent per year, which produces impacts of $56 trillion over the lifetime of today’s child. In other words, to make the very large effects disappear, you have to make either the very strong assumption that student learning has little effect on the U.S. economy or the equally strong assumption that teachers have little impact on students.

What Would It Take?

The majority of our teachers are hardworking and effective. The previous estimates point clearly to the key imperative of eliminating the drag of the bottom teachers. Here we can offer several alternatives.

One approach might be better recruitment so that ineffective or poor teachers do not make it into our schools. Or, relatedly, we could improve the training in schools of education so that the average teaching recruit is better than the typical recruit of today. Unfortunately, we have relatively few successful experiences with either approach as compared to considerable wishful thinking, particularly among school personnel.

An alternative might be to change a poor teacher into an average teacher. This approach is in fact today’s dominant strategy. Schools hope that through mentoring of incoming teachers, professional development, or completion of further graduate schooling, ineffective teachers can be transformed into acceptable (average) teachers. Again, however, the existing evidence is not very reassuring. While such efforts undoubtedly help some teachers, there is no substantial evidence that certification, in-service training, master’s degrees, or mentoring programs systematically make a difference in whether teachers are in fact effective at driving student achievement.

The final option is a clearer evaluation and retention strategy for teachers. Today, obtaining an entry job into teaching is virtually tantamount to an indefinite contract that stays in force regardless of actual effectiveness in the classroom. Yet the calculations above show the enormous value to individuals and society of “deselecting” the least effective teachers.

Is such a policy change feasible? If we contemplate asking 5 to 10 percent of teachers to find a job at which they are more effective so they can be replaced by teachers of average productivity, states and school districts would have to change their employment practices. They would need recruitment, pay, and retention policies that allow for the identification and compensation of teachers on the basis of their effectiveness with students. At a minimum, the current dysfunctional teacher-evaluation systems would need to be overhauled so that effectiveness in the classroom is clearly identified. This is not an impossible task. The teachers who are excellent would have to be paid much more, both to compensate for the new riskiness of the profession and to increase the chances of retaining these individuals in teaching. Those who are ineffective would have to be identified and replaced. Both steps would be politically challenging in a heavily unionized environment such as the one in place today.

Salary Politics

The above discussion also highlights the difficulties in recruiting high-quality teachers, due in part to the difficulties of paying them well. Collective bargaining mechanisms do not provide incentives for the best people to enter or remain in the profession and likely hold the average pay down: given the uniform salary structure, increases in salary are bound to be unrelated to increases in effectiveness, making large pay raises raises politically problematic. This is likely one of the main reasons that teacher salaries now lag those in other professions. In the 1940s, the salaries of male teachers were slightly above the average pay for all male college graduates, and female teachers had higher salaries than 70 percent of other female college graduates. Today, despite the collective bargaining process, the salaries of male teachers are at the 30th percentile of the distribution of all college graduates, and women who teach are at the 40th percentile of their college-educated peers.

Teachers’ salaries today are based on credentials and years of experience, factors that are at best weakly related to productivity. In a competitive marketplace, a firm must compensate employees according to their productivity or risk bankruptcy. Yet no school district goes out of business if it retains ineffective teachers and pays them as much as effective ones. Salaries become political footballs, and it is often awkward for politicians to explain why a large pay increase goes equally to ineffective and effective teachers.

The challenge of implementing reform of the teaching profession remains considerable. Most of the benefits of implementing the “thought experiment” explored here would be fully realized only many decades later, while the costs of economic, and especially political, reform must be paid at the beginning. These costs would be steep, as they would likely negatively affect some of the most vocal constituents in education policy: current teachers.

The magnitude of the above valuations of teacher effectiveness, however, suggest that we should be willing to consider more radical reforms than have been commonplace in recent decades. Salaries several times higher than those paid teachers today would be economically justified if teachers were compensated according to their effectiveness. But unless we can replace the current system with one that better links teacher recruitment, compensation, and retention to effectiveness, we should expect both our schools and our economy to underperform relative to their potential. The cost to the nation at a time of intensifying international competition is high indeed.

Eric A. Hanushek is a senior fellow at the Hoover Institution, Stanford University.

Let me admit to a prejudice: I have known John Merrow for more than a quarter of a century and greatly admire his single-minded interest in national school reform. Having said this, let me say that in all candor I liked the second half of this book, particularly the third section, better than the earlier chapters, which are, by the author’s own admission, recapitulations of interviews on NPR and PBS.

Not that Merrow does not cover the ground effectively. He deals with many of the hot-button issues by drawing on his own experiences as a student at the Taft School and his subsequent encounters in the less privileged world of urban education. Teacher evaluation he rightly insists is a complex issue (much more so than exclusively linking teacher education to results from standardized tests). Misassignment of teachers (i.e. teachers teaching out of their field or academic discipline) does not escape his attention. Teacher tenure and the mediocrity the tenure system Merrow rightly identifies as a significant hindrance to genuine reform. He underscores the importance of safe schools and early reading programs. He is astute in his approval of the charter school movement under certain circumstances. He contrasts school reformers like Michelle Rhee and Paul Vallas. He rightly identifies the success of Teach for America as a comment on the comparative mediocrity of so many other teachers trained in traditional teacher colleges. All this is good stuff, even if we have heard about most of it from other sources.

What is fresh, however, are his recommendations in his concluding chapter. After reminding us that two competing views are vying for national attention, “mediocre teachers are the heart of the problem” or “is it the job itself with its low pay, and even lower prestige,” he constructs a series of recommendation all of which are very much to the point.

Teaching would be a better job (1) When principals have authority over hiring their staff. (2) When teacher evaluations of students count as least as much as the score on a one-time standardized test (3) When employment contracts are not for life and employee evaluations are fair and thorough. (4) When everybody’s pay depends in part on the performance of students academically. (5) When teachers get students to ask good questions and not merely regurgitate answers to pre-constructed questions.

While all this may sound to the uninitiated as a litany of the obvious, in fact such changes would be revolutionary and are desperately needed. Thank you, John Merrow, for reminding us.

-A. Graham Down

You can find more book reviews by Graham Down here.