As impressive and amazing as these results by charter schools may be, it would be wrong to conclude from this that charter schools improve student achievement.  The only way to know with confidence whether charters cause better outcomes is to look at randomized control trials (RCTs) in which students are assigned by lottery to attending a charter school or a traditional public school.  RCTs are like medical experiments where some subjects by chance get the treatment and others by chance do not.  Since the two groups are on average identical, any difference observed in later outcomes can be attributed to the “treatment,” and not to some pre-existing and uncontrolled difference.  We demand this type of evidence before we approve any drug, but the evidence used to justify how our children are educated is usually nowhere near as rigorous.

Happily, we have four RCTs on the effects of charter schools that allow us to know something about the effects of charter schools with high confidence.  Here is what we know:  students in urban areas do significantly better in school if they attend a charter schools than if they attend a traditional public school.  These academic benefits of urban charter schools are quite large. 

In Boston, a team of researchers from MIT, Harvard, Duke, and the University of Michigan, conducted a RCT and found:  “The charter school effects reported here are therefore large enough to reduce the black-white reading gap in middle school by two-thirds.”

A RCT of charter schools in New York City by a Stanford researcher found an even larger effect: “On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the ‘Scarsdale-Harlem achievement gap’ in math and 66 percent of the achievement gap in English.”

The same Stanford researcher conducted an RCT of charter schools in Chicago and found:  “students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading…. To put the gains in perspective, it may help to know that 5 to 6 percentile points is just under half of the gap between the average disadvantaged, minority student in Chicago public schools and the average middle-income, nonminority student in a suburban district.”

And the last RCT was a national study conducted by researchers at Mathematica for the US Department of Education.  It found significant gains for disadvantaged students in charter schools but the opposite for wealthy suburban students in charter schools.  They could not determine why the benefits of charters were found only in urban, disadvantaged settings, but their findings are consistent with the three other RCTs that found significant achievement gains for charter students in Boston, Chicago, and New York City.

When you have four RCTs – studies meeting the gold standard of research design – and all four of them agree that charters are of enormous benefit to urban students, you would think everyone would agree that charters should be expanded and supported, at least in urban areas.  If we found the equivalent of halving the black-white test score gap from RCTs from a new cancer drug, everyone would be jumping for joy – even if the benefits were found only for certain types of cancer.

Unfortunately, many people’s views on charter schools are heavily influenced by their political and financial interests rather than the most rigorous evidence.  They don’t want to believe the findings of the four RCTs, so they simply ignore them or cite studies with inferior research designs in which we should have much less confidence.

Progress will be made in our application of research to charter school policies by encouraging everyone to focus on the most rigorous studies, of which we have several.  To do that, supporters of charter schools also have to refrain from citing weaker evidence, which only serves to legitimize the use of inferior studies by charter opponents.  As exciting as the outstanding performance of charter schools is in my own Global Report Card research, that evidence shouldn’t be used to endorse charter schools.  Supporters don’t need to rely on the Global Report Card to make the case for charter schools because they have four gold-standard RCTs on their side.  Opponents of charter schools have no equally rigorous evidence on their side.  And that’s the point we should all be making.

-Jay P. Greene

This blog entry originally appeared on the blog of the George W. Bush Institute for National Charter Schools Week.

The fundamental problem is that “best practices” analyses lack variation in the dependent variable – they only examine in detail successful organizations or countries – so they can’t link particular practices or policies to success.  To make such a link they would need to observe that the presence or absence of those practices or policies is related to the presence or absence of success.  If they only look at successful organizations, then they can’t know whether they would have been less (or more) successful had they not adopted a particular policy or practice.  They also do not rule out the possibility that others who have adopted the “best practices” do so without success.

But Tucker claims that he didn’t only look at successful countries because “the strategy we used was to compare the top performing countries to the United States.”  Making (mostly implicit) comparisons to the United States does not solve the problem.  Again, without considering a broad spectrum of successful and unsuccessful countries it is impossible to attribute the superior performance of another country to any particular policy or practice.

There are many things that are different between the U.S. and Shanghai, Finland, Japan, Singapore, and Canada.  How can Tucker or anyone know which differences caused the superior performance?  Tucker just picks and chooses the policies and practices he favors, ignoring that his recommendations are not even universally present in the handful of successful places he examines.  And by limiting variation in the dependent variable to exclude places that perform worse than the United States, Tucker is unable to discover whether lower-achieving countries are also employing the practices and policies he recommends, which would debunk his claim of having found the formula for success.

I’m far from being the only one who is aware of the problems with Tucker’s method of “selection on the dependent variable.”  Virtually every introductory text on research design warns readers not to do as Tucker and other best practices enthusiasts do when they focus only on successful organizations or countries.  For example, Gary King, Robert Keohane, and Sidney Verba, in their classic Designing Social Inquiry, make the point emphatically:

That brings us to a basic and obvious rule: selection should allow for the possibility of at least some variation on the dependent variable. This point seems so obvious that we would think it hardly needs to be mentioned. How can we explain variations on a dependent variable if it does not vary? Unfortunately, the literature is full of work that makes just this mistake of failing to let the dependent variable vary…. The cases of extreme selection bias—where there is by design no variation on the dependent variable—are easy to deal with: avoid them! We will not learn about causal effects from them.

In my review I recommend analyses of international policies and practices done by Eric Hanushek, Ludger Woessmann, Martin West, Michael Kremer, Karthik Muralidharan and Charles Glenn because, unlike Tucker and other “best practices” gurus,  they avoid the error of selection on the dependent variable by considering the full range of outcomes, not just focusing on successful places.

Tucker is apparently unable to understand the difference between what he and these reputable researchers do when he mistakenly declares:

Greene appears to realize that his war on “best practices” has led him to inadvertently attack the kinds of studies done by people whose policy prescriptions he prefers, like Ludger Woessmann and Eric Hanushek, who have done well-regarded statistical analyses of survey data from OECD-PISA and other sources…. So, in the end, all the methods we used meet with Jay Greene’s approval.  It is only our conclusions that are odious.

Tucker’s inability to understand the difference and his dismissal of the selection on dependent variable criticism as “highfalutin language” is just plain embarrassing.  It’s not so much embarrassing for him, since he appears to be proud in his ignorance, as it is embarrassing for the Gates Foundation that pays for his work and the supporters of Common Core who rely on Tucker as one of their principal architects and advocates.

There is a cynical habit in the education policy world to fund and promote analyses that people know or should know to be faulty as long as those analyses advance their cause.  Shaming those who engage in this cynical practice by revealing the obvious flaws in Tucker’s work was the purpose of my review.  I fear that it will not end the use of “best practices” in education, but I hope it will exact a price for those who engage in such hucksterism.

-Jay Greene

Even though the Miami New Times article was just a re-hash of an article they had run during the summer before, critics of special ed vouchers seized upon the piece as proof of the need to stop the rapid expansion of that type of program to other states, impose heavy regulations on Florida’s program to ensure that nothing bad could ever happen, or just shut down special ed programs because only public provision of services to disabled students could be trusted.

Diane Ravitch, in her usual scholarly and measured way, responded to the article by tweeting “Legalized child abuse in Florida?” Sara Mead, Andy Rotherham, and Ed Sector all circulated the New Times piece as proof of their earlier criticisms of McKay.  When I attempted to put the scandal in perspective relative to misconduct and incompetence that is all too common in traditional public schools, Sara Mead clucked that I was like a child trying to excuse misbehavior by crying “he did it first!”

Well, I wonder if a story out of Alabama might help put things in perspective without sounding like an unreasonable child.  It’s a story about a boy named Jose Salinas, or Little Joe, who has cerebral palsy.  His mother wondered why he was acting unusually and repeatedly claiming that he couldn’t go to school because he wasn’t feeling well.  So, she decided to attached a secret audio recording device to his wheelchair to find out what was going on at school.

Here is what she discovered:

“You drooled on the paper,” teacher’s aide Drew Faircloth could be heard saying impatiently. “That’s disgusting.”

“Keep your mouth closed and don’t drool on my paper,” teacher Alicia Brown said on the tape. “I do not want to touch your drool. Do you understand that? Obviously, you don’t.”

Over the three days of recordings, Salinas said Jose received about 20 minutes of actual instruction and spent almost the entire day sitting in silence with no one speaking to him.

“I could not believe someone would treat a child that way, much less a special needs child,” Melisha Salinas told ABCNews.com. “The anger in his voices … and the thing he was getting angry about, [Jose] just can’t help.”

“Why is my paper wet?” Brown demanded. “Look at me and answer. That’s not an answer. That’s not even a word.”

“Do you seen anybody else at this table drooling? Then, stop,” she said. “You have got drool all over your face and it is gross.”

Little Joe’s mom took the recording to school officials who suspended the teachers with pay.  But within days the teachers were back working in the school, although no longer assigned to Little Joe.  Angry parents protested the return of the teachers, who were then once again placed on administrative leave with pay.

Houston County Schools superintendent Tim Pitchford helped explain:

“I made a poor decision and re-assigned them back to school,” he said. “It was the wrong decision and I accept full responsibility.”

Alabama state law does not allow superintendents to fire teachers on the spot, Pitchford said. He has to make a recommendation to the board, which makes the final decision.

“From day one, it was obvious where this was going to end with the employees,” he said. “We knew where this process was going to end, but the process does not allow it to be immediate.”

Salinas was shocked to hear the teacher and aide were back at school.

“They were back at the school and my children were there so I got them out of school and so did several angry parents,” Salinas said. “I just lost all hope. Nobody was listening to me.”

Of course, if Alabama had a special ed voucher program, like McKay, Mrs. Salinas would not have had to secretly record misconduct, prove it to school officials, and then organize a protest to ensure that those teachers were not still in the school with her son.  She could have just followed her good mother’s perception that things were going very badly and switched her child to another school with the same amount of public funding.  How many Little Joe’s are out there without having their mistreatment recorded or protests organized?

Of course, examples of misconduct in traditional public schools is no more proof of the merits of McKay-like programs than examples of misconduct are proof of the need to regulate or eliminate special ed vouchers.  For more systematic evidence on the merits of McKay, readers may wish to read the article that Marcus Winters and I published in Educational Evaluation and Policy Analysis, the leading AERA empirical journal, which finds that McKay competition increases student achievement for disabled students who remain in traditional public schools and lowers the rate at which students are newly identified as disabled.

But some people prefer mindless tweets over systematic evidence.  And somehow I don’t expect Diane Ravitch, Sara Mead, or Andy Rotherham now to tweet that Little Joe proves the wisdom of McKay or that traditional public schools are equivalent to child abuse.  They prefer to be selective in the anecdotes they tweet.

-Jay P. Greene

Surpassing Shanghai: An Agenda for American Education Built on the World’s Leading Systems

As reviewed by Jay P. Greene

“Best practices” is the worst practice. The idea that we should examine successful organizations and then imitate what they do if we also want to be successful is something that first took hold in the business world but has now unfortunately spread to the field of education. If imitation were the path to excellence, art museums would be filled with paint-by-number works.

The fundamental flaw of a “best practices” approach, as any student in a half-decent research-design course would know, is that it suffers from what is called “selection on the dependent variable.” If you only look at successful organizations, then you have no variation in the dependent variable: they all have good outcomes. When you look at the things that successful organizations are doing, you have no idea whether each one of those things caused the good outcomes, had no effect on success, or was actually an impediment that held organizations back from being even more successful. An appropriate research design would have variation in the dependent variable; some have good outcomes and some have bad ones. To identify factors that contribute to good outcomes, you would, at a minimum, want to see those factors more likely to be present where there was success and less so where there was not.

“Best practices” lacks scientific credibility, but it has been a proven path to fame and fortune for pop-management gurus like Tom Peters, with In Search of Excellence, and Jim Collins, with Good to Great. The fact that many of the “best” companies they featured subsequently went belly-up—like Atari and Wang Computers, lauded by Peters, and Circuit City and Fannie Mae, by Collins—has done nothing to impede their high-fee lecture tours. Sometimes people just want to hear a confident person with shiny teeth tell them appealing stories about the secrets to success.

With Surpassing Shanghai, Marc Tucker hopes to join the ranks of the “best practices” gurus. He, along with a few of his colleagues at the National Center on Education and the Economy, has examined the education systems in some other countries with successful outcomes so that the U.S. can become similarly successful. Tucker coauthors the chapter on Japan, as well as an introductory and two concluding chapters. Tucker’s collaborators write chapters featuring Shanghai, Finland, Singapore, and Canada. Their approach to greatness in American education, as Linda Darling-Hammond phrases it in the foreword, is to ensure that “our strategies must emulate the best of what has been accomplished in public education both from here and abroad.”

But how do we know what those best practices are? The chapters on high-achieving countries describe some of what those countries are doing, but the characteristics they feature may have nothing to do with success or may even be a hindrance to greater success. Since the authors must pick and choose what characteristics they highlight, it is also quite possible that countries have successful education systems because of factors not mentioned at all. Since there is no scientific method to identifying the critical features of success in the best-practices approach, we simply have to trust the authority of the authors that they have correctly identified the relevant factors and have properly perceived the causal relationships.

But Surpassing Shanghai is even worse than the typical best-practices work, because Tucker’s concluding chapters, in which he summarizes the common best practices and draws policy recommendations, have almost no connection to the preceding chapters on each country. That is, the case studies of Shanghai, Finland, Japan, Singapore, and Canada attempt to identify the secrets to success in each country, a dubious-enough enterprise, and then Tucker promptly ignores all of the other chapters when making his general recommendations.

Tucker does claim to be drawing on the insights of his coauthors, but he never actually references the other chapters in detail. He never names his coauthors or specifically draws on them for his conclusions. In fact, much of what Tucker claims as common lessons of what his coauthors have observed from successful countries is contradicted in chapters that appear earlier in the book. And some of the common lessons they do identify, Tucker chooses to ignore.

For example, every country case study in Surpassing Shanghai, with the exception of the one on Japan coauthored by Marc Tucker, emphasizes the importance of decentralization in producing success. In Shanghai the local school system “received permission to create its own higher education entrance examination. This heralded a trend of exam decentralization, which was key to localized curricula.” The chapter on Finland describes the importance of the decision “to devolve increasing levels of authority and responsibility for education from the Ministry of Education to municipalities and schools…. [T]here were no central initiatives that the government was trying to push through the system.” Singapore is similarly described: “Moving away from the centralized top-down system of control, schools were organized into geographic clusters and given more autonomy…. It was felt that no single accountability model could fit all schools. Each school therefore set its own goals and annually assesses its progress toward meeting them…” And the chapter on Canada teaches us that “the most striking feature of the Canadian system is its decentralization.”

Tucker makes no mention of this common decentralization theme in his conclusions and recommendations. Instead, he claims the opposite as the common lesson of successful countries: “students must all meet a common basic education standard aligned to a national or provincial curriculum… Further, in these countries, the materials prepared by textbook publishers and the publishers of supplementary materials are aligned with the national curriculum framework.” And “every high-performing country…has a unit of government that is clearly in charge of elementary and secondary education…In such countries, the ministry has an obligation to concern itself with the design of the system as a whole…”

Conversely, Tucker emphasizes that “the dominant elements of the American education reform agenda” are noticeably absent from high-performing countries, including “the use of market mechanisms, such as charter schools and vouchers….” But if Tucker had read the chapter on Shanghai, he would have found a description of a system by which “students choose schools in other neighborhoods by paying a sponsorship fee. It is the Chinese version of school choice, a hot issue in the United States.” And although the chapter on Canada fails to make any mention of it, Canada has an extensive system of school choice, offering options that vary by language and religious denomination. According to recently published research by David Card, Martin Dooley, and Abigail Payne, competition among these options is a significant contributor to academic achievement in Canada.

There is a reason that promoters of best-practices approaches are called “gurus.” Their expertise must be derived from a mystical sphere, because it cannot be based on a scientific appraisal of the evidence. Marc Tucker makes no apology for his nonscientific approach. In fact, he denounces “the clinical research model used in medical research” when assessing education policies. The problem, he explains, is that no country would consent to “randomly assigning entire national populations to the education systems of another country or to certain features of the education system of another country.” On the contrary, countries, states, and localities can and do randomly assign “certain features of the education system,” and we have learned quite a lot from that scientific process. In the international arena, Tucker may want to familiarize himself with the excellent work being done by Michael Kremer and Karthik Muralidharan utilizing random assignment around the globe.

In addition, social scientists have developed practices to observe and control for differences in the absence of random assignment that have allowed extensive and productive analyses of the effectiveness of educational practices in different countries. In particular, the recent work of Ludger Woessmann, Martin West, and Eric Hanushek has utilized the PISA and TIMSS international test results that Tucker finds so valuable, but they have done so with the scientific methods that Tucker rejects. Even well-constructed case study research, like that done by Charles Glenn, can draw useful lessons across countries. The problem with the best-practices approach is not entirely that it depends on case studies, but that by avoiding variation in the dependent variable it prevents any scientific identification of causation.

Tucker’s hostility to scientific approaches is more understandable, given that his graduate training was in theater rather than a social science. Perhaps that is also why Tucker’s book reminds me so much of The Music Man. Tucker is like “Professor” Harold Hill come to town to sell us a bill of goods. His expertise is self-appointed, and his method, the equivalent of “the think system,” is obvious quackery. And the Gates Foundation, which has for some reason backed Tucker and his organization with millions of dollars, must be playing the residents of River City, because they have bought this pitch and are pouring their savings into a band that can never play music except in a fantasy finale.

Best practices really are the worst.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute.

In 2010 I told you about how the Department of Health and Human Services delayed the release of the previous round of disappointing research results about the lasting effects of Head Start.  When the extremely high quality study, involving a random-assignment design on a representative sample of all Head Start programs nationwide, was finally released three years after the data collection was complete, it found that students randomly assigned to Head Start performed no better on cognitive measures by the end of kindergarten and first grade.

Despite these null results, HHS issued a statement that in typical Orwellian fashion declared the program a huge success.  Assistant Secretary for Children and Families Carmen Nazario was quoted in the statement concluding that “Head Start has been changing lives for the better since its inception.” And Health and Human Services Secretary Kathleen Sebelius was quoted declaring that “research clearly shows that Head Start positively impacts the school readiness of low-income children”

If the government’s proclivity to delay the release of politically undesirable results and to manipulate — actually, completely distort – the findings is not enough to engender skepticism among reporters, researchers, and policymakers, I have no idea what will.  But I continue to see reporters, researchers, and policymakers invoke government research as authoritative without the least bit of critical scrutiny.

This uncritical acceptance of government press releases as gospel by reporters is particularly disgraceful.  I understand that reporters are miserably paid and stretched beyond their limit as staffs are reduced, but the heart of a reporter’s responsibility is to challenge the powerful.  And there is no one more powerful than the government.  They are so powerful that they can delay the release of research and declare that up is down when the results do come out.

-Jay Greene

You really notice the strides that have been made when you step back and think abut what the education reform discussion used to be like a decade ago and what it looks like today.  A decade ago when people like Greg, Matt, and I were talking about the benefits of school choice and the need to address the perverse incentives of lifetime employment tenure for ineffective teachers and salary schedules that reward endurance more than performance, we were treated like dangerous extremists.

Now the issues of choice, tenure, merit pay, testing, and accountability are a normal part of the discussion.  And most interestingly, these are parts of the normal discussion among Democrats — a party that had traditionally been too fearful of the teacher unions to treat discussion of these issues as acceptable.

Matt recently noted this remarkable development of a bipartisan consensus around reform issues.  And both Greg and I have touched on this in the past. But I was reminded of just how far the discussion has come, especially among Democrats, while watching this entire show of Morning Joe on MSNBC focused on education reform.  It featured a panel, almost all of whom were Democrats, and almost all of whom agreed about the same essential issues of education reform that Matt, Greg, and I could barely utter in polite company a decade ago.

To be sure, there is much to do before we make real progress in education.  But at least we are having a productive discussion about ideas that may bear fruit.  That’s progress.

One striking thing about the discussion on Morning Joe and among other groups of Democrats is that none of them acknowledge the change that has occurred in the last decade.  None of them acknowledge that the people who raised the same issues a decade earlier were branded (often by people like them) as right-wing radicals.  They all just act as if they had discovered these education reform ideas all on their own.

At first this annoyed me, but Greg reminded me that victory requires not caring about who takes credit.  If the Democrats for Education Reform-types want to believe that they invented ed reform, who cares as long as it helps produce progress.  And those DFER folks are making huge strides, at least in getting us to talk and think about useful reforms.  And frankly, that progress could only be achieved by having them talk about it, not us.

That’s what victory looks like — someone else who is more likely to be effective taking your ideas forward even if they do so without acknowledging you.

-Jay P. Greene

They find that access to a private school with a voucher in Milwaukee significantly increases the probability that students will graduate from high school:

“Our clearest positive finding is that the Choice Program boosts the rates at which students graduate from high school, enroll in a four-year college, and persist in college,” said John Witte, professor of political science and public affairs at the University of Wisconsin-Madison. ”Since educational attainment is linked to positive life outcomes such as higher lifetime earnings and lower rates of incarceration, this is a very encouraging result of the program.”

They also find that “when similar students in the voucher program and in Milwaukee Public Schools were compared, the achievement growth of students in the voucher program was higher in reading but similar in math.”  Unfortunately, the testing conditions changed during the study because the private school testing went from being low stakes to high stakes, making it difficult to draw strong conclusions about the effects of the program on test scores.

In addition, it should be remembered that the design of the Milwaukee study is a matched comparison, which is less rigorous than random-assignment.  The more convincing random-assignment analyses are significant and positive in 9 of the 10 that have been conducted, with the tenth having null effects.  You can find a summary and links to all of them here.

Perhaps the most interesting part of the new Milwaukee results is the report on special education rates in the choice program.  As it turns out, Wisconsin’s Department of Public Instruction grossly under-stated the percentage of students in the choice program who have disabilities.  Some reporters and policymakers act as if the Department of Public Instruction’s reports are reliable and insightful because they are a government agency, while the reports of university professors are distorted and misleading.  Read this report on special education rates and I think you’ll learn a lot about how politically biased government agencies like the Department of Public Instruction can be.

-Jay Greene

Rather, I think the reluctance to contribute [to a debate in support of Common Core] is due to hubris, impatience to focus on implementation, political naivete, and disdain for what they see as mean-spirited carping….

There are long rows of argument and persuasion still to be hoed. And, if you’re eager to overhaul what gets taught in forty-odd states serving forty million or more students, that’s probably as it should be. If Common Core-ites don’t have the patience or stomach for that task, they should let us know now–and save everyone a whole lot of grief.

The notion that Common Core proponents needn’t make their case is an affront to democratic values.

Well, Ed Next managed to find someone to argue for and against the quality of Common Core standards, producing a really excellent and illuminating exchange.  W. Stephen Wilson took the pro side and Ze’ev Wurman was on the con side.  I would encourage you to read the entire debate yourself, but here is my takeaway:  They were mostly in agreement about the quality of Common Core.  Both seemed to agree that Common Core was better than the standards previously in place in most states but worse than in a non-trivial number of other states.  They also agreed that Common Core standards are significantly weaker than the ones in most high-achieving countries.

So if they agree that Common Core is sort of mediocre, why does Wilson support them while Wurman oppose them?  Wilson sees the improvement on the standards of 30 or more states to be substantial progress.  He sees this as a first step toward developing stronger national standards that would be comparable to those of our overseas competitors and better than all previously existing state standards.

Wurman sees Common Core as significantly lowering the bar relative to several previously existing state standards, including very large states like California.  More importantly, he sees Common Core as the end of progress in improving standards rather than the beginning.  Once put in place, he sees no incentive for anyone to toughen national standards since no state will be competing to offer a more rigorous education in order to attract residents and businesses.  He also sees national standards as more easily captured and dummied-down by teachers unions and other entrenched interests who would prefer to have their members (and students) jump over a lower bar.

-Jay P. Greene

UPDATE — Stephen Wilson contacted me over at the Jay P. Greene blog to object to the description of his views as supporting the adoption of Common Core. He thinks Common Core math standards are much better than those that previously existed in 30 states but still lagging those in other states and high achieving countries. And he generally has no opinion on whether universal adoption of Common Core would represent progress or not or is desirable or not.

It appears that I was wrong. The Ed Next forum was more a discussion among critics than a debate between a supporter and opponent.

So we are back to Rick’s original complaint. We still don’t have anyone who was willing to debate in favor of the national adoption of Common Core based on the quality of the standards.

It’s pretty pathetic that supporters of Common Core couldn’t produce anyone to take the “pro” side of this debate. And it’s even more pathetic that supporters are determined to cram Common Core down our throats without feeling the need to intellectually defend it.

Yes, answers Roland Fryer in an amazing study released this month.  Based on earlier work, he identified 5 features of charter schools that helped them produce strong results: “increased time, better human capital, more student-level differentiation, frequent use of data to inform instruction, and a culture of high expectations.”  Fryer then somehow convinced the superintendent and school board in Houston to pursue these five reforms in a serious way in 9 struggling traditional public schools. (CORRECTION — the Houston folks report that they were eager to pursue some promising reforms and required no convincing.  They should be commended for that.) Here, in brief, is what they did:

To increase time on task, the school day was lengthened one hour and the school year was lengthened ten days. This amounts to 21 percent more school than students in these schools obtained in the year pre-treatment and roughly the same as successful charter schools in New York City. In addition, students were strongly encouraged and even incentivized to attend classes on Saturday. In an effort to significantly alter the human capital in the nine schools, 100 percent of principals, 30 percent of other administrators, and 52 percent of teachers were removed and replaced with individuals who possessed the values and beliefs consistent with an achievement-driven mantra and, wherever possible, a demonstrated record of achievement. To enhance student-level differentiation, we supplied all sixth and ninth graders with a math tutor in a two-on-one setting and provided an extra dose of reading or math instruction to students in other grades who had previously performed below grade level. This model was adapted from the MATCH school in Boston – a charter school that largely adheres to the methods described in Dobbie and Fryer (2011b). In order to help teachers use interim data on student performance to guide and inform instructional practice, we required schools to administer interim assessments every three to four weeks and provided schools with three cumulative benchmarks assessments, as well as assistance in analyzing and presenting student performance on these assessments. Finally, to instill a culture of high expectations and college access for all students, we started by setting clear expectations for school leadership. Schools were provided with a rubric for the school and classroom environment and were expected to implement school-parent-student contracts. Specific student performance goals were set for each school and the principal was held accountable for these goals.

And the result:

In the grade/subject areas in which we implemented all five policies described in Dobbie and Fryer (2011b) – sixth and ninth grade math – the increase in student achievement is dramatic. Relative to students who attended comparison schools, sixth grade math scores increased 0.484σ (.097) in one year. In seventh and eighth grades, the treatment effect in math is 0.125σ (.065) and is statistically significant. A very similar pattern emerges in high school math: large effects in ninth grade and a more modest but statistically significant effect in tenth and eleventh grade, which suggest that two-on-one tutoring is particularly effective. The results in reading exhibit a different pattern. If anything, the reading scores demonstrate a slight decrease in middle school, though not statistically significant, and a modest increase in high school. Impacts on attendance – which are positive and statistically insignificant – are difficult to interpret given the longer school day and longer school year.

Strikingly, both the magnitude of the increase in math and the muted effect for reading are consistent with the results of successful charter schools. Taking the treatment effects at face value, treatment schools in Houston would rank third out of twelve in math and fifth out of twelve in reading among charter schools in NYC with statistically significant positive results in the sample analyzed in Dobbie and Fryer (2011b).

Using data from the National Student Clearinghouse, we investigate treatment effects on two college outcomes: whether a student enrolled in any college (extensive margin) and whether they chose a four-year college, conditional on enrolling in any college (intensive margin). Calculated at the mean, students are 6.2 percentage points less likely to attend college, though the effect is not statistically significant. Conditional on attending college, however, treatment students are 17.7 percentage points more likely to enroll in a four-year institution, relative to a mean of 46% in comparison schools – a 40% increase.

Traditional public schools can get results like a KIPP school without having to actually become KIPP schools. They just have to imitate a few of the key features employed by KIPP and other successful charter schools. This is incredibly encouraging news. It means that traditional public schools are really capable of making significant progress if only they become more open to learning from successful charter schools. They can make that progress without having to cure poverty and all other social ills (although I’m sure that would be nice too).

Of course, there are serious concerns about bringing these reforms to scale, which Fryer considers in his conclusion. He dismisses union opposition as a serious obstacle based on the fact that the unionized school system in Denver is pursuing a similar reform strategy. I’m not so easily convinced that unions nationwide will jump aboard a plan that involves huge turnover in staffing and significantly more hours and days per year. Cost is another barrier to bringing this reform strategy to scale, but he notes that the marginal cost is only $1,837 per student and the rate of return on that investment would be roughly 20%.

But the most serious concerns seem to be fidelity to implementation and shortages of quality labor. We could all be heart surgeons if we just did what heart surgeons do. But there are only so many people capable of doing that work and not every office building can be re-organized as a hospital. Then again, successful teaching isn’t exactly heart surgery (although it can be just about as important), so perhaps there is real hope of bringing this to scale. We won’t know until we try it in more places with more schools.

- Jay Greene

Supporters of digital learning, many of whom were among the strongest supporters of national standards, have organized in opposition to the imposition of a single test on the nation’s schools.  As it stands, the federal government is dumping several hundred million dollars on two testing consortia to develop assessments based on the federally “incentivized” Common Core standards.  A choice of two tests is not the same as a single test, but it is darn close.  It’s like the old joke where you have a choice between death or roo-roo.

The backers of digital learning organized by Innosight issued a group letter in which they express their desire for a multitude of testing options because they (finally) recognize the connection between choice and innovation:

Create a dynamic testing ecosystem, not another one-size-fits-all assessment. Rather than a single common test, the federal-funded opportunity offers the potential to create a vibrant assessment ecosystem comprised of multiple platforms, open-item banks, and multiple testing options that encourages deeper learning. An assessment ecosystem, rather than a single common test, will give states the flexibility to take advantage of innovations in digital learning over time while maintaining interoperability and comparability.

Signatories to this anti-national testing statement include Clayton M. Christensen, Michael B. Horn, Gisele Huff, Terry  Moe, Tom Vander Ark, Bob Wise, and Julie. E. Young in addition to dozens of others.

I’m not sure why backers of digital learning have taken so long to recognize the threat posed by the nationalization movement.  And I really can’t understand why some of them have been ardent supporters of national standards.  The adoption of national standards only has the possibility of having an effect if it is tightly connected to national testing and curriculum.

The “tight-loose” idea that we can nationally impose standards but allow a wide range of assessments, curricula, and teaching methods is just an empty slogan used to conceal the inevitability of nationalizing all of these aspects of the education system if the standards are to mean anything.  If we don’t have a common way of assessing, how can we be sure that everyone is adhering to the national standards?  And if the national standards are more than vague generalities, they inevitably drive  what is in the curriculum and how it must be taught.  You can have a little bit of nationalization about as much as you can be a little bit pregnant.

Despite the intellectual incoherence of some of these digital learning backers of national standards but opponents of national testing, it is nice to see the nationalization train starting to go off the tracks.  As the train moves further along and the full implications of nationalizing key aspects of the education system become more obvious to everyone, more and more people will jump that train. Without significant coercion it will be very hard to keep everyone on board until they reach the station where standards, assessments, and curriculum are all centrally imposed.

-Jay P. Greene

In the current issue of the Education Gadfly and on the Education Next blog Checker Finn offers an unusual argument for adoption of K-12 national standards.  He likens opposition to national standards to rooting for the Euro to fail:

If you hope the Euro crashes, that this week’s Brussels summit fails, and that European commerce returns to francs, marks, lira, drachma, and pesetas, you may be one of those rare Americans who also seeks the demise of the Common Core State Standards Initiative in U.S. education.

It’s odd that Checker should pick the Euro as a way to make the case for national standards since the Euro’s difficulties wonderfully illustrate the problems with national standards.  The Euro is in trouble because it was an attempt to impose a common currency on countries that were too diverse in their economic needs and political traditions.  The Euro is too strong of a currency for countries with un-competitive labor forces and undisciplined budget deficits, like Greece, Italy, and Spain.  But if the European Central Bank significantly loosens the currency to bail out these countries, it will create serious inflation problems in countries like Germany and others with more skilled labor forces and reasonable deficits.

The Euro is not in trouble because some people “hope the Euro crashes.”  It’s in trouble because it is a centralized institution that does not fit the diversity of its members.

Similarly, national standards will fail because it is not possible to have a centrally determined set of meaningful standards that can accommodate the legitimate diversity of needs, goals, and values of all of our nation’s school children.  To have an effect national standards inevitably drive the assessments that are used to measure student achievement as well as the methods of instruction that are used to produce that achievement.  ”Tight-loose” is just an empty slogan (or part of a drinking game).  In reality standards, assessments, and instruction are closely connected unless they are just irrelevant things.

In a country as large and diverse as ours there is no single, right set of knowledge for all students to possess, no single, best way to assess that knowledge, and no single, best method for teaching it.  The attempt to impose a nationalized system onto this diversity is doomed to fail just as the Euro is doomed to fail in imposing a common currency on such diverse economies and political systems.

The fact that the Euro is in such trouble and creating such political and economic turmoil ought to scare us away from trying to impose a centralized solution on too much diversity.  The Euro crisis is an argument against national standards, unless we are eager to have similar difficulties here.

No one is rooting for those failures, per se.  Some of us just recognize that reality is not created by repeating slogans to each other over catered lunches at DC think tank conferences.  Reality actually exists out there in the world and no matter how many chardonnays I’ve had while listening to the keynote speaker and no matter how many grants the Gates Foundation sprinkles on me and my friends, centrally imposing institutions on too much diversity is doomed to fail.

Of course, there is a way to overcome that diversity and improve the chances for centrally imposed institutions to succeed — force.  If European countries relinquish power to make their own budgets to a central authority, the Euro might work.  Similarly, if individual schools, school districts, and states relinquish power over daily operations to a central authority, the nationalized education movement might succeed.

But achieving that type of centralization in the face of diversity requires an enormous amount of coercion.  People who disagree have to be suppressed, or at least denied the ability to do anything about their dissent.  Local folks no longer get to make the meaningful decisions.  They can just implement the decisions that are centrally made.

This could work but it would be awful.  Some people say they would favor a World Government if only it were possible to do it.  I’m not one of those people.  World Government would be awful because it would require an enormous amount of coercion to overcome local diversity.  To a much lesser degree, a nationalized education system in the US could be done but it would run roughshod over the needs and legitimate interests of many individuals.

But some people are nevertheless attracted to centralized solutions.  I think Tears for Fears has a song that might explain why.

The stories were embarrassing, but the reaction by the New Times and others has been completely lacking in perspective.  Organizations receiving government funds are unfortunately even more prone to misconduct than typical organizations.  This is also true of public schools.  For example in the Detroit Public Schools we see:

Five Detroit Public Schools employees have been charged with embezzlement in an ongoing probe into the “culture of corruption” that took hold in the state’s largest district, a prosecutor said Wednesday….

A series of audits into district finances have been ordered. Two separate audits announced last week revealed the district has been paying $2.1 million per year for health coverage for ineligible dependents, and bought 160 unused BlackBerries and 11 motorcycles.

“It has been said that the accomplice to corruption is frequently our own indifference, and I agree wholeheartedly with that,” Worthy told reporters in announcing the charges.

“My office was not surprised about the culture of corruption that we’ve been seeing in the past in the Detroit Public Schools system,” she said. “What did surprise even us, though … is how rampant, how overt and how conspicuous and downright bold-faced the corruption is, allegedly, in some of the cases that we’ve been looking at.”

And from Springfield, MA public schools we learn:

A 13-month audit recently concluded at Putnam Vocational Technical High School found that some employees abused a student association checking account that operated independently from the city and school system in apparent violation of Massachusetts law.

McCaskill was in charge of that unauthorized account, which averaged about $200,000 annually in transactions since late 2005, but was managed with a manual ledger that never matched bank statements, according to the report from Springfield’s Office of Internal Audit….

“There is no excuse for the disgraceful, dishonest practices that appeared to have run rampant among a group of employees at the school for several years,” Ingram also wrote in a post on his official blog.

And right in the backyard of the Miami New Times we find the mysterious absence of $3.8 million from the Broward County teacher union accounts, the prior head of the Broward teacher union in jail for soliciting sex from a minor, and the former head of the Miami-Dade County teacher union in jail for corruption and embezzlement.

And while the New Times was repeating the complaints of Miami-Dade superintendent  Alberto Carvalho about McKay, they somehow failed to mention Carvalho’s own history of manipulating newspaper coverage through a reporter with whom he was reportedly having an affair.

But these are just selected anecdotes.  In a systematic study of scandals in public and private schools, Greg Forster and Matthew Carr found that misconduct was actually slightly more likely in regulated public schools than in largely unregulated private schools.  That is, some amount of scandal is unfortunately unstoppable and increasing regulation or government operation of schools is unlikely to eliminate the problem.

Of course, the existence of misconduct in traditional public schools in no way excuses the misconduct that has been uncovered in the McKay program.  But then again no one calls for the public school system to be shut down as a result of these scandals like folks are calling for an end to McKay.  And Diane Ravitch, in her typical, scholarly fashion, responds to the McKay reports by tweeting “Legalized child abuse in Florida?”, but appears to have no reaction to similar reports from traditional public schools.

My point is that the reaction to reports of misconduct in the McKay program are lacking perspective.  Yes, abuses need to be stopped.  And the regulations on the books, if enforced, could keep those abuses to a minimum. As former Senator John McKay told a Florida newspaper in response to calls for more regulation:

Kriseman suggested nine issues to increase accountability, including mandatory site inspections of facilities. He said the Department of Education should review and sign off on personnel criminal background checks in facilities seeking to receive McKay dollars. And teachers in a school accepting McKay dollars should have a state teaching certificate.

Former State Senate President John McKay — who created the law — agrees. McKay listened to Kriseman’s full list of suggestions.

“A number of his suggestions are quite positive,” McKay said. “Many of the things he’s asking for are already in the statute.”

McKay suggested asking officials with the Department of Education to enforce the law.

“It’s nice to have words in a statute,” McKay said. “Unless someone does something, it’s kind of meaningless.”

And of course, all of these criticisms of McKay fail to mention the proven positive effects of the program.  It improves student achievement for disabled students, reduces the rate of new identification of disabilities, increases the chances that students will receive needed services, and is overwhelmingly loved by parents.  I wish we could say the same about all traditional public schools, including those riddled with misconduct.

-Jay P. Greene

A package of model legislation opposing the common standards gained ground yesterday at the American Legislative Exchange Council.

The organization’s education task force approved the package, we learned from a couple of folks who attended those sessions of ALEC’s meeting this week in Scottsdale, Ariz.

Gewertz added that the measures do not become official ALEC policy until they are approved by the board of directors.  A similar proposal was proposed last summer by members of the education task force but was tabled until the recent meeting.  Allies of Jeb Bush and the long, gilded arm of the Gates Foundation pulled out the stops to block the measure and may yet succeed at the board level.

I fear that even if the measure is approved by ALEC’s board, the battle over adoption may effectively be finished.  An effort to repeal Common Core standards in Alabama failed despite the fact that the governor proposed the repeal and votes on the state board of education.   If you can’t repeal national standards in Alabama under such favorable conditions, it may be very hard to repeal it in any of the other 40-some states that have signed on.

But just because the adoption debate is winding down doesn’t mean the national standards war is over.  Far from it.  So far states have done the costless and non-constraining step of adopting a set of standards.  Once the nationalizers try to make the standards concrete and binding by incorporating them into newly designed high-stakes testing, we are likely to see a lot more resistance.  And adopting those new tests, revising teacher training, professional development, and textbooks to fit the national standards and testing will require considerable effort and expense — causing more states to rethink their initial support for Common Core.

The ALEC anti-Common Core measure will be important for mobilizing opposition as those next hurdles have to be jumped.  Even if the nationalization effort successfully runs this gauntlet, which they may do, the probability that national standards and assessments will actually produce the end goal — significantly improved student achievement over the long term — is near zero.  If nationally setting goals and ordering progress toward those goals were the path to success, the Soviet Gosplans would have produced their economic triumph over the West.  We all know how well that turned out.

-Jay P. Greene

I don’t want to repeat what I wrote in Ed Next and I don’t want to speak for Bob, so I would just urge you to read these pieces for yourself.  But just to anticipate objections, let me emphasize that I have no problem with unionization and collective bargaining in a competitive private market.  People should be free to associate and free to negotiate the terms of providing their labor.

The problem with teacher unions and public sector collective bargaining is that the checks and balances provided by market competition are absent.  So, public sector unions can get “management” to increase revenue for the industry and for union members without having to improve productivity.  They can just increase taxes or shift spending from other public purposes.  Private sector collective bargaining is constrained by the reality that they cannot just print their own money and must agree on productivity improvements so that there is more revenue to split.

In addition to the lack of incentives to improve productivity in public sector collective bargaining, we have the additional political distortions that unions, as a more concentrated and well-organized interest, have enormous political influence.  So, the unions are essentially sitting on both sides of the bargaining table.  This problem is more severe at the local level, since local political contests are less salient and more easily captured by well-organized interests.  At least in the private sector management usually tries to represent the interests of shareholders, but in the public sector the diffused interests of taxpayers are much less likely to be represented.

And in case any of you have idealized visions of teacher unions protecting the worker dancing in your head, a little snippet from the Education Intelligence Agency should awake you from your slumber:

In August, the American Federation of Teachers began an audit of the Broward Teachers Union’s (BTU) finances. Who at BTU asked for the audit is a matter of contention, but AFT uncovered several anomalies in the course of its two-month investigation.

Among them was the apparent reimbursement out of union dues for campaign contributions made by 26 ”employees, board members and their relatives.” This is, needless to say, illegal. The Broward State Attorney’s Office and the Florida Elections Commission were notified, and both agencies opened an official investigation.

Members of BTU’s executive board accused union president Pat Santeramo of not only being complicit in the reimbursement, but also covering up a $3.8 million budget shortfall and accepting salary overpayments….

Whatever Santeramo has done, he is actually the least reprehensible recent BTU president. He took over the position in 2001 after his predecessor was charged and plead guilty to attempting to entice a minor into a sex act and sending child pornography over the Internet. He was sentenced to 48 months in prison. And Santeramo’s actions are small potatoes when placed aside those of Pat Tornillo.

-Jay Greene

Richard D. Kahlenberg: Wisconsin governor Scott Walker’s campaign earlier this year to significantly curtail the scope of bargaining for the state’s public employees, including teachers, set off a national debate over whether their long-established right to collectively bargain should be reined in, or even eliminated.

If you’re a Republican who wants to win elections, going after teachers unions makes parochial sense. According to Terry Moe, the National Education Association (NEA) and the American Federation of Teachers (AFT) gave 95 percent of contributions to Democrats in federal elections between 1989 and 2010. “Collective bargaining is the bedrock of union well-being,” Moe notes, so to constrain collective bargaining is to weaken union power. The partisan nature of Walker’s campaign was revealed when he exempted two public-employee unions that supported him politically: those representing police and firefighters.

But polls suggest that Americans don’t want to see teachers and other public employees stripped of collective bargaining rights. A USA Today/Gallup poll found that by a margin of 61 to 33 percent, Americans oppose ending collective bargaining for public employees. A Wall Street Journal/NBC poll discovered that while Americans want public employees to pay more for retirement benefits and health care, 77 percent said unionized state and municipal employees should have the same rights as union members who work in the private sector. Is the public wrong in supporting the rights of teachers and other public employees to collectively bargain? I don’t think so.

Richard D. Kahlenberg

The NEA has existed since 1857 and the AFT since 1916, but teachers didn’t have real influence until they began bargaining collectively in the 1960s. Before that, as Albert Shanker, one of the founding fathers of modern teachers unions, noted, teachers engaged in “collective begging.” Educators were very poorly compensated; in New York City, they were paid less than those washing cars for a living. Teachers were subject to the whims of often autocratic principals and could be fired for joining a union.

Some teachers objected to the idea of collective bargaining. They saw unions as organizations for blue-collar workers, not for college-educated professionals. But Shanker and others insisted that teachers needed collective bargaining in order to be compensated sufficiently and treated as professionals.

Democratic societies throughout the world recognize the basic right of employees to band together to pursue their interests and secure a decent standard of living. Article 23 of the 1948 Universal Declaration of Human Rights provides not only that workers should be shielded from discrimination, but also that “everyone has the right to form and to join trade unions for the protection of his interests.”

Collective bargaining is important, not only to advance individual interests but to give unions the power to serve as a countervailing force against big business and big government. Citing the struggle of Polish workers against the Communist regime, Ronald Reagan declared in a Labor Day speech in 1980, “where free unions and collective bargaining are forbidden, freedom is lost.”

The majority of Americans believe that citizens don’t give up the basic right to collective bargaining just because they work for the government. In free societies across the globe, from Finland to Japan, public school teachers have the right to form unions and engage in collective bargaining.

In the United States, only seven states outlaw collective bargaining for teachers. Thirty-four states and the District of Columbia authorize collective bargaining for such employees, and another nine permit it. It is no accident that the seven states that prohibit collective bargaining for teachers are mostly in the Deep South, the region of the country historically most hostile to extending democratic citizenship to all Americans.

Terry Moe finds that collective bargaining for teachers has strong support among candidates for school boards. He writes, “the vast majority of school board candidates, 66 percent, have positive overall attitudes toward collective bargaining. Even among Republicans—indeed, even among Republicans who are not endorsed by the unions—the majority take a positive approach to this most crucial of union concerns.”

Nonetheless, some (including Moe) would prefer that collective bargaining for teachers be severely curtailed, or even outlawed. Ironically, one argument advanced by critics is that collective bargaining is undemocratic. The other major argument is that teacher collective bargaining is bad for education. Both claims are without basis.

Those who argue that collective bargaining for teachers is stacked, even undemocratic, say that, unlike in the private sector, where management and labor go head-to-head with clearly distinct interests, in the case of teachers, powerful unions are actively involved in electing school board members, essentially helping to pick the management team. Moreover, when collective bargaining covers education policy areas, such as class size or discipline codes, the public is shut out of the negotiations, some assert. Along the way, they conclude, the interests of adults in the system are served but not the interests of children.

But these arguments fail to recognize that in a democracy, school boards are ultimately accountable to all voters, not just teachers, who often live and vote outside the district in which they teach, and in any event represent a small share of total voters. Union endorsements matter in school board elections, but so do the interests of general taxpayers and parents and everyone else who makes up the community. If school board members toe a teachers union line that is unpopular with voters, those officials can be thrown out in the next election.

Indeed, one could make a strong argument that any outsized influence that teachers unions exercise in school board elections provides a nice enhancement of democratic decisionmaking on education policy because teachers, as much as any other group in society, can serve as powerful advocates for those Americans who cannot vote: schoolchildren. The interests of teachers and their unions don’t always coincide with those of students, but on the really big issues, such as overall investment in education, the convergence of interests is strong. Certainly, the interests of teachers in ensuring adequate educational investment are far stronger than they are for most voters, who don’t have children in the school system and may be more concerned about holding down taxes than investing in the education of other people’s kids.

American society consistently underinvests in children compared with other leading democratic societies. According to the Organisation for Economic Co-operation and Development (OECD), the child poverty rate in the United States is 21.6 percent, the fifth-highest among its 40 member nations. Only Turkey, Romania, Mexico, and Israel have higher child-poverty rates. Put differently, we’re in the bottom one-eighth in preventing child poverty. By contrast, when the interests of children are connected with the interests of teachers, as they are on the question of public education spending, the U.S. ranks close to the top one-third. Among 39 OECD nations, the U.S. ranks 14th in spending on primary and secondary education as a percentage of gross domestic product.

Some critics argue that strong teachers unions make for inefficient spending and bad education policies in the instances when teacher and student interests diverge. For example, it is frequently claimed that teachers unions, through collective-bargaining agreements, protect incompetent members and prevent good teachers from being paid more.

This sometimes occurs, and when it does, it is troublesome. But a number of reform union leaders, going back to Al Shanker, have embraced “peer review” plans, which weed out bad teachers in Toledo, Ohio; Montgomery County, Maryland; and elsewhere. These reform plans put the lie to the notion that the average teacher has an interest in her union protecting incompetent colleagues. To the contrary, dead wood on the faculty makes every other teacher’s job more difficult. Likewise, numerous local unions have adopted pay-for-performance plans, when the measurement of performance is valid and incentives are in place to encourage good teachers to share innovative teaching techniques rather than hoarding them.

Moreover, many of the things that teachers collectively bargain for are good for kids. The majority of students benefit when teachers can more easily discipline unruly students, for example. (Principals, by contrast, often want to take a softer line so the school’s suspension rates don’t look bad.) Higher compensation packages attract higher-quality teacher candidates and reduce disruptive teacher turnover.

If collective bargaining were really a terrible practice for education, we should see stellar results where it does not occur: in the American South and in the charter school arena, for example. Why, then, aren’t the seven states that forbid collective bargaining for teachers (Arizona, Georgia, Mississippi, North Carolina, South Carolina, Texas, and Virginia) at the top of the educational heap? Why do charter schools, 88 percent of which are nonunion, only outperform regular public schools 17 percent of the time, as a 2009 Stanford University study found? Why, instead, do we see states like Massachusetts, and countries like Finland, both with strong teachers unions, leading the pack?

Opponents of collective bargaining will immediately point out that poverty rates are high in the American South, and low in Finland, which is an entirely valid point. But doesn’t that suggest that the national obsession with weakening teachers unions may be less important than finding ways to reduce childhood poverty?

Moreover, scholarly studies that seek to control for poverty find that collective bargaining is associated with somewhat stronger, not weaker, student outcomes. Sociologist Robert Carini’s 2002 review of 17 studies found that “unionism leads to modestly higher standardized achievement test scores, and possibly enhanced prospects for graduation from high school.” Even Terry Moe, an outspoken opponent of collective bargaining for teachers (see “Seeing the Forest Instead of the Trees,” book reviews, page 77), suggests that research on the impact of collective bargaining on student outcomes “has generated mixed findings (so far) and doesn’t provide definitive answers.”

For a variety of reasons, collective bargaining for teachers should not be constrained, much less eliminated. Indeed, if teachers are to be partners in innovative education reform (see “A Different Role for Teachers Unions?” features, page 16), the scope of collective bargaining should be expanded. When the United Federation of Teachers first began to bargain collectively in the early 1960s, Albert Shanker was distressed that the New York City school board was willing to discuss only traditional issues like wages and benefits and rejected the idea of bargaining over broader policies that the union proposed, such as the creation of magnet schools.

Shanker saw that by reducing the scope of collective bargaining, critics created a political trap for unions. Union leaders were told they could only address bread-and-butter issues and then were criticized for caring only about their own selfish concerns rather than student achievement or larger policy issues. Moreover, Shanker believed that teachers had a lot of good ideas that could be incorporated into collective bargaining agreements, such as teacher peer review, suggestions for the types of curricula that work best in the classroom, and what sorts of programs would lure teachers into high-poverty schools. He also knew that reforms that draw on teacher wisdom are more likely to be effectively implemented when the classroom door closes.

In the end, Shanker’s frustration with the traditional constraints of collective bargaining spurred him to propose, in a 1988 speech at the National Press Club, the creation of “charter schools,” where teachers would draw upon a wealth of experience to try innovative ideas. Much to Shanker’s dismay, the charter school movement went in a very different direction, becoming a vehicle for avoiding unions and reducing teacher voice (and thereby increasing teacher turnover). And charters still educate a very small fraction of students.

Expanding collective bargaining for teachers to more states and to more education issues will give educators greater voice, and in so doing, indirectly strengthen the voice of students. Overall, the evidence suggests that Scott Walker has it exactly wrong, and the American public, which overwhelmingly supports the right to collective bargaining, has it right.

Jay P. Greene: Asking if teachers unions are a positive force in education is a bit like asking if the Tobacco Institute is a positive force in health policy or if the sugar lobby is helpful in assessing the merits of corn syrup. The problem is not that teachers unions are hostile to the interests of students and their families, but that teachers unions, like any organized interest group, are specifically designed to promote the interests of their own members and not to safeguard the interests of nonmembers. To the extent that teachers benefit from more generous pay and benefits, less-demanding work conditions, and higher job security, the unions will pursue those goals, even if achieving them comes at the expense of students. That is what interest groups do. Unfortunately, a public education system that guarantees ever-increasing pay and benefits while lowering work demands on teachers, who virtually hold their positions for life regardless of performance, harms students.

Collective bargaining is the primary vehicle through which the unions enact their preferred policies regarding pay, benefits, job security, and work conditions. It is also the mechanism by which unions collect fees from teachers that provide them with the resources to prevail politically. Until the ability of teachers unions to engage in collective bargaining is restrained, we should expect unions to continue to use it to advance the interests of their adult members over those of children, their families, and taxpayers.

Teachers unions only won the privilege of engaging in collective bargaining in the last 50 years, about when student achievement began to stagnate and costs to soar. A return to the pre–collective bargaining era may be the tonic our education system needs to return to growth in achievement and restraint in costs.

Jay P. Greene

The nature and function of organized interest groups is widely known and understood. Of course, there is nothing wrong with people organizing interest groups to advocate for themselves. That is an essential part of the freedom of assembly, protected by the U.S. Constitution. If people dislike what an interest group is advocating, they can organize other interest groups to compete in the marketplace of ideas and advocate for other concerns. The normal process of checks and balances among competing interest groups, however, has failed when it comes to education.

There are three factors that have contributed to the failure of other groups to check the power of teachers unions. First, there is an asymmetry in the ability of groups to organize in education, significantly favoring the teachers unions. Teachers unions have a huge advantage in organizing and advocating for their interests. Employees of the public school system are physically concentrated in school buildings, making it easier for them to organize. And because current employees are in a good position to know how they can benefit from the system, they can be mobilized relatively easily to advocate for those policies. Parents, taxpayers, and members of the general public are geographically dispersed, making it harder for them to organize. And because they are not immersed in education matters, they cannot easily envision how policy changes might help or hurt, making it harder to mobilize them on those issues. It is hardly unique to education that concentrated interests have an advantage over diffuse interests, but this is one factor contributing to teachers union dominance.

Second, teachers unions have fooled a large section of the general public and elites into thinking of them as something other than a regular interest group advocating for their own concerns.

The teachers unions have worked hard to convince people that they are a collection of educators who love our children almost as much as the parents do. They’re like the favorite aunt or uncle who dotes on our children. This image of the teachers unions as part of our family is facilitated by the fact that virtually every college-educated household (the households with the greatest political influence) has at least one current or former public school teacher sitting at the dining table when they gather for Thanksgiving. This impression is also fostered by ad campaigns featuring teachers buying school supplies out of their own pockets and movie portraits of heroic teachers believing in students, even as their parents have abandoned them.

Of course, some teachers really do buy school supplies with their own money (which should make people wonder what kind of education system would make that necessary after spending an average of more than $12,000 per student each year). And some teachers really are like the doting aunt or uncle who sticks with kids, even when the parents have given up. But loving children and being part of the family is certainly not what teachers unions are about. They are about accumulating the power necessary to advocate for the interests of their members. In a moment of candor, Bob Chanin, former general counsel of the National Education Association, explained the key to the union’s effectiveness: “Despite what some among us would like to believe, it is NOT because of our creative ideas. It is NOT because of the merit of our positions. It is NOT because we care about children, and it is NOT because we have a vision of a great public school for every child. NEA and its affiliates are effective advocates because we have power.”

The disarming image of teachers unions as Mary Poppins has begun to morph into that of a burly autoworker, as teachers union advocacy has become more militant in recent years. As states attempt to trim very generous benefit packages for teachers, the unions have organized large demonstrations, occupied state capitols, and chanted angry slogans. The public image of teachers unions fighting like autoworkers for the benefit to retire at 55 with full medical coverage and 66 percent of their peak salary while the economy is in shambles and the quality of their industry stagnates has done much to undermine the doting aunt or uncle meme. The angry slogans emanating from Diane Ravitch’s and Valerie Strauss’s Twitter feeds may soothe disgruntled teachers, but they are eroding the public perception that teachers unions are somehow different from other interest groups. Media and policy elites are increasingly treating teachers union claims with the same skepticism that they used to apply only to other interest groups.

A third factor is that unions have significant influence over who is elected or appointed to negotiate with them over pay, benefits, and work conditions. In the private sector, the power of unions is constrained by the competing organized interests of management. When they sit down to negotiate pay, benefits, and work conditions, members of management are inclined to represent the interests of shareholders, not those of employees. But in education, as in other public-sector collective bargaining, the interests of employees are represented on both sides of the table. The employees, as citizens, can organize, finance, and vote for elected officials who favor the union’s interests. It is precisely for this reason that public employees historically did not have collective bargaining rights.

But didn’t the lack of collective bargaining rights sometimes leave teachers vulnerable to arbitrary and discriminatory treatment by school administrators? Yes, but unionization and collective bargaining were neither necessary nor efficient means of correcting those abuses. We can look to other public employees, such as members of the armed forces, who still do not have collective bargaining rights, to see how progress could have occurred without unionization. The military, like public schools, was once racially segregated. African American servicemen and servicewomen were treated horribly. And sometimes officers treated all soldiers in an arbitrary and unfair manner. These abuses were not corrected by unionization and collective bargaining in the military. They were corrected by executive orders and changing legislation governing those public employees. The same path could have been taken with public school employees without the political distortions that public employee unions introduce by virtue of having their interests represented on both sides of the bargaining table.

It may have taken longer than many would like to integrate the military, expand the roles of women in the armed forces, and end “don’t ask, don’t tell,” but we were able to achieve all of those through an open, public process of changing laws and regulations. Unionized collective bargaining might also have addressed those issues, but it would have been done mostly behind closed doors and would have been accompanied by provisions to protect the narrow interests of the unions at the expense of the public interest. Perhaps the use of drones would have been restricted because it displaces jobs for Air Force pilots; perhaps there would be caps on the hours soldiers could engage in combat. Who knows what else a unionized military might have produced? The point is we rightly restrict the ability of members of the armed forces from unionizing and engaging in collective bargaining, just as we once did and could again for teachers. The claim that public employees have a “right” to unionize and collectively bargain and that exercising this “right” necessarily advances the public interest is obviously false.

The proper mechanism for improving compensation and work conditions in the public sector is through changes in law and regulation. The salary, benefits, job security, and work conditions of public employees are just as much a matter of public policy as the work that those employees are supposed to do. We don’t allow smoky backroom deals arrived at in collective bargaining to dictate the goals, structure, or existence of the public education system, so neither should we use that process to determine compensation and work condition policies.

What evidence is there that teachers unions have actually had negative effects on students and the education system? The research literature generally finds that unionization is associated with higher per-pupil costs and lower student achievement, but those findings are not very large and are sometimes inconsistent. A 1996 article by Caroline Hoxby in the Quarterly Journal of Economics is widely considered the most methodologically rigorous analysis of the issue. Claremont Graduate University professor Charles Kerchner described Hoxby’s study in a literature review prepared for the National Education Association as “the most sophisticated of the econometric attempts to isolate a union impact on the student results and school operations …” Hoxby finds that unionization is associated with higher student dropout rates as well as higher spending.

But the reality is that it is very hard to produce rigorous research on the effects of teachers unions on education. For one thing, teachers unions are powerful and active almost everywhere. Even in states without collective bargaining, the unions push state legislatures to put into law what is normally put into collective bargaining agreements. This is less than ideal for the unions, because they don’t collect dues in exchange for pushing through legislation like they can for representing members to achieve the same ends through collective bargaining. Unions operate these money-losing operations in right-to-work states to make sure that there is no meaningful policy variation on their key issues. They’d rather that we not discover that the world does not end without a mandatory step-and-ladder pay scale, fair dismissal procedures, and favorable work rules. The lack of policy variation hinders researchers, because outcomes are not likely to be very different where the policies are not very different.

But we don’t need a wealth of evidence on teachers unions specifically as long as we know about the effects of interest groups and recognize that teachers unions are indeed interest groups. Seeking to produce evidence on the effects of each interest group separately, especially when there are empirical challenges to doing so, is a bit like trying to prove that gravity operates in every room of a house. We could drop a bowling ball in each room to see if it hits the floor, but sometimes there are tables, couches, or beds in the way. If we don’t get the result we expected, it doesn’t mean that gravity only applies in certain places; it just means that research constraints prevent us from seeing in a particular situation what we know to be true in general.

In general, we know that interest groups advocate for the benefits of their members, even if it comes at the expense of others. We know that teachers unions are interest groups. And we know that the interests of teachers unions are not entirely consistent with the needs of students and taxpayers. Thus, teachers unions are likely to be negative forces for the education system and certainly should not be seen as helpful. The most rigorous research that does exist bears this out, but we also know this from our more general knowledge of how interest groups affect policy.

It is not currently practical to forbid the unionization of teachers, as we forbid the unionization of members of the armed forces. But if we want to limit the ability of teachers unions to advance their own interests at the expense of children, their families, and taxpayers, we need to consider ways of restricting their ability to engage in collective bargaining. Restricting collective bargaining would force teachers unions to pursue their interests through the legislative process, where competing interests might have a better chance to check their power. And forcing unions to operate through legislation rather than backroom collective-bargaining negotiations would improve transparency, which could also place a check on the unions’ ability to satisfy their own interests at the expense of others.

RDK: Jay Greene’s opening line, comparing teachers unions to the Tobacco Institute, is very telling about his overall analysis. He’s right, of course, that both are “interest groups,” but does he not see a massive difference between an entity that is devoted to getting more kids addicted to deadly cigarettes so they’ll be lifelong clients and a group representing rank-and-file teachers whose life’s work is educating children?

Greene complains that teachers unions have become “more militant in recent years.” But teacher strikes, which were quite common in the 1960s and 1970s, dropped 90 percent by the mid-1980s and are now, as one education report noted, essentially “relics of the past.” To the extent that teachers have rallied, it’s in response to unprecedented attacks on them in places like Wisconsin, where a half century of labor law was radically rewritten. Astonishingly, Greene would go further than Wisconsin Republicans and “return to the pre–collective bargaining era.”

Greene says providing teachers with better pay and benefits is bad for kids, but where is his evidence? Don’t better compensation packages attract brighter talent, or are the laws of supply and demand suddenly suspended when it comes to teachers?

Finally, Greene is correct to suggest that teacher and student interests are not perfectly aligned, but who are the selfless adults who better represent the interests of kids? The hedge fund managers who support charter schools and also want their income taxed at lower rates than regular earned income, thereby squeezing education budgets? Superintendents who sometimes junk promising initiatives for which they cannot take credit? I’d rather place my faith in the democratically elected representatives of educators who work with kids day in and day out.

JPG: Richard Kahlenberg places his faith in “democratically elected representatives of educators,” that is, the teachers unions, to safeguard the interests of children. Note that he does not say the democratically elected representatives of the people, or the voters. Kahlenberg is perfectly comfortable with a school system whose policies and practices are dominated by its employees, not by the citizens who pay for it or by the families whose children are compelled to attend it. Rather than seeing a system controlled by its employees as one characterized by self-interested adults maximizing their benefits at the expense of children, Kahlenberg sees it as the ideal.

In my ideal vision, we would put our faith in parents, not teachers unions, to represent the interests of children. If we had a robust system of parental school choice, I would have no problem with teachers unions and collective bargaining. In the private sector, if unions ask for too much, at least they experience the natural consequences of destroying their own companies or industries (to wit, the auto industry). But in the public sector, unions are almost entirely insulated from the consequences of making unreasonable demands, since governments never go out of business. Public sector unions can drive total revenue for their industry higher without any improvements in productivity simply by getting public officials to increase taxes.

Unfortunately, we lack a robust system of school choice and instead have to rely on democratic institutions, like school boards and state legislatures, to determine most school policies and practices. But unless we also restrict the collective bargaining rights of school employees, teachers unions will dominate the decisions of those democratic institutions, given their advantages in funding and organization, to distort elections and policy decisions.

Teachers unions almost certainly raise salaries and benefits, as Kahlenberg suggests, but that doesn’t necessarily attract better teachers if the salary schedule does nothing to reward excellence. Similarly, union-imposed dismissal procedures make it virtually impossible to fire ineffective teachers. The alignment that Kahlenberg sees between teachers unions’ desire to increase education spending and the interests of students would only be a real concordance if the unions facilitated the use of those funds in ways that actually improved outcomes.

American education has problems, almost everyone is willing to concede, but many think those problems are mostly concentrated in our large urban school districts. In the elite suburbs, where wealthy and politically influential people tend to live, the schools are assumed to be world-class.

Unfortunately, what everyone knows is wrong. Even the most elite suburban school districts often produce results that are mediocre when compared with those of our international peers. Our best school districts may look excellent alongside large urban districts, the comparison state accountability systems encourage, but that measure provides false comfort. America’s elite suburban students are increasingly competing with students outside the United States for economic opportunities, and a meaningful assessment of student achievement requires a global, not a local, comparison.

We developed the Global Report Card (GRC) to facilitate such a comparison. The GRC enables users to compare academic achievement in math and reading between 2004 and 2007 for virtually every public school district in the United States with the average achievement in a set of 25 other countries with developed economies that might be considered our economic peers and sometime competitors. The main results are reported as percentiles of a distribution, which indicates how the average student in a district performs relative to students throughout the advanced industrialized world. A percentile of 60 means that the average student in a district is achieving better than 59.9 percent of the students in our global comparison group. (Readers can find all of the results of the Global Report Card at http://globalreportcard.org. The web site contains a full description of the method by which we calculated the results. For a summary, see the methodology sidebar.)

For the purposes of this article, we focus on the 2007 math results, although the GRC contains information for both math and reading between 2004 and 2007. We focus on 2007 because it is the most recent data set, and we focus on math because it is the subject that provides the best comparison across countries and is most closely correlated with economic growth. Readers should feel free to consult the GRC web site to find reading results as well as results for other years.

The Example of Beverly Hills

It is critically important to compare exclusive suburban districts against the performance of students in other developed countries, as these districts are generally thought to be high-performing. The most wealthy and politically powerful families have often sought refuge from the ills of our education system by moving to suburban school districts. Problems exist in large urban districts and in low-income rural areas, elites often concede, but they have convinced themselves that at least their own children are receiving an excellent education in their affluent suburban districts.

Unfortunately, student achievement in many affluent suburban districts is worse than parents may think, especially when compared with student achievement in other developed countries. Take for example Beverly Hills, California. The city has a median family income of $102,611 as of 2000, which places it among the top 100 wealthiest places in the United States with at least 1,000 households. The Beverly Hills population is 85.1 percent white, 7.1 percent Asian, and only 1.8 percent black and 4.6 percent Hispanic. The city is virtually synonymous with luxury. A long-running television show featured the wealth and advantages of Beverly Hills high-school students (as well as their overly dramatic personal lives). If Beverly Hills is not the refuge from the ills of the education system that elite families are seeking, it’s not clear what would be.

But when we look at the Global Report Card results for the Beverly Hills Unified School District, we don’t see top-notch performance. The math achievement of the average student in Beverly Hills is at the 53rd percentile relative to our international comparison group. That is, one of our most elite districts produces students with math achievement that is no better than that of the typical student in the average developed country. If Beverly Hills were relocated to Canada, it would be at the 46th percentile in math achievement, a below-average district. If the city were in Singapore, the average student in Beverly Hills would only be at the 34th percentile in math performance.

Of course, people don’t think of Beverly Hills as a school district with mediocre student achievement. This is partly because people assume that affluent suburbs must be high achieving and partly because state accountability results inflate achievement by comparing affluent suburban school districts with large urban ones. According to California’s state accountability results, the average student in Beverly Hills is at the 76th percentile in math achievement relative to other students in the state. But outperforming students in Los Angeles, which is only at the 20th percentile in math relative to a global comparison group, should provide little comfort to Beverly Hills parents.

Los Angeles Unified is not the main source of competitors for Beverly Hills students, so the state accountability system encourages the wrong comparison. If Beverly Hills graduates are to have the kinds of jobs and lifestyles that their parents hope for them, they will have to compete with students from Canada, Singapore, and everywhere else. Beverly Hills students have to be toward the top of achievement globally if they expect to get top jobs and earn top incomes.

Results from Affluent Suburbs Nationwide

We can repeat the story of Beverly Hills all across the country. Affluent suburban districts may be outperforming their large urban neighbors, but they fail to achieve near the top of international comparisons (see Figure 1). White Plains, New York, in suburban Westchester County, is only at the 39th percentile in math relative to our global comparison group. Grosse Point, Michigan, outside of Detroit, is at the 56th percentile. Evanston, Illinois, the home of Northwestern University outside of Chicago, is at the 48th percentile in math. The average student in Montgomery County, Maryland, where many of the national government leaders send their children to school, is at the 50th percentile in math relative to students in other developed countries. The average student in Fairfax, Virginia, another suburban refuge for government leaders, is at the 49th percentile. Shaker Heights, Ohio, outside of Cleveland, is at the 50th percentile in math. The average student in Lower Merion, Pennsylvania, near Philadelphia, is at the 66th percentile. Ladue, Missouri, a wealthy suburb of St. Louis, is at the 62nd percentile. And the average student in Plano, Texas, near Dallas, is at the 64th percentile in math relative to our global comparison group.

All of these communities are among the wealthiest in the United States. All are overwhelmingly white in their population. All of them are thought of as refuges from the dysfunction of our public school system. But the sad reality is that in none of them is the average student in the upper third of math achievement relative to students in other developed countries. Most of them are barely keeping pace with the average student in other developed countries, despite the fact that the comparison is to all students in the other countries, some of which have a per-capita gross domestic product that is almost half that of the United States. In short, many of what we imagine as our best school districts are mediocre compared with the education systems serving students in other developed countries.

Pockets of Excellence

While many affluent suburban districts have lower achievement than we might expect, some districts are producing very high achievement even when compared with that of students in other developed countries. For example, the average student in the Pelham school district in Massachusetts is at the 95th percentile in math. That means that if we were to relocate Pelham to another developed country in our comparison group, the average student in Pelham would outperform 95 percent of the students in math. That’s very impressive.

Of course, Pelham is a small district that is home to Amherst College, among other institutions of higher learning, and serves a rather select group of students. But not all college-town school districts are equally high achieving. As we have already seen, Evanston, Illinois, is at the 48th percentile in math in a global comparison. Palo Alto, California, the home of Stanford University, is at the 64th percentile. And the average student in Ann Arbor, Michigan, home to the University of Michigan, is at the 58th percentile in math relative to students in other developed countries. So, the 95th percentile math achievement in Pelham is outstanding, even for college towns.

Spring Lake, New Jersey, has a similarly impressive record of having the average student at the 91st percentile in math. It is a very small and affluent community on the New Jersey shore that has somehow escaped the influence of Snooki and The Situation. Waconda, Kansas, a small rural community, also is at the 91st percentile. Highland Park, Texas, an affluent community near Dallas, is at the 88th percentile.

Interestingly, of the top 20 U.S. public-school districts in math achievement, 7 are charter schools (some states treat charter schools as separate public-school districts). And most of the 13 traditional districts remaining are in rural communities rather than in a large suburban “refuge” from urban education ills.

Pools of Failure

In total, only 820 of the 13,636 public-school districts for which we have 2007 math results had average student achievement that would be among the top third of student performance in other developed countries. That is, 94 percent of all U.S. school districts have average math achievement below the 67th percentile. There aren’t that many truly excellent districts out there.

Of the 13,636 districts, 9,339, or 68 percent, have average student math achievement that is below the 50th percentile compared with that of the average student in other developed countries. Most of our large school districts are well below the 50th percentile. This is especially alarming, because these lower-performing large districts comprise a much greater share of the total student population than do the relatively small higher-performing districts.

The average student in the Washington, D.C., school district is at the 11th percentile in math relative to students in other developed countries. In Detroit, the average student is at the 12th percentile. In Milwaukee, the average student is at the 16th percentile. Cleveland is at the 18th percentile. The average student in Baltimore is at the 19th percentile in math relative to students in other developed countries. In Los Angeles, the average student is at the 20th percentile. The average student in Chicago is at the 21st percentile in math. Atlanta is at the 23rd percentile. The average student in New York City is at the 32nd percentile in math. And in Miami-Dade County, the average student is at the 33rd percentile in math.

Not 1 of the largest 20 school districts is above the 50th percentile in math relative to other developed countries. Those districts contain almost 5.2 million students or more than 10 percent of the country’s schoolchildren. The rare and small pockets of excellence in charter schools and rural communities are overwhelmed by large pools of failure.

Previous Research

The Global Report Card is not the first analysis to compare the performance of U.S. students to international peers. Eric A. Hanushek, Paul E. Peterson, and Ludger Woessmann (see “Teaching Math to the Talented,” features, Winter 2011) used a very similar method to compare the performance of students in each state to students in other countries and arrived at similarly gloomy conclusions. Using state NAEP results for 8th-grade students and PISA results for 15-year-olds internationally, the researchers focused on the percentage of students performing at an advanced level in math. In almost every state, they found that we had far fewer advanced students than most of the countries taking PISA. They also narrowed the comparison to white students in the U.S. and to students whose parents had a college education to show that even advantaged students in the U.S. failed to achieve at an advanced level in math relative to their international peers. More recently, Hanushek et al. updated their analysis to examine the percentage of students in each state and across countries performing at the proficient level in math and reading.  The results were similarly disappointing.

The main difference between the GRC and the Hanushek et al. analyses is that in our study we push the comparison down to the district level. By focusing on white students and children of college-educated parents, Hanushek et al. clearly mean to convey that even students in elite suburban districts have mediocre achievement. Our contribution with the GRC is to name the districts so that people do not indulge the fantasy that their suburb’s record is somehow different from the disappointing performance of others with advantaged students in their state.

There are other important differences between the GRC and the Hanushek et al. analyses. We incorporate test results for U.S. students in all available grades (typically grades 3 through 8 and grade 10) rather than focusing on the grade closest to the 15-year-olds in the PISA sample. We could have focused only on 8th-grade results, as Hanushek et al. did, but in doing so we would have greatly reduced the number of test results on which we were doing the calculations for school districts. We preferred to gain precision in estimating the achievement in each district by increasing our sample size rather than restricting the sample to 8th graders in order to gain comparability in the age of the students under review.

The GRC analysis also differs from those of Hanushek et al. in that the latter focus on students performing at the advanced or proficient level, while we focused on the average student performance in both math and reading. Hanushek et al. concentrated on advanced or proficient performance because they were trying to compare our best students with the best abroad to show that even our best are mediocre. We did the same by highlighting the results for elite suburban school districts. Focusing on the average also avoids any dispute about how “advanced” or “proficient” are defined across different tests.

Gary Phillips at the American Institutes for Research has also conducted a series of analyses comparing state achievement on NAEP to international performance on a different international test, the Trends in International Mathematics and Science Study (TIMSS). Phillips arrives at somewhat less gloomy conclusions about U.S. performance, but that is because the countries included in TIMSS differ from those covered by PISA. Hanushek et al. rightly note that PISA provides a much more appropriate comparison for the U.S.: “Put starkly, if one drops from a survey countries such as Canada, Denmark, Finland, France, Germany, and New Zealand, and includes instead such countries as Botswana, Ghana, Iran, and Lebanon, the average international performance will drop, and the United States will look better relative to the countries with which it is being compared.”

This has sparked a debate among researchers about whether TIMSS or PISA provides a better set of countries against which we should compare the U.S. The Global Report Card circumvents this dispute by developing its own set of countries against which we compare U.S. students. The comparisons provided by TIMSS and PISA depend on which countries decide to take each test each time it is administered. And PISA scales its scores against the results for members of the OECD, which excludes countries like Singapore while including countries like Mexico. Our comparison group depends on PISA results, but it is also based on objective criteria, like per-capita GDP, to identify a set of developed economies that can be reasonably compared with that of the U.S. Our comparison group is a significant improvement on the self-selection of countries that choose to take a test as well as an improvement upon arbitrary membership in an organization like the OECD.

No Refuge

The elites, the wealthy families that have a disproportionate influence on politics, clearly recognize the dysfunction of large urban school districts and have sought refuge in affluent suburban districts for their own children. But the reality is that there are relatively few pockets of excellence to which these families can flee.

In four states, there is not a single traditional district with average student achievement above the 50th percentile in math. In 17 states, there is not a single traditional district with average achievement in the upper third relative to our global comparison group. And apart from charter school districts,  in over half of the states, there are no more than three traditional districts in which the average achievement would be in the upper third.

The elites in those states have almost nowhere to find an excellent public education for their children. But state accountability systems and the desire to rationalize the lack of quality options have encouraged the elites to compare their affluent suburban districts to the large urban ones in their state. These inappropriate comparisons have falsely reassured them that their own school districts are doing well.

This false reassurance has also perhaps undermined the desire among the elites to engage in dramatic education reform. As long as the elites hold onto the belief that their own school districts are excellent, they have little desire to push for the kind of significant systemic reforms that might improve their districts as well as the large urban districts. They may wish the urban districts well and hope matters improve, but their taste for bold reform is limited by a false contentment with their own situation.

But the elites should not take comfort from the stronger performance of affluent suburban districts relative to large urban districts. As the Global Report Card reveals, even our best public-school districts are mediocre when compared with the achievement of students in a set of countries with developed economies.

Of course, the Global Report Card does not isolate the extent to which schools add or detract from student performance. Factors from student backgrounds, including their parents, communities, and individual characteristics, have a strong influence on achievement. But the GRC does tell us about the end result for student achievement of all of these factors, schools included. And that end result, even in our best districts, is generally disappointing.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute. Josh B. McGee is vice president for public accountability initiatives at the Laura and John Arnold Foundation.

According to the Global Report Card that Josh McGee and I developed, tiny Waconda, Kansas is one of the top-performing school districts in the United States.  Other than being the home to what residents claim is the world’s largest ball of twine (pictured here), one might not think that there was anything exceptional about this rural, farm community in north central Kansas.

But in 2007 the average student in Waconda performed better than 91% of students in our 25 country comparison group in math achievement.  If we relocated Waconda to Finland, the average student in Waconda would outperform 88% of the students in Finland in math.

A reporter for Yahoo News was curious about what they were doing right in Waconda.  Here is what she found:

So why are Waconda kids–65 percent of whom live in poverty–doing so well? And can other schools follow their lead?

The Waconda district comprises four small towns–Cawker City, Downs, Glen Elder and Tipton–and seven schools spread over 411 square miles. Most people in the area work in agriculture or in manufacturing.

The district’s superintendent of seven years, Jeff Travis, told Yahoo News that after years of high test scores, the community expects its students to excel. Most years, he added, no one drops out of high school. The district won 14 state Governor Achievement Awards and one national “Blue Ribbon Award School” over the past four years.

“It’s a tradition now, and they expect themselves to do well,” Travis said. “Like a ball team that continues to win because of a tradition, we have an academic tradition.”

Still, the community doesn’t quite seem to get how exceptional they are. “Everybody’s pretty happy [but] nobody understands how big a deal it is,” he said.

Travis says the students’ high level of achievement is even more extraordinary given that 65 percent of them qualify for free or reduced federal lunches, an indication that they live in poverty. High poverty schools are often dogged by low test scores and high dropout rates. Many educational observers indeed blame the nation’s sky-high child poverty level for the country’s comparatively low performance in math.

One theory Travis has is that Waconda school kids have no sense that they’re materially deprived. “North Central Kansas is rural, and urban poverty is kind of different [from] rural poverty,” he said. “A lot of our people don’t even understand that they’re living in poverty.” According to state data, most of the students are white, and no kids need English language learning classes.

About 10 percent of the students in the school district are foster kids, Travis says. “We just [have] a lot of adults that care about kids, so it’s been a popular thing for parents to take in foster children.”

But Josh adds a useful note of caution at the end of the reporter’s piece:

One of the Global Report Card’s authors, Josh McGee, says the small size of Waconda schools may have skewed the results slightly, since randomness has a greater impact on a smaller sample size. Most of the best-performing school districts in his ranking were small, and many of them were also made up of charter schools. You can read more about his methodology here.

We may not be able to generalize much from the success in Waconda, but it is a fun and impressive story.

Meanwhile, coverage of the Global Report Card continues to stream in.  For an updated list of media for the Global Report Card (with links), click here.

In addition to embodying the spirit of “The Al,” Jobs had quite a lot of smart things to say about education reform.  I’m grateful to Whitney Tilson for reminding me of this.  Here are some selected remarks from Steve Jobs on education:

[On Unions]

I’m a very big believer in equal opportunity as opposed to equal outcome. I don’t believe in equal outcome because unfortunately life’s not like that. It would be a pretty boring place if it was. But I really believe in equal opportunity. Equal opportunity to me more than anything means a great education. Maybe even more important than a great family life, but I don’t know how to do that. Nobody knows how to do that. But it pains me because we do know how to provide a great education. We really do. We could make sure that every young child in this country got a great education. We fallfar short of that…. The problem there of course is the unions. The unions are the worst thing that ever happened to education because it’s not a meritocracy. It turns into a bureaucracy, which is exactly what has happened. The teachers can’t teach and administrators run the place and nobody can be fired. It’s terrible.

[On Vouchers]

But in schools people don’t feel that they’re spending their own money. They feel like it’s free, right? No one does any comparison shopping. A matter of fact if you want to put your kid in a private school, you can’t take the forty-four hundred dollars a year out of the public school and use it, you have to come up with five or six thousand of your own money. I believe very strongly that if the country gave each parent a voucher for forty-four hundred dollars that they could only spend at any accredited school several things would happen. Number one schools would start marketing themselves like crazy to get students. Secondly, I think you’d see a lot of new schools starting. I’ve suggested as an example, if you go to Stanford Business School, they have a public policy track; they could start a school administrator track. You could get a bunch of people coming out of college tying up with someone out of the business school, they could be starting their own school. You could have twenty-five year old students out of college, very idealistic, full of energy instead of starting a Silicon Valley company, they’d start a school. I believe that they would do far better than any of our public schools would. The third thing you’d see is I believe, is the quality of schools again, just in a competitive marketplace, start to rise. Some of the schools would go broke. Alot of the public schools would go broke. There’s no question about it. It would be rather painful for the first several years

DM: But deservedly so.

SJ: But far less painful I think than the kids going through the system as it is right now.

[On Digital Learning]

The market competition model seems to indicate that where there is a need there is a lot of providers willing to tailor their products to fit that need and a lot of competition which forces them to get better and better. I used to think when I was in my twenties that technology was the solution to most of the world’s problems, but unfortunately it just ain’t so… We need to attack these things at the root, which is people and how much freedom we give people, the competition that will attract the best people. Unfortunately, there are side effects, like pushing out a lot of 46 year old teachers who lost their spirit fifteen years ago and shouldn’t be teaching anymore. I feel very strongly about this. I wish it was as simple as giving it over to the computer….

As you’ve pointed out I’ve helped with more computers in more schools than anybody else in the world and I absolutely convinced that is by no means the most important thing. The most important thing is a person. A person who incites your curiosity and feeds your curiosity; and machines cannot do that in the same way that people can. The elements of discovery are all around you. You don’t need a computer. Here – why does that fall? You know why? Nobody in the entire world knows why that falls. We can describe it pretty accurately but no one knows why. I don’t need a computer to get a kid interested in that, to spend a week playing with gravity and trying to understand that and come up with reasons why.

DM: But you do need a person.

SJ: You need a person. Especially with computers the way they are now. Computers are very reactive but they’re not proactive; they are not agents, if you will. They are very reactive. What children need is something more proactive. They need a guide. They don’t need an assistant. I think we have all the material in the world to solve this problem; it’s just being deployed in other places. I’ve been a very strong believer in that what we need to do in education is to go to the full voucher system. I know this isn’t what the interview was supposed to be about but it is what I care about a great deal.

(Source: Smithsonian Institution Oral and Video Histories)

The above interview was from 1995, but it is clear that Jobs did not significantly change his mind over time.  In 2007 he reiterated that unions and lifetime employment for teachers were at the heart of the problem.  This is from PC World:

During a joint appearance with Michael Dell that was sponsored by the Texas Public Education Reform Foundation, Jobs took on the unions by first comparing schools to small businesses, and school principals to CEOs. He then asked rhetorically: “What kind of person could you get to run a small business if you told them that when they came in, they couldn’t get rid of people that they thought weren’t any good? Not really great ones, because if you’re really smart, you go, ‘I can’t win.’ “

He went on to say that “what is wrong with our schools in this nation is that they have become unionized in the worst possible way. This unionization and lifetime employment of K-12 teachers is off-the-charts crazy.”

After Steve Jobs made these comments I wrote an op-ed for the NY Sun, which stated:

There is a price to be paid for this kind of frank analysis and Steve Jobs knows it. “Apple just lost some business in this state, I’m sure,” Mr. Jobs said. Of course, Apple sells a large portion of its computers to public school systems. By taking a stance against school unionization, Mr. Jobs may lose some school sales for Apple.

Sharing the stage with Mr. Jobs was Michael Dell, the chief executive officer of Dell, a competing computer manufacturer. By comparison, according to the description of the event, Mr. Dell “sat quietly with his hands folded in his lap,” during Mr. Jobs’ speech while the audience at an education reform conference “applauded enthusiastically.”

Mr. Dell followed Mr. Jobs by defending the rise of unions in education: “the employer was treating his employees unfairly and that was not good. … So now you have these enterprises where they take good care of their people. The employees won, they do really well and succeed.”

Whether Mr. Jobs or Mr. Dell is right about the role unions have played in public education, one thing is perfectly clear – attacking the unions is a controversial and potentially costly choice for corporate CEOs.

The safe thing is to make bland declarations about the need to improve the quality of education without getting into any of the messy particulars that might be necessary to produce a better education. Changing the status quo in education almost certainly requires ruffling someone’s feathers, but doing that is almost certainly bad for business.

In part this is why we see highly successful entrepreneurs who survive in a world of ruthless competition abandon these business principles when they turn to education philanthropy. People who would never endorse the idea that businesses should be granted local monopolies, offer workers lifetime tenure, or pay employees based solely on seniority, embrace a status quo public system that has all of these features.

While some CEOs may sincerely believe that education is somehow different from the rest of the world in which they live, others have been cowed into submission. Teachers are a very large, well-organized, and relatively affluent consumer and political bloc….

Steve Jobs has embarked on a perilous path, but with solid evidence and persuasive arguments, he can move all of us toward higher quality schools. He should be applauded for having the courage to say out loud what scores of other business leaders are too sheepish to say.

Unfortunately, Steve Jobs will no longer be with us as we try to advance on this perilous path of education reform.

-Jay P. Greene

(Edited somewhat for brevity.  See Jobs’ full interview at Smithsonian Institution Oral and Video Histories)

There are an endless number of interesting stories that could be told with this information, but the one that really stood out to us is that achievement in many of our affluent suburban public school districts barely keeps pace with that of the average student in a developed country.  People who flee from urban education ills thinking that their children will get a top world-class education in the suburbs may be disappointed.  The suburban education is usually better than in the city, but it would may not be preparing students to compete for top paying jobs in an a globalized jobs market.

We highlighted this result in an article in the forthcoming issue ofEducation Next, “When the Best is Mediocre.” The methodological appendix for the GRC can be found here.  In addition, Education Next has a video interview and a podcast discussing this research.

In addition to the discussion of the GRC in Education Next, here is the media coverage to date:

Wall Street Journal (video interview)

Dallas Morning News (subscription required, although a version can be read here)

Arkansas Democrat Gazette (subscription required)

Roll Call (article by Morton Kondracke)

Education Week

United Press International

Milwaukee Journal Sentinel

St. Pete Times

Delaware News-Journal

School Library Journal

My Fox DFW

Dallas Observer

Market Watch

In addition, a number of bloggers wrote about the Global Report Card, including:

Joanne Jacobs

Mackinac Center

Illinois Rising

Ed is Watching

Gotham Schools

Fordham’s Education Gadfly

Bacon’s Rebellion

The Locker Room

The Western Wrangler

Choice Remarks

TPE Post

Missouri Education Watchdog

Whiteboard Advisors

Jorge Werthein

School Finance 101

The last blog post contained some criticisms about whether the assumptions for the analysis were reasonable.  Josh McGee replied in the comment section of that post.  And NCES Commissioner, Jack Buckley, told Education Week that ”The methodology in this report is highly questionable.”  This assessment is a little strange because what we did was similar to what the U.S. Department of Education has done in several past reports linking international test results to state NAEP results.  (See for example this.)  We just bring the results down to the district level.  If ours is highly questionable, then the U.S. Department of Education’s efforts must be at least questionable.

As we write in the methodological appendix:

We make no claims that this Global Report Card is a perfect reflection of school district student achievement relative to international norms. The question is whether the limitations of the Global Report Card are acceptable for a first attempt. In essence, we want to know whether we have more information with the Global Report Card than we would have were it never developed and publicized.

But this nearly exclusive focus on improving the education of the poor has concealed the sub-par education being provided in many of our most affluent school districts.  As the new article Josh McGee and I wrote for Education Next shows, suburban public school districts may look good when compared against their urban neighbors, but when compared with students in 25 other developed countries many affluent suburbs barely keep pace.  That is, our best is often mediocre.

If the children of affluent suburbanites want to maintain their parents’ high standard of living, they need to be performing near the top relative to student overseas with whom they now have to compete for high-paying jobs in an increasingly globalized economy.  Doing better than the kids in big city school districts should provide suburbanites with little comfort.

But this is precisely the comparison we encourage suburbanites to make.  State accountability testing shows suburban districts doing better than the rest of the state, which consists largely of big urban districts.  Policymakers and reformers talk endlessly about the “achievement gap,” highlighting how much worse low-income and minority students are doing.  As Rick Hess recently noted, “our achievement gap mania” has stifled the innovation we need to improve education across the board.

It’s an old saying in public policy that “programs for the poor are poor programs.”  The same is true in education.  If we focus exclusively on improving the education in big cities we fail to engender the support education reform needs from suburban elites if it is to be successful.  As long as suburbanites think that education reform is something for those poor kids in large urban districts, they will never fully commit to the kind and scale of reform that is really needed to improve things in big cities as well as everywhere else.  They’re afraid to muck up what they think is a successful education system for their own children.

As our new Education Next piece shows, this suburban complacency is not well-founded.  Suburbanites need education reform for the sake of their own children and not just for the poor kids in the big cities.  If suburban elites commit to education reform for their own children,we may finally get improvement for low-income kids in the cities as well.

Student achievement in virtually every one of the nearly 14,000 public school districts in the United States compared to students overseas can be found at The Global Report Card’s interactive web site.  With the support of the George W. Bush Institute, we’ve been able to provide this information so that everyone can look up their own and other districts to see that the need for education reform is not confined to big cities.

-Jay P. Greene

The study, “When the Best is Mediocre,” by Jay Greene and Josh McGee, will appear in the Winter 2012 issue of Education Next, and is now available online.

Readers can check out the rankings of 13,636 U.S. school districts, and see how students in each district compare to students in 25 other nations, in a Global Report Card available on the website of the George W. Bush Institute. There readers can also find a detailed explanation of the methods used to conduct the analysis.

There will be an interactive web site containing all of the results.  And Josh and I have an article discussing some important findings from the Global Report Card that will go up on the Education Next web site tomorrow.

Also watch for Laura Bush on the Today Show tomorrow as part of NBC’s Education Nation .

In the past two decades, technology has revolutionized the way Americans communicate, get news, socialize and conduct business. But technology has yet to transform our classrooms. At its full potential, technology could personalize and accelerate instruction for students of all educational levels. And it could provide equitable access to a world-class education for millions of students stuck attending substandard schools in cities, remote rural regions, and tribal reservations.

But then they advocate for a federal government-backed corporation to realize digital learning’s potential:

Too often, the market for educational technology has been inefficient and fragmented. The nation’s 14,000 school districts, more than a few of which have byzantine procurement systems, have been inefficient consumers and have failed to drive consistent demand. And a robust R&D base for improving and refining educational technology has been sadly lacking.

To help remedy those gaps, the Department of Education is launching a unique public-private partnership called Digital Promise.

The last thing digital learning needs is a government funded outfit to develop it. The government is particularly bad at picking technological winners and losers. And if the government pours money into Digital Promise and signals to states and districts that they should adopt what Digital Promise endorses, they will stifle a developing vibrant marketplace that will experiment with different technologies and approaches to learn what work best.

If you don’t believe me that the government is particularly incapable of picking winners and losers in technology, just look at the example of Solyndra. The government poured more than half a billion dollars of stimulus money into Solyndra’s technology for solar energy, believing that it would be the wave of the future. As it turns out, they backed a more expensive technology that failed to win in the marketplace. Solyndra recently declared bankruptcy, laying off more than 1,000 workers and blowing more than half a billion dollars of taxpayer money.

In addition to blowing taxpayer money by backing the wrong technology, Digital Promise is the digital learning equivalent of mandating Betamax. If we privilege the wrong technology we will crowd out better solutions and productive innovation.

Giving taxpayer money to certain outfits also runs the risk of corruption, since political connections may well influence which company and technologies get backed. This leads to Crony Capitalism, or crapitalism.

For the sake of digital learning, Mr. Secretary, please stop “helping” it with a government backed company, like Digital Promise.

(Correction: Digital Promise is a Non-Profit Organization, but all the points still apply)

Notice that I did not include the lack of a national set of standards as a significant barrier to the expansion of digital learning.  I understand that a number of backers of digital learning support the national standards movement because they believe it will allow digital learning providers to achieve scale and offer products in all 50 states without having to contend with 50 different sets of state standards.

But at the recent Harvard conference, Shantanu Prakash, the head of Educomp Solutions, one of the largest digital learning providers in the world, was asked whether different sets of standards were a major obstacle to his company’s operations.  He conceded that the markets in which they operate, principally India, have numerous different standards.  But he also said that this was a trivial barrier because one of the strengths of digital learning is that it typically consists of many small modules that can easily be added or dropped to fit every set of standards.

If backers of digital learning think we need to streamline state regulation to achieve scale, they should be focusing on teacher certification and seat-time requirements rather than standards.  But would any of them really support the idea of having teacher certification and time requirements decided at the national level?  Wouldn’t the opponents of digital learning be able to seize a national regulatory regime to block the expansion of digital learning everywhere?  If so, why is the same concern not true for national standards?

The reality is that the biggest opponent of digital learning will be the teacher unions, who must recognize that digital learning allows cost-savings by replacing labor with capital.  Digital learning backers will have to fight the unions in each state to ease teacher certification, seat-time, and the immobility of funding.  At least they have beach-heads now in states that have a more accommodating regulatory environment.  But if digital learning folks support the construction of a national regulatory regime, they may be marginalized everywhere.

Here is a brief answer to that question: Philanthropists with billions of dollars to devote to education reform should build new institutions and stop trying to fix old ones.

In general, existing institutions don’t want to be fixed.  There are reasons why current public schools operate as they do and the people who benefit from that will resist any effort to change it.  Those who benefit from status quo arrangements also tend to be better positioned than reformers to repel attempts by outsiders to make significant changes.  The history of education reform is littered with failed efforts by philanthropists.

Instead, private donors have had much better success addressing problems by building new institutions.  And competition from newly built institutions can have a greater positive impact on existing institutions than trying to reform them directly.

Let’s consider one of the greatest accomplishments in American education philanthropy.  In the late 19th century, America’s leading universities (Harvard, Yale, Princeton) were badly in need of reform.  They were still operated primarily as religious seminaries and not as modern, scientific institutions.  Rather than trying to reform them directly, major philanthropists built new universities modeled after German scientific institutions.  John D. Rockefeller and Marshall Field helped found the University of Chicago.  Leland Stanford built Stanford University.  A group of private donors built Johns Hopkins.  Cornelius Vanderbilt founded Vanderbilt.  All of these universities imitated German universities with their emphasis on the scientific method and research and were enormously successful at it.  Eventually Harvard, Yale, and Princeton recognized the competitive threat from these German-modeled upstarts and made their own transition from a seminary-focus to a scientific focus.

The reform of the U.S. higher education system did not come from a government mandate or “incentives.”  It did not happen by philanthropists giving money directly to the leading universities of the time to convince them to change their ways.  It happened by philanthropists building new institutions to compete with the old ones.

The same could be done for K-12 education.  Matt Ladner has written a series of posts on “The Way of the Future.”  He, along with Terry Moe, Clay Christensen, Paul Peterson, and others, envision large numbers of  hybrid virtual schools offering higher quality customized education at dramatically lower costs.  Students would attend school buildings, but the bulk of their instruction would be delivered by interactive software.  The school would need significantly fewer staff, who would concentrate mostly on assisting students with the technology and managing behavior.

Obviously, this kind of school would not be good for everybody.  But it could appeal to large numbers of students and be offered at such a low cost that it could be affordable even to low-income families without needing public subsidy or adoption by the public school system.

Gates or someone else with billions to devote to education could build a national chain of these virtual hybrid schools to compete with existing public and private schools.  It’s true that Gates is already investing in the development and refinement of the virtual hybrid school model, but a complete commitment to building new rather than reforming old would give him the potential to do what Rockefeller, Stanford, and others did to higher education.  Virtual hybrid schools could be the disruptive technology, as Christensen calls it, to produce real reform in education.

Another benefit of the “building new” strategy for philanthropists is that it avoids the Emperor’s New Clothes problem, where philanthropists are encouraged to pursue flawed strategies to reform existing institutions because everyone is afraid to criticize the wealthy donor from whose largess they benefit.  With the “build new” strategy there is ultimately a market test of the wisdom of the strategy.  If the new institutions are not better, people won’t choose them.  If the University of Chicago had been a flawed model, it wouldn’t have attracted enrollment and would have failed to apply competitive pressure to Harvard, Yale, and Princeton.  Similarly, if the virtual hybrid school is a bad model, then it won’t attract students and compete with existing public and private schools.

Edison Schools is an example of a “build new” strategy that failed the market test.  They failed to develop technologies or other efficiencies to bring down the costs of operating private schools.  And their revised strategy of operating public schools under contract with public school districts was flawed by an underestimation of the political resistance they would face and their inability to control costs or quality within the public system.

But we also have successful examples of the “build new” strategy adopted by philanthropists.  In addition to the string of scientific universities built in the latter half of the 19th century, we also have the example of Andrew Carnegie and public libraries.  Carnegie helped promote literacy and cultural knowledge by supporting the construction of hundreds of new libraries around the country.  He didn’t try to reform existing book-sellers, he just built new.  Another example (outside of education) can be seen in John D. Rockefeller’s role in the development of a national park system.  Rockefeller privately acquired large chunks of what are now the Acadia, Grand Teton, Great Smoky Mountains and Yellowstone national parks.  Rockefeller didn’t try to reform the operations of the existing Interior Department.  Instead, he effectively privately built nature reserves and then donated them to the U.S. to become national parks.

Of course, this “build new” strategy has limited potential for smaller-scale philanthropy.  But for the very wealthy, like Gates, the path to making a significant and lasting difference is to build new rather than reform old.  The lasting benefits of what Rockefeller did in higher education and national parks and Carnegie did with libraries are still noticeable today.  If Gates and others with billions to devote to education continue to focus on reforming the old rather than building new, I fear their efforts will soon be forgotten after the Emperor’s New Clothes adulation fades when they stop having large sums to give.

-Jay P. Greene

In particular, the Gates interview confirmed two things about the Foundation’s education efforts: 1) they’ve realized that the focus of their efforts has to be on the political control of schools and 2) they are uninterested in using that political influence to advance market forces in education. Instead, the basic strategy of the Gates Foundation is to use science (or, more accurately, the appearance of science) to identify the “best” educational practices and then use political influence to create a system of national standards, curricular materials, and testing to impose those “best practices” on schools nationwide.

The Gates Foundation came to understand the necessity of political influence over schools with the failure of their previous small schools strategy.  Under that strategy they tried to achieve reform by paying school districts to break-up larger high schools into smaller ones.  The problem with that strategy is that even the Gates Foundation does not have nearly enough money to buy systemic reform one school at a time.

School districts currently spend over $600 billion per year and the Gates Foundation only has $34 billion in total assets.  With the practice of spending only about 5% of assets each year and given the large (and effective) efforts the Foundation makes in developing country health-care, Gates only spends a couple hundred million dollars on education reform each year. Given the small share of total education spending Gates could offer, most public districts refused to entertain the Gates strategy of smaller schools, others took the money but failed to implement it properly, and others reversef the reform once the Gates subsidies ended.

The way I described the situation in my chapter “Buckets into the Sea” in the 2005 book, With the Best of Intentions, edited by Rick Hess is:

Philanthropists simply don’t have enough resource to reshape the education system on their own; all their giving put together amounts to only a tiny fraction of total education spending, so their dollars alone can’t make a significant difference.  In order to make a real difference, philanthropists must support programs that redirect how future public education dollars are spent.

And in 2008 I repeated this claim, saying: “total private giving to public education is a tiny portion of total spending on schools.  All giving, from the bake sale to the Gates Foundation, makes up less than one-third of 1% of total spending.  It’s basically rounding error.”

I don’t know whether the Gates Foundation was influence by my writing or whether they arrived at the same conclusions independently, but they are now articulating those same conclusions, often with the same exact words:

“It’s worth remembering that $600 billion a year is spent by various government entities on education, and all the philanthropy that’s ever been spent on this space is not going to add up to $10 billion. So it’s truly a rounding error.”

This understanding of just how little influence seemingly large donations can have has led the foundation to rethink its focus in recent years. Instead of trying to buy systemic reform with school-level investments, a new goal is to leverage private money in a way that redirects how public education dollars are spent.

While the focus of the Gates Foundation on influencing education policy is sensible, the particular political approach they have chosen is doomed to fail and attempting it is likely to be counter-productive.  In Part 2 of this post I will explain how the new strategy Gates has decided to pursue is flawed.

To give you a taste of what is coming in Part 2, the arguments can be summarized as: 1) Education does not lend itself to a single “best” approach, so the Gates effort to use science to discover best practices is unable to yield much productive fruit; 2) As a result, the Gates folks have mostly been falsely invoking science to advance practices and policies they prefer for which they have no scientific support; 3) Attempting to impose particular practices on the nation’s education system is generating more political resistance than even the Gates Foundation can overcome, despite their focus on political influence and their devotion of significant resources to that effort; 4) The scale of the political effort required by the Gates strategy of imposing “best” practices is forcing Gates to expand its staffing to levels where it is being paralyzed by its own administrative bloat; and 5) The false invocation of science as a political tool to advance policies and practices not actually supported by scientific evidence is producing intellectual corruption among the staff and researchers associated with Gates, which will undermine their long-term credibility and influence.

Tune in for Part 2.

-Jay P. Greene

While this may all feel like success to the teacher unions, I suspect that it is actually breeding failure.  The unions succeed by intimidating politicians with their raw power while convincing the public that teacher unions love their children almost as much as the parents do.  Maintaining this double-game is essential because it disarms parents, media elites, and others who might otherwise mobilize against teacher unions and apply their own direct pressure to politicians.

As long as teacher unions act like Mary Poppins to parents, media elites, and others, the general public is willing to suspend their normal inclination to desire choice and competition in the goods and services they consume.  Mary Poppins is an extension of the family and we don’t apply market principles to our family.  The family is a refuge from the rough and tumble of the market which is instead governed by a sense of mutual obligations and affection.  Where the family ends, the market begins and people think the market needs choice and competition to stay healthy.

But when the public face of the teacher unions is the Army of Angry Teachers, they no longer seem like Mary Poppins and begin to look a lot more like longshoremen beating their opponents with metal pipes.  Diane Ravitch and Valerie Strauss may provide psychological comfort to angry teachers (some of whom seem so irate that they may need professional psychological help to manage their anger), but it undermines the double-game that is at the heart of the teacher union strategy.

Giant mobs of yelling protesters and blogs filled with tirades may increase the intimidation politicians feel, but it seriously undermines the image of teachers as an extension of our family.  And as that Mary Poppins image is significantly eroded, media elites and the general public will increasingly think of education as something in the marketplace that requires choice and competition.  And this erosion is extremely hard for teacher unions to reverse.

What feels like success to angry teachers is actually sowing the seeds of failure for the teacher union.

-Jay P. Greene

So, if we believe these OECD numbers (which the WSJ apparently did in this blog post), U.S. teachers work 15.3% more hours per year than do their colleagues in other developed countries.

But if you believe the OECD comparison I have a lovely bridge to sell to you.  According to the report’s methodological appendix, the method by which the U.S. information was collected was different (and clearly less reliable)  than how it was collected from all of the other countries.  In every country except the U.S. the hours worked was derived from teacher contracts or laws.  But in the U.S. the information was drawn from self-reported responses to a survey of teachers.  (See p. 75 of the appendix).

A valid comparison would require that the information be collected in similar ways across all countries — either we rely upon self-reports in surveys of teachers for all countries or we rely on contractual hours for everyone.  But using self-reports for the U.S. and contractual hours for everyone else produces obvious distortions.  People may be inclined to exaggerate the hours they work in a survey.  And the definition of time worked is ambiguous.  If I think about my students while I am brushing my teeth or running on the treadmill am I working during that time?

We have good reason to suspect that the self-reports from U.S. teachers are over-stated.  If teachers really worked 1,913 hours over 180 days, as the report claims, they would be working 10.63 hours per day.  And the numbers I’ve provided are just for primary school students.  For high schools, the OECD report claims U.S. teachers are working 1,998 hours over 180 days, which works out to 11.1 hours per day.  I know some teachers are very conscientious and work long hours but I simply do not believe that the average high school teacher is working 11.1 hours per day.

I know this might invite the wrath of Diane Ravitch’s Army of Angry Teachers, but I suspect that the average hours worked by U.S. teachers is significantly less than the OECD says (and the WSJ repeats).  And I know that the comparison between U.S. and other countries is flawed by collecting the information from self-reports in the U.S. but from contracts everywhere else.

-Jay P. Greene

This point is nicely illustrated by a new report released by Andrew Coulson at Cato today.  Andrew examines academic progress by students in different charter school networks in California.  He then looks at which charter networks receive the most financial backing from philanthropies.  He finds:

The results are discouraging. There is effectively no correlation between grant funding and charter network performance, after controlling for individual student characteristics and peer effects, and addressing the problem of selection bias.

For example, the three highest-performing charter school networks perform dramatically above the level of conventional public schools on the California Standards Tests, but rank 21st, 27th, and 39th in terms of the grant funding they have received, out of 68 charter networks. The AP results are worse; the correlations between charter networks’ AP performance and their grant funding are negative, though negligible in magnitude.

The problem is not that foundations need to be smarter in their giving, although I have written a book chapter on how they could be smarter.  The problem is that foundations are no substitute for market forces in identifying what works and what doesn’t for kids.  Rather than focusing on picking winners and losers, foundations should focus on pushing the idea that we need stronger market forces.  In particular, foundations could back the idea that we need a broader set of options for students (including charters); that whatever public subsidies exist for schools should be equal across all schools in this market; and that schools should be allowed to compete on price as well as quality.  The last item could be achieved with something like educational savings accounts that were recently passed in Arizona, where families could keep any savings between the state subsidy and school costs in an account to be used for future educational needs.  Another option is to allow families to top-off the state subsidy with their own funds.

The point is that foundations need to beware of the corruption that frequently follows the concentration of wealth and power, just like governments.  There is a danger that foundation officials will begin to imagine that they know what people should want, just like government officials, academics, and D.C. pundicrats often do, rather than allowing people to figure out what works for them.  Foundations, like government, can play a useful role by trying to create sensible rules for markets so that they can function efficiently.  They can also alleviate particular suffering and misfortune.  But if they start focusing the bulk of their money on picking winners and losers in the educational marketplace, they are very likely to get it wrong.  Central planning doesn’t work any better for foundations than it does for governments.

I should add that profit-seeking corporations are as prone to corruption as they concentrate wealth and power as are non-profits and governments.  The difference is that, absent government protection, corporations suffer the consequences of this hubris.  If they stray from their mission and become swollen with power-seeking administrative bloat, they tend to lose in the marketplace to leaner, more focused organizations.

The danger for non-profits (and governments) is that there is no similar mechanism of accountability.  If they back foolish ideas or suffer from administrative bloat, they never have to stop as long as they can continue to extract funds without demonstrating effectiveness.

Gigantism in the foundation world is a real problem.  Organizations, including Gates, Ford, Carnegie have the funds to keep foisting foolish ideas on people for a long, long time without any accountability.  And as they get big financially, they get even bigger in their staffing so that they become a self-perpetuating power-seeking bureaucracy, much like the government (except without taking funds by force).

For example, it is interesting to note that over the last decade the Gates Foundation doubled its assets, but it increased its staffing by almost ten-fold.  Empire building afflicts the non-profit and university world almost as much as government.  Wise foundations avoid building empires and focus on promoting sensible rules for efficient market operation as well as the alleviation of misfortune that occurs within markets.

For the most part, organizations are incapable of innovating.  Most organizations are founded with a particular mission and method for pursuing that mission.  If circumstances require that the mission or method be changed, organizations generally can’t do it.  They’ll just keep doing what they were initially established to do until they can no longer continue operating.

Progress occurs not by turning around failing institutions, but by replacing those organizations with new ones that have a better mission and/or method. Of the original 500 companies included in the S&P 500 in 1957 only 74 (15%) exist today as independent companies.  In the private sector, innovation primarily occurs by replacing or fundamentally re-organizing organizations and not by “reforming” them.

And while U.S. real GDP has nearly quintupled since 1970, education achievement of 17 year-olds and high school graduation rates have remained basically unchanged over the same time period.  Perhaps the reason for progress in the economy but not in education stems from our willingness to allow new organizations to replace old ones in the private sector, but not in education.

Public school systems almost never close and the creation of new ones is highly constrained.  Plenty of our public schools are failing, but we almost never admit that they have failed and allow that organization to be replaced with new ones.

Let’s stop trying to fix Detroit, LA, or Chicago public schools.  Let’s let the reality of their failure become official.  They, like most organizations, cannot innovate.  They need to be replaced with new organizations with new missions and new methods of education.  That’s how we can reform schools — by replacing them.

- Jay P. Greene

Our criticism of the nationalization of standards, curriculum, and assessments elicited the following statement from Peter Cunningham, spokesperson for the U.S. Department of Education: “Just for the record: we are for high standards, not national standards and we are for a well-rounded curriculum, not a national curriculum. There is a big difference between funding development of curriculum—which is something we have always done—and mandating a national curriculum—which is something we have never done. And yes—we believe in using incentives to advance our agenda.”

Let’s leave aside the double-speak of how incentivizing is somehow different from mandating.  Instead, let’s focus on his admission that the Department is “funding development of curriculum” and is “using incentives to advance our agenda.”

The 1979 law by which the U.S. Department of Education is authorized in its current form clearly prohibits these activities.  It states (in section 103b): “No provision of a program administered by the Secretary or by any other officer of the Department shall be construed to authorize the Secretary or any such officer to exercise any direction, supervision, or control over the curriculum, program of instruction, administration, or personnel of any educational institution, school, or school system, over any accrediting agency or association, or over the selection or content of library resources, textbooks, or other instructional materials by any educational institution or school system, except to the extent authorized by law.” (emphasis added)

So, the spokesperson for the U.S. Department of Education says that they are funding development of curriculum, but the Department is expressly not authorized to direct, supervise, or control curriculum.  They are are also prohibited from directing, supervising, or controlling textbooks or other instructional materials.

The Department seems to think that it is on solid footing as long as it does not mandate or control curriculum.  But the 1979 law restricts the Department more broadly.  It may not even direct or supervise curriculum.  I have no idea how the Department could fund the development of curriculum without also exercising some direction and supervision over that curriculum.

Nor can the Department justify its current activities by claiming that they are only funding the development of curricular frameworks and instructional materials.  The Department is also explicitly prohibited from directing, supervising, or controlling the content of instructional materials.

As far as I know, no law has specifically authorized the Department to engage in these activities from which they are otherwise prohibited.

I think they have been caught red-handed.

-Jay P. Greene

Charles Miller, the former chair of the Board of Regents of the University of Texas and one of the chief architects of Texas’ accountability system, sent me the following note.  He observed the extensive use of passive voice in the Fordham reply, which serves to conceal who is supposed to be doing the described actions.  He wrote:

So much of the discussion by the advocates of Common Core Standards is filled with references to steps which have to be taken or events which have to take place without identifying specifically how and by whom it happens.  Almost always, there is the implication that some set of elite experts or the federal government will handle what’s necessary with the utmost competence…and virtue—and no unintended consequences, the bane of central planners.  For my own satisfaction, I took this paragraph below from the Gadfly’s response to the Counter-Manifesto [it should always be in [bold]] and tried to fill in specifically who the implementing agents of change will or should be.

Here’s what I came up with:

“So here’s where we stand: First, states should be encouraged [by the federal government's funding lever] to stay the course with the Common Core standards and assessments, at least until we [the federal government] see what the tests look like. While the standards aren’t perfect, they are vastly better than what they are replacing in most states [as judged by the federal government]. Second, à la the Shanker manifesto, efforts should be made [by the federal government] to develop all manner of tools, materials, lesson plans, professional development, curricula, and more that [the federal government determines] will help teachers implement the standards in their classrooms—and to help students master them [as determined by the federal government]. We have no particular concern with the federal government—or philanthropists and venture capitalists, big and small—helping to pay for those activities, as has been done so often in the past [because the federal government never exercises control or significant influence when it spends money]. But, third, it should be made crystal clear  [by the federal government] that the use of all such materials will be completely voluntary for states and, we would argue, for districts within states, schools within districts, and teachers within schools. And fourth, the two consortia now building new Common Core assessments should take pains [with perhaps a loyalty oath to the federal government] not to cross the Rubicon into micromanaging schools’ curricular and instructional decisions.”

I would also add that Fordham’s continued assertion that this entire nationalization project is voluntary is getting downright annoying.  The adoption of the national standards was coerced by making state receipt of federal funds at least partially dependent  on endorsing them.  Fordham did not lift a finger to object to this federal coercion on standards, so why would we believe their new-found conviction that all of this should “be completely voluntary for states”?

Fordham’s credibility in claiming that this nationalization project is voluntary is further undermined by the fact that they recommended that the reauthorized ESEA should:

Require states to back-map achievement standards down to at least third grade, so that passing the state assessment in each grade indicates that a student is on track to graduate from twelfth grade ready for college or a career. States that opt out of the state assessment consortia funded by Race to the Top (RTT) would have their standards peer reviewed at the federal level by a panel of state officials and content-matter experts.

Of course, standards, curriculum, and assessment are all connected.  Once the federal government coerces states to adopt a set of standards, as it has already done without Fordham’s objection, and once states are compelled to adopt a particular set of assessments, as Fordham proposes the federal government should do, then we have a de facto national curriculum regardless of whatever else is done.

The signers of the Counter-Manifesto do not necessarily agree with each other about whether standards, curriculum, and assessments are best handled at the school, district, or state level, but we all agree that centralization to the federal level is undesirable.  Fordham’s facile suggestion that we should find centralization to the federal government acceptable because some of us find centralization to the state level acceptable, assumes that centralization is the same regardless of the level to which power is allocated.  If Fordham is so comfortable with centralization and finds the “hodge-podge of standards, tests, textbooks, curricular guides, lesson plans” so bothersome because it lacks “coherence,” then why wouldn’t they support centralization to the U.N.?  Why should math be any different in Mexico than it is in the U.S.?  A fair number of children cross the border.

The point is that some level of centralization involves the delegation of power to people who are too far removed from the circumstances to be effective, even if they were perfectly benevolent in their exercise of power (which we generally trust less as power is aggregated further).  The signers of the Counter-Manifesto are consistent with the sentiments of the Founders, the legislative authorizers of the Department of Education, and the American people in understanding that education standards, curriculum, and high-stakes assessments should not be done at the national level.

Fordham, in coalition with its friends at Gates, Pearson, AFT, and the US Department of Education are trying to subvert this historical and legal consensus against federal control by failing to be candid about what they are proposing.  That’s why they love the passive voice so much in addition to the use of weasel words.  And that’s why Charles Miller’s clarification of the actor in each sentence is so useful.

- Jay P. Greene

Instead of a straightforward and open defense of their agenda, I anticipate that the U.S. Department of Education-Gates-AFT-Fordham coalition (along with their two hired assessment consortia and corporate backers) will respond with weasel words.  We’ve already seen some hints of their response, so my predictions will not be as impressive as those in the Fordham Report Drinking Game, but feel free to drink nevertheless.

They’ll say that they are not actually advocating a national curriculum.  Instead, they will say that they are only developing  “curricular roadmaps,” “multiple curriculum resources,” ”instructional materials,” “content frameworks,” “model instructional units, “content modules” or similar such weasel words.  Their talking points clearly instruct them to 1) use curriculum as an adjective instead of a noun since “curricular [whatever]” sounds like less than “curriculum,” 2) emphasize the plural so it sounds less uniform, 3) substitute a synonym for curriculum, such as “framework” or “model” so that you avoid clearly stating what you are developing.  Credulous reporters may sometimes buy the claim that these weasel words represent important distinctions, but I suspect that members of Congress are less likely to be as easily fooled when Department of Ed officials are called for hearings to explain the legislative authority by which they are developing a national curriculum.  And I suspect those hearings are not too far in the future.

The nationalization folks may also hide behind the fact that there are two consortia, so clearly they do not desire a single national set of curriculum and assessments.  Having a choice among two federally funded products is a bit like the old joke where you have a choice between death and roo roo.  If you haven’t heard it, you might guess or check this out, but I think you’ll agree that this is hardly a choice.

Or perhaps the U.S. Department of Education-Gates-AFT-Fordham coalition will respond that the consortia are primarily devoted to developing new assessments, not curriculum (or curricular [whatever]).  Just remember that assessment can drive everything else.  Once you have high stakes national assessments you have a de facto national curriculum.

And I am certain that we will hear that the entire enterprise is voluntary.  Of course, there is nothing voluntary about mandating that states and localities comply if they wish to receive Title I funds when ESEA is re-authorized.  If U.S. Department of Education-Gates-AFT-Fordham pledge to do nothing in the ESEA re-authorization or future Race to the Top to reward, incentivize, encourage or otherwise coerce states and localities to adopt the national curriculum and assessments that are being developed, then this claim might have some credibility.  But they haven’t declared this as their position and they won’t do so precisely because they are not seeking a voluntary arrangement.  We have already seen fiscal coercion from the previous round of Race to the Top to get states to adopt the Common Core national standards.  Expect more of this.  And saying that states and localities can choose to forgo federal funds if they don’t wish to comply sounds about as voluntary as saying that paying your income tax is completely voluntary because you can always refuse and choose to go to jail.  Taking money from people and only offering them a share back if they comply is coercion.

The nationalizing coalition uses weasel words because their entire project depends on stealth.  If we have an open and vigorous debate about whether it is desirable for our large, diverse country to have a uniform national set of standards, curriculum, and assessments, I am confident that they would lose.  Time and time again the American people through their political and educational leaders have rejected nationalization of education when it has been proposed in a straightforward way.  Having learned from those failures  the U.S. Department of Education-Gates-AFT-Fordham coalition is trying to advance nationalization with piece-meal steps disguised in weasel words.  With the new Manifesto against nationalization I think we have brought the debate out into the open and the U.S. Department of Education-Gates-AFT-Fordham‘s agenda cannot survive in the open.

- Jay P. Greene

The Manifesto is being announced with 118 original signatories who come from a variety of backgrounds and perspectives.  The list includes former Attorney General Edwin Meese, education professor Joel Spring, law professor Richard Epsein, U.S. Civil Rights Commissioner Abigail Thernstrom, and many more. To see the Manifesto and a full list of those who have endorsed it, click here.  Now that the document is public more names will be added as people add their signatures.

Greg Forster has already penned a short essay on the manifesto, which you can read here.

-Jay P. Greene

UPDATE : Catherine Gewertz at Education Week also has this piece.

See below for the text of the press release:

Broad Coalition Opposes National Curriculum Initiative by U.S. Dept. of Education

Over 100 leaders sign manifesto against nationalization of schooling

Stanford, Calif. & Fayetteville, Ark. – A broad coalition of over 100 educational and other leaders representing diverse viewpoints released a manifesto today opposing ongoing federal government efforts to create a national curriculum and testing system.

The manifesto, entitled “Closing the Door on Innovation,” is available at www.k12innovation.com. It argues that current U.S. Department of Education efforts to nationalize curriculum will stifle innovation and freeze into place an unacceptable status quo; end local and state control of schooling; lack a legitimate legal basis; and impose a one-size-fits-all model on America’s students.

Congress is now preparing to debate renewal of the Elementary and Secondary Education Act, the main law authorizing federal aid to K-12 education. Meanwhile, the U.S. Department of Education has been quietly funding efforts by two assessment groups to develop a national K-12 curriculum, along with a national testing system that tests every public-school student multiple times each year. This federal initiative will create a national system of academic-content standards, tests, and curriculum. It is in line with the goals of a manifesto released on March 7, 2011, by the Albert Shanker Institute that calls for a single nationalized curriculum in every K-12 subject.

“A one-size-fits-all national curriculum based on mediocre high-school standards will stifle the educational innovation essential to closing the racial gap in academic achievement,” said Abigail and Stephan Thernstrom in a joint statement on why they signed the new manifesto. Abigail Thernstrom is vice-chair of the U.S. Commission on Civil Rights and a former member of the Massachusetts Board of Education; Stephan Thernstrom is a professor of history at Harvard University.

“Closing the Door promotes what is for high schools the most important innovation in a century,” said signatory Blouke Carus, leading children’s magazine publisher, math and reading textbook developer, and chairman of the Carus Corporation. “Our schools need to offer each student a choice among six or more challenging and rigorous high school curricula, as do other, higher-performing countries.”

“The federal government’s effort to impose a national curriculum on all schools spells trouble for the educational system,” said Richard Epstein, law professor at New York University, also a signatory. “No one in Washington can craft a curriculum that works well throughout this diverse nation. Once errors are built in at the national level, corrections will be ever more difficult to make at the local level. Only decentralized control over education can prove nimble enough to root out errors and spur innovation. Washington bureaucrats should not trumpet their own omniscience, but should become more cognizant of their own fallibility.”

“To some, a national curriculum sounds like a redemptive cure-all for the shame of our public schools’ failures,” said signatory Shelby Steele of Stanford University’s Hoover Institution. “And a national curriculum gives the education establishment elite a powerful warrant for ‘doing good.’ But we must not discard the proven constitutional discipline of our federalist system. Decentralization has been the engine of educational innovation. We shouldn’t trade our federalist birthright for a national-curriculum mess of pottage.”

“National curriculum becomes, in effect, a nationalization of what teachers teach,” said former Attorney General Edwin Meese III, another signatory. “We must always evaluate policy proposals in light of principles like rule of law and the logic of our constitutional system. The Education Department’s sponsoring and funding of national curriculum runs counter to both laws of Congress and the wisdom of the Founders.”

The coalition of leaders releasing its counter-manifesto today opposes both the Shanker Institute Manifesto and the U.S. Department of Education initiative on a variety of grounds:

• These efforts are against federal law and undermine the constitutional balance between national and state authority.
• The evidence doesn’t show a need for national curriculum or a national test for all students.
• U.S. Department of Education is basing its initiative on inadequate content standards.
• There is no research-based consensus on what is the best curricular approach to each subject.
• There is not even consensus on whether a single “best curricular approach” for all students exists.

With federal education law coming to the top of Congress’s agenda, the U.S. Department of Education’s push to create national curriculum and assessment is becoming a hot topic.

The manifesto opposing a national curriculum was organized by Bill Evers, research fellow at Stanford University’s Hoover Institution; Greg Forster, senior fellow at the Foundation for Education Choice; Jay Greene and Sandra Stotsky, professors at the University of Arkansas; and Ze’ev Wurman, executive at a Silicon Valley start-up.

(Here and in the list of founding signers, all affiliations are given for identification only.)

‘Private school vouchers are not an effective way to improve student achievement,” said the White House in a statement on March 29. “The Administration strongly opposes expanding the D.C. Opportunity Scholarship Program and opening it to new students.” But less than three weeks later, President Obama signed a budget deal with Republicans that includes a renewal and expansion of the popular D.C. program, which finances tuition vouchers for low-income kids to attend private schools.

School reformers cheered the administration’s about-face though fully aware that it was motivated by political expediency rather than any acknowledgment that vouchers work.

When Mr. Obama first moved to phase out the D.C. voucher program in 2009, his Education Department was in possession of a federal study showing that voucher recipients, who number more than 3,300, made gains in reading scores and didn’t decline in math. The administration claims that the reading gains were not large enough to be significant. Yet even smaller positive effects were championed by the administration as justification for expanding Head Start….

The positive effects of the D.C. voucher program are not unique. A recent study of Milwaukee’s older and larger voucher program found that 94% of students who stayed in the program throughout high school graduated, versus just 75% of students in Milwaukee’s traditional public schools. And contrary to the claim that vouchers hurt public schools, the report found that students at Milwaukee public schools “are performing at somewhat higher levels as a result of competitive pressure from the school voucher program.” Thus can vouchers benefit even the children that don’t receive them.

Research gathered by Greg Forster of the Foundation for Educational Choice also calls into question the White House assertion that vouchers are ineffective. In a paper released in March, he says that “every empirical study ever conducted in Milwaukee, Florida, Ohio, Texas, Maine and Vermont finds that voucher programs in those places improved public schools.” Mr. Forster surveyed 10 empirical studies that use “random assignment, the gold standard of social science,” to assure that the groups being compared are as similar as possible. “Nine [of the 10] studies find that vouchers improve student outcomes, six that all students benefit and three that some benefit and some are not affected,” he writes. “One study finds no visible impact. None of these studies finds a negative impact.”

Such results might influence the thinking of an objective observer primarily interested in doing right by the nation’s poor children. But they are unlikely to sway a politician focused on getting re-elected with the help of teachers unions.

“I think Obama and Duncan really care about school reform,” says Terry Moe, who teaches at Stanford and is the author of a timely new book, “Special Interest: Teachers Unions and America’s Public Schools.” “On the other hand they have to be sensitive to their Democratic coalition, which includes teachers unions. And one way they do that is by opposing school vouchers.”

The reality is that Mr. Obama’s opposition to school vouchers has to do with Democratic politics, not the available evidence on whether they improve outcomes for disadvantaged kids. They do—and he knows it.

Fischel’s basic argument is that our educational institutions have largely evolved in response to consumer demands.  That is, the consolidation of one-room schoolhouses into larger districts, the development of schools with separate grades, the September to June calendar, and the relatively common curriculum across the country all came into being because families wanted those measures.  And in a highly mobile society, even more than a century ago, people often preferred to move to areas with schools that had these desired features.  In the competitive market between communities, school districts had to cater to this consumer demand.  All of this resulted in a remarkable amount of standardization and uniformity across the country on basic features of K-12 education.

Hearing Fischel’s argument made me think about how ill-conceived the nationalization effort led by Gates, Fordham, the AFT, and the US Department of Education really is.  Most of the important elements of American education are already standardized.  No central government authority had to tell school districts to divide their schools into grades or start in the Fall and end in the Spring. Even details of the curriculum, like teaching long division in 4th grade or Romeo and Juliet in 9th grade, are remarkably consistent from place to place without the national government ordering schools to do so.

Schools arrived at these arrangements through a gradual process of market competition and adaptation.  Parents didn’t want to move from one district to another only to discover that their children would be repeating what they had already been taught or were  inadequately prepared for what was going to be taught.  To attract mobile families, districts informally and naturally began to coordinate what they taught in each grade.  Of course, not everything is synced, but the items that are most important to consumers often are.

That’s how standardization in market settings works and we have a lot of positive experience with this in industry.  VHS became the standard medium for home entertainment because the market gravitated to it, not because some government authority mandated it.  If we followed the logic of Gates-Fordham-AFT-USDOE we would want some government-backed committee to decide on the best format and provide government subsidies only to those companies that complied.

Instead of ending up with VHS, they may well have imposed Betamax on the country, even though market competition would have shown that approach to be inferior.  Sony was the industry leader and if a government-backed committee were in charge they almost certainly would have had the most influence.  The Fordham folks might want to keep this in mind.  A government-backed committee is almost certain to prefer what the AFT wants over what Fordham may envision since the teacher unions are like Sony except only 100 times more powerful.

Even worse, once government-enforced standardization occurs it becomes extremely difficult to change.  If we had a government-backed panel decide on Betamax, we may have been stuck with that format for decades.  We almost certainly would have stifled the innovation that led to DVDs and now Blue-Ray.  Once Sony had entrenched their format, what incentive would they have had to change it?

Similarly, once the Gates-Fordham-AFT-USDOE coalition settles on the details of nationalizing standards, curriculum, and testing, it will become extremely difficult to change anything about education.  Terry Moe and Paul Peterson’s dreams of technology-based instruction may never leave the dream stage because it may fail to comply with certain provisions of the national regime.  If I were the AFT, I’d almost certainly insert those details into the regime to prevent the reductions that technology may bring to the need for teaching labor.  No one should be naive enough to think the Edublob won’t figure out how to use nationalization to block that and other threatening innovations.

I’m also sure that Bill Gates would have preferred being able to get a government-backed committee to enshrine Microsoft-DOS or Windows forever.  But thanks to market competition we have Google innovating with cloud computing.  And I’d bet that Google would love to get government backing for their approach if they could.  Dominant companies almost always favor government regulation.

So I understand why the AFT, USDOE, and Gates favor the current effort to nationalize education.  The mystery to me is why Fordham is protecting the right-flank of this movement or why some conservative governors have gone along.  Don’t they realize that it will enshrine arrangements that favor the teacher unions and are bad for kids?

-Jay P. Greene

Few merit pay programs for teachers manage to overcome union-fueled political opposition to be adopted and implemented.  We estimate, based on data from Vanderbilt’s National Center for Performance Incentives, that no more than 3.5% of all districts have anything even remotely resembling merit pay.

But even the few programs that aren’t blocked are largely co-opted and diluted so that they are little more than MPINO.  They tend to define merit as additional credentialing, such as paying for national board certification or simply additional degrees.  The bonuses tend to be small add-ons to the traditional salary schedule based entirely on seniority and credentials.  And the plans are frequently not fully implemented or quickly reversed.

The problem is that merit pay programs are trying to simulate the compensation systems that one might develop in a competitive market. But without the pressure and discipline of the market there is nothing to keep these plans sensible or permit the constant tinkering necessary to address gaming or other design weaknesses.  In short, we hold out little hope for merit pay improving achievement in the absence of meaningful choice and competition given the union ability to block, dilute, or co-opt merit pay proposals.

In addition, we suggest that the most powerful form of merit pay is the concern that inadequate performance might cause one to lose one’s job.  Without ending tenure and burdensome fair dismissal procedures, merit pay is unlikely to do much to change a teaching workforce that cannot lose jobs for sub-par performance.

Even if we see more programs that are called merit pay, we are unlikely to get more than MPINO.  Unfortunately, this won’t even result in  SAINO (Student Achievement in Name Only).

-Jay P. Greene

As education policy churns through fad after fad, merit pay is really hot right now. The U. S. Department of Education asked states to include proposals for implementing teacher merit pay—pay based on classroom performance—in their 2010 applications for Race to the Top (RttT) monies, and many applicants promised action on this front. In Washington, D.C., former schools chancellor Michelle Rhee negotiated a strikingly original merit-pay plan, despite strong union opposition. According to the latest Education Next poll, public support for merit pay gained significant ground over the past year and now outdistances opposition by a 2:1 margin.

Replacing the standardized salary schedule, where the only factors that determine teacher salaries are the number of years on the job and academic credentials, seems a worthwhile goal. In theory, pay-for-performance plans both provide a clear monetary incentive to teachers to find the best way to motivate and instruct their students and, over the longer term, attract and retain those more-effective teachers who wish to work in a field that rewards professionals for the quality of their efforts. But enacting high-quality performance pay plans in the United States is easier said than done. Last year, the Florida legislature enacted one of the more stringent proposals any state has ever attempted—only to have the bill vetoed by Governor Charlie Crist as a way of jump-starting his ultimately doomed bid to become Florida’s first independent U.S. senator. That is not the only time a merit pay bill has seemed on the verge of success, only to founder or be undermined by the need to compromise. In general, merit pay plans are more likely to be symbolic than substantive and more likely to be promised than delivered.

Most often, they are not even promised. Even if one counts the most token of performance pay plans, they are to be found in no more than 500 school districts out of some 14,000 districts nationwide, a mere 3.5 percent of the total.

When new merit-pay plans are proposed, teachers unions often block their enactment or water down their provisions. In Cincinnati and Philadelphia, for example, merit pay policies were blocked just before they were about to be implemented. Denver’s Professional Compensation for Teachers (ProComp) plan, widely heralded as the leading national example of performance pay, awards more money for earning another degree than for demonstrated performance in the classroom. In Houston, merit was defined so broadly that it included an overwhelming majority of the teachers. In Florida, Iowa, and Texas, the legislatures have encouraged local districts to enact performance pay plans. But unions have been able to dissuade local districts from participating in the state-authorized programs. Only a handful of Florida districts participate in merit pay, for example, even though state funds cover the cost of the initiative.

A strong, well-designed merit-pay plan requires more than offering a bonus to high-performing teachers while paying the remainder according to the standard schedule. To be truly effective, pay for performance must mean in education what it does in other industries—salary increases for the successful, and salary reductions, even dismissals, for poor performers. State laws governing teacher tenure in most states make implementation of such plans unlikely.

All of this leads us to measured skepticism about the merit of merit pay, unless it is coupled with school choice innovations hefty enough to instigate sustained competition among schools and school sectors. Only then would local districts have the incentive to both lobby states for changes in state laws and to negotiate tough contracts with teacher unions. Only then would they find it important, if merely to retain their student enrollments, to structure their pay systems so as to attract top-notch employees and give them strong incentives to strive for excellent performances.

But we have covered a lot of ground very quickly. Let’s step back and consider carefully the propositions we have set forth.

Does It Work?

High-quality research on this topic within the United States is sparse and results are mixed. Matt Springer and his colleagues at Vanderbilt released a study recently on a well-designed randomized trial of a merit pay experiment in Nashville. The program involved bonuses of up to $15,000, which would presumably be large enough to affect individual incentives. Yet virtually no effect was seen on test scores (outside of 5th-grade math, an effect that disappeared for those same children the next year). That said, the Nashville study did not examine long-term effects on the composition of the teacher workforce.

The Bloomberg administration in New York City made headlines in late 2007 by announcing a pilot merit-pay initiative, the School-Wide Performance Bonus Program. The New York City Department of Education randomly assigned eligible schools to treatment or control groups, which has enabled scholars to conduct rigorous evaluations. Early results with respect to student achievement are not promising overall, although the program appears to have had a positive impact in schools with fewer teachers (see “Does Whole-School Performance Pay Improve Student Learning?” research). The researchers theorize that the group benefit feature of the merit pay program made it unlikely that it would have an impact on teacher behavior in any but the smallest schools.

The international evidence on performance pay is more encouraging, including a recent worldwide look that indicates that students learn more in countries with performance pay plans, all other known factors held constant. Ludger Woessman (see “Merit Pay International,” research) looked at 27 Organisation for Economic Cooperation and Development (OECD) countries and found that students in countries with some form of performance pay for teachers score about 25 percent of a standard deviation higher on the international math test than do their peers in countries without teacher performance pay.

Union Roadblocks

If merit pay seems promising (and certainly not harmful), convincing tests of its performance are difficult to undertake within the United States, simply because merit pay proposals typically end up being blocked, co-opted, or diluted by established interests. Admittedly, it is not easy to identify the various instances where merit pay has been proposed but then blocked from enactment, and therefore we cannot provide an explicit enumeration. In all likelihood, most potential proposals are never articulated, simply because likely sponsors regard the cause as hopeless. When 96.5 percent of all districts rigidly follow a standard salary schedule, it takes an energetic and devoted innovator to brave the odds and try to break from tradition nonetheless.

Still, there are several telling examples of established interests blocking merit pay proposals. Governor Mitt Romney proposed merit pay in Massachusetts back in 2005–06, as part of an education budget that included tens of millions in new spending. That proposal went down to defeat; as the Lowell Sun reported, “the Massachusetts Teachers Association [MTA] and United Teachers of Lowell opposed the idea. Catherine Boudreau, president of the MTA, called teacher bonuses ‘inequitable and divisive.’”

Philadelphia tried to institute a pilot merit-pay program in 2000, but later ditched the initiative, “calling it too expensive, too difficult to administer, and a failure at giving teachers useful feedback” according to the Philadelphia Inquirer. Then, in 2006, Philadelphia received a $20.5 million grant from the U.S. government to develop a merit pay program. Said the Inquirer, “At the time, the federal grant was announced with much fanfare—the union would be the district’s partner, officials said, ensuring the plan would succeed where others failed. But the deal fell apart.” The local union abandoned the program in the face of a “surprise $180 million budget deficit,” and the district gave the money to charter schools instead.

Another example comes from Cincinnati. That city’s merit-pay plan proposed in 2002 was overwhelmingly voted down by teachers (1892 to 73), even though it did not base bonuses on student test scores. As Education Week noted, the plan “was based on an extensive evaluation system, which determines whether teachers advance in five career categories…. The evaluations entail multiple classroom observations by fellow teachers and administrators and portfolios that include logs of parent contacts, lesson plans, student work, and more.” Education Week quoted a former associate superintendent of the Cincinnati schools, who blamed the proposal’s failure on the fact that it “would have applied to nearly all teachers, rather than allowing veterans the choice of opting into the new system.”

In Alabama, the state’s “Race to the Top” application originally proposed merit pay and a “new salary schedule that would give more money to math, science and special-education teachers,” but that portion of the application was deleted, reported the Press-Register (Mobile), “after Alabama Education Association leader Paul Hubbert wrote state Superintendent Joe Morton a letter…opposing them”

If special interests fail to block a merit pay program, they may still be able to make it temporary. A Little Rock, Arkansas, performance-pay program lasted only three years and was not renewed by the local school board, despite evidence of positive effects on student achievement in math, reading, and language. Similarly, the Alaska School Performance Incentive Program was canceled after three years.

Special interests are also able to repeal merit pay based on putative budgetary constraints. The state of North Carolina suspended incentive awards to high-performing schools in 2008–09 due to budget problems. Winston-Salem/Forsyth County, North Carolina, suspended its bonus program due to budget difficulties as well.

Unions have been similarly successful at preventing local districts from participating in statewide programs, as the experience in Florida, Iowa, and Texas shows. Florida’s “Merit Award Program” provides state money to local school districts. According to the Florida Department of Education, “Each district will determine an amount equal to at least 5% and no more than 10% of that district’s average teacher salary to be awarded to all of the top performing personnel in the district, regardless of years of experience.” Even though this program involves free money from the state for districts to hand out to teachers, the political forces opposing merit pay were able to prevent 88 percent of Florida districts from participating in 2009.

Similarly, Iowa’s statewide Career Ladder and Pay-for-Performance grant program was passed in 2007, but only 3 Iowa districts, out of 360, bothered to apply. Only 20 percent of Texas districts opted into the District Awards for Teacher Excellence program in 2009–10. In other words, as a result of political opposition, the vast majority of school districts, even in conservative Texas, turned down extra money from the state rather than adopt merit pay.

Merit Pay in Name Only

When interest groups succeed in diluting or co-opting a merit pay plan, the plan ends up rewarding teachers mostly or entirely for inputs (e.g., professional development, graduate degrees, national certification) rather than for outputs (test scores, graduation rates, or even supervisor assessments).

One example is Arizona’s Classroom Site Fund (CSF), a mandatory statewide program that involved a couple of new taxes. Districts had to “allocate forty per cent of the monies for teacher compensation increases based on performance and employment related expenses, twenty per cent of the monies for teacher base salary increases and employment related expenses and forty per cent of the monies for maintenance and operation purposes.”

According to a 2010 report from the Arizona Auditor General, out of 222 districts receiving CSF funding, the auditor could identify only 29 “with strong performance pay plans that did a good job of linking teacher performance pay to student achievement.” The report noted that “allowing districts the freedom to determine performance pay goals can help gain district and teacher buy-in,” but that such freedom “has also led to inconsistent performance pay plans and to situations in which teachers receive similar performance pay for significantly different levels of effort and related performance results.”

One example from the auditor’s report deserves to be highlighted:

One district awarded performance pay to eligible employees if freshman students’ algebra test scores increased by at least 10 percent between a pre- and post-test. The actual increase in test scores was almost 90 percent. Since the pre-test is given to freshman students who have never been exposed to algebra and the post-test is given to them after receiving a full year of algebra instruction, it should be expected that scores would increase significantly more than 10 percent.

In other words, algebra teachers were being rewarded merely for getting students to learn 10 percent more about algebra than they knew before studying that subject at all. This is not a high hurdle to clear.

Denver’s ProComp program has been heralded as a political and policy success. Then Senator Barack Obama said, “Cities like Denver have already proven that by working with teachers, this can work, that we can find new ways to increase pay that are developed with teachers, not imposed on them and not just based on an arbitrary test score.” But the Denver ProComp program may be less than meets the eye. For one thing, it exempts teachers hired before January 1, 2006, from having to join, which means that the vast majority of teachers whose pay depends on seniority rather than on merit are able to keep their old pay structure in place. And if older teachers opt to enter the ProComp program, they keep their old base salary; the ProComp program merely offers them a chance for bonuses on top of that old salary.

The ProComp program also rewards the old definition of “merit” more immediately and to a greater extent than it does anything that improves student achievement. The largest monetary award is for earning a graduate degree: a $3,300 permanent salary increase plus a tuition or student loan subsidy of $1,000 per year for up to four years. By comparison, teachers receive a one-time award, not a bump up in base salary, of up to $2,403.26 if their students exceed “district expectations” for student growth.

Moreover, as Paul Teske, a principal evaluator of the ProComp program, noted in the Christian Science Monitor, bad teachers face no penalty under the ProComp or similar merit-pay programs: “I guess your salary stays low, and maybe that sends the message that you should look at another career. But ProComp doesn’t directly address that.”

The federal Teacher Incentive Fund (TIF) provides grants to school districts that promise to develop merit pay programs “for raising student achievement and for taking positions in high-need schools.” Currently, the Department of Education’s website lists 33 TIF grantees, including some small districts and a few major city districts. But these programs may also end up being diluted or co-opted.

For example, the TIF program in Charlotte-Mecklenburg (North Carolina) includes substantial bonuses for professional development, working at hard-to-staff schools or in hard-to-staff subjects, and for taking on leadership roles. To the extent the program involves student achievement, it bases awards on “student learning objectives” as “created by individual teachers, with the approval of site-based administrators”; these objectives “will be measured by a combination of existing assessment instruments, and teacher designed tools,” as well as by state standardized tests. The superintendent of Charlotte-Mecklenburg schools recently announced a plan to bring performance pay to the entire district.

Perhaps it is desirable to have teachers receive more professional development, work in hard-to-staff schools or subjects, and assume leadership roles, but these are inputs, not student outcomes. The bulk of the Charlotte-Mecklenburg TIF program, like many such programs, is MPINO—merit pay in name only.

Some locales have diluted the merit pay concept by making the bonuses to teachers small and setting the bar for receiving the bonuses low, thereby converting merit pay into something approximating an across-the board pay raise.

For example, the Texas Educator Excellence Grant (TEEG) program began in 2006–07 and ended after the 2008–09 year; it was funded at approximately $100 million per year. After analysts at the National Center on Performance Incentives (NCPI) reported no positive effects on student test scores, the Dallas Morning News declared the program a failure. NCPI report coauthor Lori Taylor speculated that “one possible cause of the program’s failure was that bonuses were relatively small and were given to most teachers at each school—about 70 percent—so that the incentive for individual teachers to push for higher scores was ‘relatively weak.’ In addition, campuses that qualified already had to be higher performers, so it was difficult to register much improvement.”

The same thing seems to be happening in Houston, where a merit pay program has existed since January 2007. The district announced financial awards totaling $40.4 million in 2010. The district’s webpage notes, “in all, 15,688 HISD employees received performance pay [in 2010], ranging from $25 to $15,530. That’s 88 percent of eligible HISD employees.”

Minnesota’s oft-heralded “Q Comp” program offers yet another example of a “merit pay” program that ends up as an across-the-board pay raise. As the Minneapolis Star Tribune recently reported, “In 22 school districts whose Q Comp practices were analyzed by the Star Tribune in 2009, more than 99 percent of teachers in the program received merit raises during the preceding school year. Only 27 of the roughly 4,200 teachers eligible did not get a pay raise.”

The New York City School-Wide Performance Bonus Program mentioned above may also have been undermined by its structure. Some 180 schools were eligible in the 2007–08 school year for a collective $14 million in bonuses, or $3,000 per union teacher, if they met test score goals established by the district. In a key factor that enabled the plan to draw union support, committees composed of a principal, a person of the principal’s choosing, and two union representatives were allowed to decide how the bonuses should be distributed at any given school. Researchers identified a number of drawbacks to the program design, including the possibility that bonuses based on school-wide improvements weaken the incentives for individual teachers to increase their efforts.

Making Merit Pay Work

The prospects for merit pay are not promising, despite both theoretical and empirical reasons for expecting that the programs would produce positive results for students. Our findings are consistent with the theory that school districts are not primarily educational institutions where policies are organized around maximizing student achievement. Instead, they are best understood, at least when it comes to compensation policies, as political entities shaped by powerful interest groups, including organized groups of employees.

Viewed in that light, it is unsurprising that public school systems have relatively little interest in authentic merit-pay programs. If some teachers could earn improvements in their wages and working conditions from their own efforts rather than from the efforts of their organized representatives or affiliated politicians, then more-effective teachers would have little reason to support the unions financially or politically. Their interests would be at odds with those of less-effective teachers. In short, the single salary schedule by which almost all public school teachers are paid is essential to the financial and political power of established interests.

One way to diminish the power of established interests and permit the adoption and implementation of merit pay is to expand choice and competition in education. If students choose their school, those schools have incentives to adopt and implement policies and practices that will improve their quality and attract students as well as the resources they generate. If merit pay systems help attract and motivate effective teachers, schools in a more competitive environment will have incentives to adopt those systems. They are more likely to design and maintain merit pay systems in a sensible way, since their revenue depends on it.

Schools that already compete for students appear more open to including merit pay in their personnel policies. According to University of Washington’s Daniel Goldhaber and his colleagues, charter schools are more likely than traditional public schools to use merit pay. Michael Podgursky, professor of economics at the University of Missouri, looked at data from the 1999–2000 Schools and Staffing Survey and found that when school administrators were asked whether they used salaries to reward “excellence,” only 6 percent of traditional public school administrators answered yes, while “the rates for charter (36 percent) and private schools (22 percent) were much higher.” Even those charter and private schools without a formal performance-pay plan are typically able to offer higher salaries to teachers they hope to retain and, as important, to readily dismiss teachers deemed ineffective.

Attaching continued employment and level of compensation to job performance is something that frequently occurs among private enterprises in competitive markets. The difficulty with merit pay in education is that it attempts to simulate a market-based practice in a nonmarket environment. None of the forces that cause organizations to seek effective merit pay systems, or to maintain and alter them effectively over time, exist in public education.

Imposing merit pay on an unwilling education system is like trying to get kids to eat their vegetables when the kids are 25 years old and stronger than their parents. No matter how nutritious green beans may be, powerful adults who don’t want to eat them can usually keep them off their plates and can almost always keep them out of their mouths.

Stuart Buck is a doctoral fellow in education reform at the University of Arkansas. Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute.

More than at any other time in history, mankind faces a crossroads. One path leads to despair and utter hopelessness. The other, to total extinction. Let us pray we have the wisdom to choose correctly.

In one essay, Paul Peterson, Marci Kanstoroom, and Chester Finn reject my rosy assessment of progress in the war of ideas about education reform, saying “It’s way, way too early to declare victory. Atop the cliffs and bastions that reformers are attacking, the opposition has plenty of weapons with which to hold its territory…. It’s dangerous to think a battle is over when it has just begun.”

In the other essay, Frederick Hess, Martin West, and Michael Petrilli go even further in their gloom, arguing not only that the war has hardly begun, but that the reform warriors are really the enemy:

First, reform “support” resides with a mostly uninformed, unengaged public—one that isn’t especially sold on their ideas and that, in any event, is often outmatched by well-organized, well-funded, and motivated special interests. And second, and more unfortunately, many reformers are eagerly overreaching the evidence and touting simplistic, slipshod proposals that are likely to end in spectacular failures. In short, some forces of reform are busy marching into the sea and turning notable victories into Pyrrhic ones. To quote that wizened observer of politics and policy, Pogo: We’ve met the enemy, and he is us.

That’s funny.  I thought the enemy was a monopolistic, bureaucratized 19th century school system propped up by teacher unions and their allies who place the interests of adults over the needs of children.  I guess I was wrong in not understanding that it is really the opponents of that system who are the problem.

In truth, I don’t really disagree with much of what either essay has to say.  It is all just a matter of emphasis and framing. In my declaration of victory I was careful to acknowledge that the war over policy has barely begun and reformers have a long and difficult road ahead:

We won!  At least we’ve won the war of ideas.  Our ideas for school reform are now the ones that elites and politicians are considering and they have soundly rejected the old ideas of more money, more money, and more money.

Now that I’ve said that, I have to acknowledge that winning the war of ideas is nowhere close to winning the policy war.  As I’ve written before, the teacher unions are becoming like the tobacco industry.  No one accepts their primary claims anymore, but that doesn’t mean they don’t continue to be powerful and that people don’t continue to smoke.  The battle is turning into a struggle over the correct design and implementation of the reform ideas that are now commonly accepted.  And the unions have shown that they are extremely good at blocking, diluting, or co-opting the correct design and implementation of reforms.

Rick Hess correctly demonstrated how important design and implementation are almost two decades ago in his books, Spinning Wheels and Revolution at the Margins.   And it is always useful for him and others to remind reformers of the dangers that lurk in those union-infested waters.  But for a moment can’t we just bask in the glow of our intellectual victory — even if our allies are a new crop of naive reformers?

Yes, there is a danger in thinking that the policy war is over when it has barely begun.  And yes, there is a danger in over-promising and over-simplifying reform ideas.  But there is also a danger in reform burn-out.  The struggle over school reform has been going on for decades and will almost certainly take several decades more.  Donors have grown frustrated and advocates have jumped to ill-conceived quick fixes that would set the cause of reform back significantly, like adopting national standards and assessments.  If we don’t periodically note our policy progress and intellectual victories, we will have great difficulty sustaining the reform movement.

My view does not really differ substantially from the two essays in Education Next except that they see a greater danger in over-confidence and I see a greater danger in burnout.  And I don’t mind being used as the straw man for their arguments.  The Straw Man had a brain.

-Jay P. Greene

That’s true, but I am now more aware of the opposite failing — believing that we can resolve all policy disputes and identify the “right way” to educate all children solely by relying on science.  Science has its limits.  Science cannot adjudicate among the competing values that might attract us to one educational approach over another.  Science usually tells us about outcomes for the typical or average student and cannot easily tell us about what is most effective for individual students with diverse needs.  Science is slow and uncertain, while policy and practice decisions have to be made right now whether a consensus of scientific evidence exists or not.  We should rely on science when we can but we also need to be humble about what science can and can’t address.

I was thinking about this while reflecting on the Gates Foundation’s Measuring Effective Teachers Project.  The project is an ambitious $45 million enterprise to improve the stability of value-added measures while identifying effective practices that contribute to higher value-added performance.  These are worthy goals.  The project intends to advance those goals by administering two standardized tests to students in 8 different school systems, surveying the students, and videotaping classroom lessons.

The idea is to see if combining information from the tests, survey, and classroom observations could produce more stable measures of teacher contributions to learning than is possible by just using the state test.  And since they are observing classrooms and surveying students, they can also identify certain teacher practices and techniques that might be associated with greater improvement.  The Gates folks are using science to improve the measures of student progress and to identify what makes a more effective teacher.

This is a great use of science, but there are limits to what we can expect.  When identifying practices that are more effective, we have to remember that this is just more effective for the typical student.  Different practices may be more effective for different students.  In principle science could help address this also, but even this study, with 3,000 teachers, is not nearly large enough to produce a fine-grained analysis of what kind of approach is most effective for many different kinds of kids.

My fear is that the researchers, their foundation-backers, and most-importantly, the policymaker and educator consumers of the research are insensitive to these limitations of science.  I fear that the project will identify the “right” way to teach and then it will be used to enforce that right way on everyone, even though it is highly likely that there are different “right” ways for different kids.

We already have a taste of this from the preliminary report that Gates issued last month.  Following its release Vicki Phillips, the head of education at the Gates Foundation, told the New York Times: “Teaching to the test makes your students do worse on the tests.”  Science had produced its answer — teachers should stop teaching to the test, stop drill and kill, and stop test prep (which the Gates officials and reporters used as interchangeable terms).

Unfortunately, Vicki Phillips mis-read her own Foundation’s report.  On p. 34 the correlation between test prep and value-added is positive, not negative.  If the study shows any relationship between test prep and student progress, it is that test prep contributes to higher value-added.  Let’s leave aside the fact that these were simply a series of pairwise correlations and not the sort of multivariate analysis that you would expect if you were really trying to identify effective teaching practices.  Vicki Phillips was just plain wrong in what she said.  Even worse, despite having the error pointed out, neither the Gates Foundation nor the New York Times has considered it worthwhile to post a public  correction.  Science says what I say it says.

And this is the greatest danger of a lack of humility in the application of science to public policy.  Science can be corrupted so that it simply becomes a shield disguising the policy preferences of those in authority.  How many times have you heard a school official justify a particular policy by saying that it is supported by research when in fact no such research exists?  This (mis)use of science is a way for authority figures to tell their critics, “shut up!”

But even if the Gates report had conducted multivariate analyses on effective teaching practices and even if Vicki Phillips could accurately describe the results of those analyses, the Gates project of using science to identify the “best” practices is doomed to failure.  The very nature of education is that difference techniques are more effective in different kinds of situations for different kinds of kids.  Science can identify the best approach for the average student but it cannot identify the best approach for each individual student.  And if students are highly varied in their needs, which I believe they are, this is a major limitation.

But as the Gates Foundation pushes national standards with new national tests, they seem inclined to impose the “best” practices that science identified on all students.  The combination of Gates building a national infrastructure for driving educator behavior while launching a gigantic scientific effort to identify the best practices is worrisome.

There is nothing wrong with using science to inform local practice.  But science needs markets to keep it honest.  If competing educators can be informed by science, then they can pick among competing claims about what science tells us.  And they can learn from their experience whether the practices that are recommended for the typical student by science work in the particular circumstances in which they are operating.

But if the science of best educator practice is combined with a national infrastructure of standards and testing, then local actors cannot adjudicate among competing claims about what science says.  What the central authorities decide science says will be infused in the national standards and tests and all must adhere to that vision if they wish to excel along these centralized criteria.  Even if the central authority completely misunderstands what science has to say, we will all have to accept that interpretation.

I don’t mean to be overly alarmist.  Gates has a lot of sensible people working for them and there are many barriers remaining before we fully implement national standards and testing.  My concern is that the Gates Foundation is being informed by an incorrect theory of reform.  Reform does not come from science identifying the right thing to do and then a centralized authority imposing that right thing on everyone.  Progress comes from decentralized decision-makers having the freedom and motivation to choose among competing claims about what is right according to science.

- Jay P. Greene

Addendum

I just wanted to add a few thoughts to my post yesterday.  Readers may be wondering what is wrong with using science to identify the best educational practices and then implementing those best practices.  If they are best, why wouldn’t we want to do them?

Let me answer by analogy.  We could use science to identify where we could get the highest return on capital.  If science can tell us where the highest returns can be found, why would we want to let markets allocate capital and potentially make a lot of mistakes?  Government could just use science and avoid all of those errors by making sure capital went to where it could best be used.

Of course, we tried this approach in the Soviet Union and it failed miserably.  The primary problem is that science is always uncertain and susceptible to corruption.  We can run models to measure returns on capital, but we have uncertainty about the models and we have uncertainty about the future.  Markets provide a reality test to scientific models by allowing us to choose among competing models and experience the consequences of choosing wisely or not.  Science can advise us, but only choice, freedom, and experience permit us to benefit from what science has to offer.

And even more dangerous is that in the absence of choice and competition among scientific models, authorities will allow their own interests or preferences to distort what they claim science has to say.  For an excellent example of this, check out the story of Lysenko and Soviet research on genetics.  For decades Soviet science was compelled to believe that environmental influences could be inherited.

Science facilitates progress through the crucible of market tests.  Science without markets facilitates stronger authoritarianism.

There is always something arbitrary and crappy about these rankings, but Rick is right when he argues, “For all their imperfections, I think these [ranking] systems convey real information—and do an effective job of sparking discussion (about questions that are variously trivial and substantial).”  Recognizing that these kinds of rankings are part recreation and part reality, I’ve made a slightly revised ranking presented below (with help from Misty Newcomb).

One of the problems with the ranking Daniel and Rick developed is that it combines some measures that accumulate over one’s career with other measures that only count accomplishments in the last year.  The career measures, Google Scholar and books published, will tend to be higher for people who have had longer careers.  Given that the ranking is meant to capture the current influence of education academics, these career items are biased in favor of senior scholars whose work may have been influential in the past, but less so in the present.

A more junior colleague pointed out this distortion to me, so I have tried to standardize the Google Scholar and book measures so that those with longer careers would have no particular advantage.  In particular, I calculated the sum of the two “career measures” — Google Scholar and books published.  Then I divided that sum by the years since the scholar received his or her terminal degree.  And to ensure that books and articles would still have the same weight in the overall score, I multiplied by the mean number of years since degrees were earned, about 23.2.

In making this adjustment I am assuming that every scholar would maintain the same rate of book and article productivity over his or her entire career.  So, the book and article “public presence” in the past year would be in proportion to the total book and article production per year over an entire career.

I make no changes to the 5 other measures in Daniel and Rick’s ranking: current Amazon sales as well as mentions in the education press, blogs, newspapers, and Congressional Record.  All of those measures reflect current “public presence.”  Adding the adjusted two career measures to these annual measures we get an adjusted total score.

Making the adjustment for length of career does not alter who is at the very top of the rankings.  As you can see below, Diane Ravitch and Linda Darling-Hammond still rule the roost.  But there are some significant changes below that, where more junior scholars jump in the rankings and more senior scholars drop.  For example, Martin West leaps to 10th place from his previous ranking of 69th, surpassing his mentor, Paul Peterson, who drops from 5th to 11th.  Roland Fryer moves up to 3rd from 11th.  Jacob Vigdor rises to 16th from 43rd.  Susanna Loeb goes to 18th from 49th.  Matthew Springer rises to 29th from 74th.  And Brian Jacob, Jonah Rockoff, and Sara Goldrick-Rab all jump almost 30 places.

On the other hand, some more senior scholars decline significantly in their public presence ranking once we make this adjustment.  Gene Glass sinks from 20th to 50th.  Henry Levin falls from 17th to 52nd.  David Berliner drops from 19th to 57th.  Kenneth Zeichner moves from 30th to 62nd .

These changes make sense and I think improve Rick and Daniel’s ranking.  Hotshot researchers like Roland Fryer, Jacob Vigdor, Susanna Loeb, Matthew Springer, Brian Jacob, Jonah Rockoff, and Sara Goldrick-Rab are having a large impact on current education policy discussions even though their careers have not been long enough to accumulate a longer list of books and articles.  The original ranking shortchanged these scholars in measuring their current “public presence.”

At the same time, more senior scholars, like Gene Glass, Hank Levin, David Berliner, and Kenneth Zeichner may have been given too much credit by the old ranking system for books and articles that were influential in the past but do not give them as much of a public presence in recent policy debates.

Of course, of greatest interest to me was what happened to my ranking.  I moved up to 21st from 39th.  This must be a better ranking.

Click on the images below to see the original and adjusted results for all 89 education academics that Rick and Daniel included in their “super-sized” ranking.  Have fun and, as David Letterman would say, please… no wagering.

If we were talking about the 1960s, I could easily offer Jonathan Kozol’s Death at an Early Age as the articulation of that era’s strategy of increasing resources devoted to education, particularly for minority students. The revival of progressive education, with open classrooms, student-centered learning, and whole language, which was all the rage in the 1970s, could be found in a few influential books of that time. Neil Postman and Charles Weingartner’s Teaching as a Subversive Activity and Charles Silberman’s Crisis in the Classroom come to mind. If we were talking about the 1980s and the growth of the standards and accountability movement, we could credit E. D. Hirsch’s Cultural Literacy. And the case for school choice was laid out in the 1990s by John Chubb and Terry Moe’s Politics, Markets, and America’s Schools.

The first decade of the 21st century has also had a dominant strategy: incentive-based reforms, such as increasing competition among charter and district schools, merit-pay plans to improve teacher quality, and school-level accountability based on testing. But no single book or set of books stands out as the voice of these reforms.

Rather than articulating a broad, theoretical case for reforms that have been embraced by policymakers, the books of the “aughts” were more likely to engage in debates over evidence, articulate a strategy that had not been adopted, or do battle against the strategies that policymakers did adopt. (See the results of a web poll that invited readers to vote for their favorite education books.)

My own book, Education Myths, may have bolstered efforts to enact the incentive-based reforms that dominated the decade, but it did not provide the conceptual rationale for the movement. William Howell and Paul Peterson’s Education Gap was more a review of the evidence from voucher experiments than it was a call to arms for incentive-based reforms. Eric Hanushek and Alfred Lindseth’s Schoolhouses, Courthouses, and Statehouses and Frederick Hess’s Common Sense School Reform both make a case for incentive-based reforms, but they are also primarily reviews of the current research rather than the articulation of a new reform strategy.

Some books from the aughts did make theoretical arguments for new reforms, but those reforms have not been embraced by policymakers, at least not yet. Terry Moe and John Chubb’s Liberating Learning, Paul Peterson’s Saving Schools, and Clayton Christensen et al.’s Disrupting Class all make the case for technology-based schools that substitute computers for human instruction. Someday that may be the dominant education-reform strategy, but that day is not today.

The most common type of education reform book from the period argued against the dominant strategies. Diane Ravitch’s The Death and Life of the Great American School System, Linda Darling-Hammond’s The Flat World and Education, Richard Rothstein’s Class and Schools, Daniel Koretz’s Measuring Up, Tony Wagner’s The Global Achievement Gap, and Deborah Meier’s In Schools We Trust, among many others, are notable for their opposition to incentive-based reforms. There have always been books opposing reforms embraced by the Establishment, but they were usually outliers. In the aughts, however, a large number of prominent books stood in opposition.

Why is it so difficult to identify a book that embodies the incentive-based reforms of the decade and relatively easy to list books that argue against them? One reason is that books have lost their place as primary vehicles for shaping education policy. Just like in other realms, books are being displaced by other media.

A film like Waiting for “Superman” can have considerably more influence over education policy than any book. Articles and reports can be released on the Internet as soon as they are written. Even blogs are swaying education policy discussions to a greater extent than books. The power of blogs is especially clear when it comes to debating the merits of the research on various policy questions. There is little point in writing a book that reviews and adjudicates research findings when online articles and blog posts can do the same thing and be available within days or even hours.

The lack of policy influence that is attributable to recent education-reform books is not for lack of sales. Some have even become national best sellers. The problem is that policymakers and other elites are less likely to be among their readers. Instead, the buyers increasingly seem to be those actively participating in education reform debates; the people actually shaping policy appear to be paying relatively little attention.

For example, teachers and others hostile to incentive-based reforms consume works by Diane Ravitch, Linda Darling-Hammond, and Tony Wagner to affirm their worldview. These books are not setting the agenda for policymakers. They are feeding the resentment of practitioners to an education reform agenda that draws its inspiration from nonbook sources and is advancing despite the hostility stirred by such books. These best-selling volumes are, in the words of their intellectual nemesis, “standing athwart history, yelling stop.”

But books are no longer up to the task of significantly altering, let alone stopping, education policy trends. Policy agendas are being shaped by online debates, articles, conferences, and documentary films—not by books. In policy terms, the education reform book is dead, even as education reform thrives.

There is hope. To paraphrase Miracle Max, the education reform book is only mostly dead. Its policy influence can be revived if authors steer clear of topics that are better addressed by other media. Blogs can evaluate research as it comes out and are quicker and cheaper to write as well as to read. Emotionally charged anecdotes can be shared to far greater effect in a documentary film. Books shouldn’t try to do what other media can do better, faster, and with greater ease.

Moreover, if book authors seek policy influence, they have to write with policy elites as their target audience. It may sell a lot of books to write for teachers or education school students, but those people no longer dominate policymaking discussions. There is a new set of elites interested in education policy who do not come from the traditional teaching or education school worlds. These people tend to be young and technology savvy, getting more of their information from the Internet than from books. They can still be reached by books, but the volume would have to be written with them in mind rather than the traditional educator audience.

Of course, there is nothing wrong with books that are not written with policy influence as their primary objective. The book geared for an academic audience or designed to encourage a partisan base will continue to have its place. But if there is a lesson from the last decade of education reform books for enhancing policy influence, it is that the education reform book is dead—or at least mostly dead.

Jay P. Greene is professor of education reform at the University of Arkansas, senior fellow at the George W. Bush Institute, and contributing editor at Education Next.

Sam Dillon took a very different tack.  His reaction was to believe that the blog post was “suggesting on the internet that I had misinterpreted an interview, and then you repeated the same thing about the Los Angeles Times. That was just a sloppy and irresponsible error.”  I’m not sure how Dillon jumps to this thin-skinned defensiveness when I clearly said I did not know where the error was made:

“I don’t know whether something got lost in the translation between the researchers and Gates education chief, Vicki Phillips, or between her and Sam Dillon at the New York Times, but the article contains a false claim that needs to be corrected before it is used to push changes in education policy and practice.”

But more importantly, Dillon failed to check the accuracy of the disputed claim with independent experts.  Instead, he simply reconfirmed the claim with Gates officials: “For your information, I contacted the Gates Foundation after our correspondence and asked them if I had misquoted or in any way misinterpreted either Vicki Phillips, or their report on their research. They said, ‘absolutely not, you got it exactly right.’”

He went on to call my efforts to correct the claim “pathetic, sloppy, and lazy, and by the way an insult.”  I guess Dillon thinks that being a reporter for the New York Times means never having to say you’re sorry — or consult independent experts to resolve a disputed claim.

If Dillon wasn’t going to check with independent experts, I decided that I should — just to make sure that I was right in saying that the claims in the NYT and LAT coverage were unsupported by the findings in the Gates report.

Just to review, here is what Dillon wrote in the New York Times: “One notable early finding, Ms. Phillips said, is that teachers who incessantly drill their students to prepare for standardized tests tend to have lower value-added learning gains than those who simply work their way methodically through the key concepts of literacy and mathematics.”  And here is what Jason Felch wrote in the LA Times: ““But the study found that teachers whose students said they ‘taught to the test’ were, on average, lower performers on value-added measures than their peers, not higher.”  And the correlations in the Gates report between test student reports of test prep and value-added on standardized tests were all positive: “We spend a lot of time in this class practicing for the state test.” (?=0.195), “I have learned a lot this year about the state test.” (?=0.143), “Getting ready for the state test takes a lot of time in our class.” ( ?=0.103).  The report does not actually contain items that specifically mention “drill,”work their way methodically through the key concepts of literacy and mathematics,” or “taught to the test,” but I believe the reporters (and perhaps Gates officials) are referencing the test prep items with these phrases.

I sent links to the coverage and the Gates report to a half-dozen leading economists to ask if the claims mentioned above were supported by the findings.  The following reply from Jacob Vigdor, an economist at Duke, was fairly representative of what they said even if it was a bit more direct than most:

I looked carefully at the report and come to the same conclusion as you: these correlations are positive, not negative.  The NYT and LAT reports are both plainly inconsistent with what is written in the report.  A more accurate statement would be along the lines of “test preparation activities appear to be less important determinants of value added than [caring teachers, teacher control in the classroom, etc].”  But even this statement is subject to the caveat that pairwise correlations don’t definitively prove the importance of one factor over another.  Maybe the reporters are describing some other analysis that was not in the report (e.g., regression results that the investigators know about but do not appear in print), but even in that case they aren’t really getting the story right.  Even in that scenario, the best conclusion (given positive pairwise correlations and a hypothetically negative regression coefficient) would be that teachers who possess all these positive characteristics tend to emphasize test preparation as well.

Put another way, it’s alway good to have a caring teacher who is in control of the classroom, makes learning fun, and demands a lot of her students.  Among the teachers who share these characteristics, the best ones (in terms of value added) appear to also emphasize preparation for standardized tets.  I say “appear” because one would need a full-fledged multivariate regression analysis, and not pairwise correlations, to determine this definitively.

Another leading economist, who preferred not to be named, wrote: “I looked back over the report and I think you are absolutely right!”  I’m working on getting permission to quote others, but you get the idea.

In addition, to confirming that a positive correlation for test prep items means that it contributes to value-added, not detracts from it, several of these leading economists emphasized the inappropriateness of comparing correlations to draw conclusions about whether test prep contributes to value-added any more or less than other teacher practices observed by students.  They noted that any such comparison would require a multivariate analysis and not just a series of pairwise correlations.  And they also noted that any causal claim about the relative effectiveness of test prep would require some effort to address the endogeneity of which teachers engage in more test prep.

As David Figlio, an economist at Northwestern University, put it:

You’re certainly correct here.  A positive pairwise correlation means that these behaviors are associated with higher performance on standardized tests, not lower performance.  The only way that it could be an accurate statement that test prep is causing worse outcomes would be if there was a negative coefficient on test prep in a head-to-head competition in a regression model — though even then, one would have to worry about endogeneity: maybe teachers with worse-performing students focus more on test prep, or maybe lower-performing students perceive test prep to be more oppressive (of course, this could go the other way as well.)  But that was not the purpose or intent of the report.  The report does not present this as a head-to-head comparison, but rather to take a first look at the correlates between practice measures and classroom performance.

There was no reason for this issue to have developed into the controversy that it has. The coverage contains obvious errors that should have been corrected quickly and clearly, just as Jason Felch is doing.   Tom Kane, Vicki Phillips, and other folks at Gates should have immediately issued a clarification as soon as they were alerted to the error, which was on Monday.

And while I did not know where the error occurred when I wrote the blog post on Monday, the indications now are that there was a miscommunication between the technical people who wrote the report and non-technical folks at Gates, like Vicki Phillips and the pr staff.  In other words, Sam Dillon can relax since the mistake appears to have originated within Gates (although Dillon’s subsequent defensiveness, name-calling, and failure to check with independent experts hardly bring credit to the profession of journalism).

The sooner Gates issues a public correction, the sooner we can move beyond this dispute over what is actually a sidebar in their report and focus instead on the enormously interesting project on which they’ve embarked to improve measures of teacher effectiveness.  An apology from Sam Dillon would be also nice but I’m not holding my breath.

-Jay P. Greene

The initial round of results were reported last week with information from the student survey and standardized tests.  In particular, the report described the relationship between classroom practices, as observed by students, and value-added on the standardized tests.

The New York Times reported on these findings Friday and repeated the following strong claim:

But now some 20 states are overhauling their evaluation systems, and many policymakers involved in those efforts have been asking the Gates Foundation for suggestions on what measures of teacher effectiveness to use, said Vicki L. Phillips, a director of education at the foundation.

One notable early finding, Ms. Phillips said, is that teachers who incessantly drill their students to prepare for standardized tests tend to have lower value-added learning gains than those who simply work their way methodically through the key concepts of literacy and mathematics. (emphasis added)

I looked through the report for evidence that supported this claim and could not find it.  Instead, the report actually shows a positive correlation between student reports of “test prep” and value added on standardized tests, not a negative correlation as the statement above suggests.  (See for example Appendix 1 on p. 34.)

The statement “We spend a lot of time in this class practicing for [the state test]” has a correlation of  0.195 with the value added math results.  That is about the same relationship as “My teacher asks questions to be sure we are following along when s/he is teaching,” which is 0.198.  And both are positive.

It’s true that the correlation for “Getting ready for [the state test] takes a lot of time in our class” is weaker (0.103) than other items, but it is still positive.  That just means that test prep may contribute less to value added than other practices, but it does not support the claim that  ”teachers who incessantly drill their students to prepare for standardized tests tend to have lower value-added learning gains…”

In fact, on page 24, the report clearly says that the relationship between test prep and value-added on standardized tests is weaker than other observed practices, but does not claim that the relationship is negative:

The five questions with the strongest pair-wise correlation with teacher value-added were: “Students in this class treat the teacher with respect.” (?=0.317), “My classmates behave the way my teacher wants them to.”(?=0.286), “Our class stays busy and doesn’t waste time.” (?=0.284), “In this class, we learn a lot almost every day.”(?=0.273), “In this class, we learn to correct our mistakes.” (?=0.264) These questions were part of the “control” and “challenge” indices. We also asked students about the amount of test preparation they did in the class. Ironically, reported test preparation was among the weakest predictors of gains on the state tests: “We spend a lot of time in this class practicing for the state test.” (?=0.195), “I have learned a lot this year about the state test.” (?=0.143), “Getting ready for the state test takes a lot of time in our class.” ( ?=0.103)

I don’t know whether something got lost in the translation between the researchers and Gates education chief, Vicki Phillips, or between her and Sam Dillon at the New York Times, but the article contains a false claim that needs to be corrected before it is used to push changes in education policy and practice.

UPDATE – The LA Times coverage of the report contains a similar misinterpretation: “But the study found that teachers whose students said they “taught to the test” were, on average, lower performers on value-added measures than their peers, not higher.”

Try this thought experiment with another observed practice to illustrate my point about how the results are being mis-reported…  The correlation between student observations that “My teacher seems to know if something is bothering me” and value added was .153, which was less than the .195 correlation for “We spend a lot of time in this class practicing for [the state test].”  According to the interpretation in the NYT and LA Times, it would be correct to say “teachers who care about student problems tend to have lower value-added learning gains than those who spend a lot of time on test prep.”

Of course, that’s not true.  Teachers caring about what is bothering students is positively associated with value added just as test prep is.  It is just that teachers caring is a little less strongly related than test prep.  Caring does not have a negative effect just because the correlation is lower than other observed behaviors.

-Jay P. Greene

Disabled students have had to fight for decades to receive an adequate education from the public school system.  Federal legislation, now called IDEA, was adopted in the mid-1970s to ensure an appropriate eduction for disabled students.  Unfortunately, having a legal right to something and actually receiving it are two very different things and many disabled students continue to be denied appropriate services despite their legal entitlements.

That’s why several states have decided to empower families with disabled children with an additional mechanism by which they could ensure receiving an appropriate education — special education vouchers that would allow them to transfer to private schools if they believe that the public schools are not serving them adequately.  Oklahoma is the latest state to offer these vouchers but it almost certainly won’t be the last as several other states are considering the idea.  And there is good evidence that special education vouchers are significantly improving outcomes.

But, according to Education Week, four Oklahoma school districts have decided not to offer these vouchers that are required by state law.  The reasons given for willfully disobeying the state law are varied.  One district, Broken Arrow, has suggested that the voucher laws violate the state’s constitution because funds would go to religious schools.

Besides the fact that the U.S. Supreme Court and several states have ruled that these vouchers would not pose this type of constitutional threat, apparently public school district officials in Oklahoma think they know better.  And they’ve discovered some new constitutional process by which school officials interpret the constitution rather than the courts.

Actually, it’s not really a new method of deciding who should interpret the constitution, since it was a method well-established by segregationist school officials and governors who believed that they had the power to block black students from exercising their civil rights no matter what the law or courts said.  Now it is disabled students who are being blocked at the schoolhouse door.

The willingness of public school officials to publicly flaunt their disobedience of the law is not even very unique in current times.  Just two years ago school officials in Georgia decided to disobey the state’s duly enacted social promotion policy simply because they disagreed with the policy. Some of the Oklahoma public school officials similarly believe that they have standing as educators to decide what is best for kids and formulate the policy regardless of what the law and the people who pay them say.

Public school officials get away with this kind of willful violation of the law far too easily.  No one will go to jail.  No one will lose their job.  No one will be sanctioned in any way.  And they wonder why having a law to ensure appropriate services for disabled students isn’t sufficient.  Maybe it’s because public school officials apparently don’t have to follow the law.

Educational Economics: Where Do School Funds Go?

University of Washington professor and Gates Foundation advisor Marguerite Roza is the Indiana Jones of school finance. In her short but powerful new book, Educational Economics: Where Do School Funds Go?, Roza uncovers the hidden caves and tunnels that store the treasure of the public school system. Revealing where the money goes requires intrepid sleuthing, detailed analysis, and occasionally braving hostile natives.

The main finding of Roza’s explorations is that education dollars are allocated in ways that are sharply at odds with the stated priorities of public school systems. Education leaders say they want to devote greater funding to low-income students, but within most school districts per-pupil spending is higher at schools with more-advantaged students. Education leaders say they want to focus resources on the core subjects of math, reading, history, and science, but per-pupil spending tends to be much higher for electives, extracurricular activities, and sports. Education leaders say they want to emphasize remedial instruction to help lagging students catch up, but in most school districts per-pupil spending is significantly greater for Advanced Placement (AP) and gifted classes than for remedial ones.

The chief culprit in this misallocation of resources relative to stated priorities is the uniform salary schedule for teachers. In virtually every public school throughout the country, teachers are paid primarily according to their credentials, seniority, and “additional” work assignments and not at all according to subject taught, number of students served, or the difficulty or importance of their assignments. The net effect of this arrangement is that labor costs, the bulk of per-pupil spending, are distributed by formulas that are completely unaligned with stated priorities.

Schools with more low-income students tend to receive less per-pupil spending within districts because the higher-paid teachers with greater experience often transfer to schools with more-advantaged students who are less difficult to educate. Non-core electives, like art, music, gym, and shop, receive higher per-pupil spending because they tend to have fewer students per class than required core subjects, like reading, math, history, and science. Since all teachers are paid the same regardless of the subject they teach, smaller classes necessarily translate into higher per-pupil spending. Extracurricular activities and sports receive higher per-pupil funding because fewer students participate and teachers receive extra pay for assuming these “additional” assignments. Per-pupil spending on AP and gifted classes exceeds remedial classes because, again, fewer students tend to be in those advanced classes.

“How can those inside and outside the system allow such blatantly contradictory spending patterns to persist in their own schools?” Roza asks. Her first explanation is ignorance: “They generally do not know these patterns exist, as district budgeting and accounting practices make it incredibly difficult to identify detailed spending patterns.” But elsewhere Roza suggests that the problem is less benign than ignorance. She writes, “Powerful forces work to protect the interests of those who benefit from the present allocation of resources. Among those who benefit from the status quo are the more experienced teachers, influential parents with children in high-achieving schools, and board members who represent wealthier neighborhoods.” She also highlights the role that teachers unions play in determining the allocation of resources by championing the uniform salary schedule, transfer rights for more experienced teachers, and work rules.

Roza’s ambiguity about the causes of the mismatch between stated priorities and actual spending undermines her ability to propose solutions. If the problem is caused primarily by ignorance, then the solution lies in greater transparency through more rigorous and open accounting policies. But if the problem is caused primarily by the influence of powerful interest groups, then a political restructuring of incentives is required. If poor kids get the short end of the education stick because teachers unions and wealthy parents pursue their own benefit with indifference to the consequences for those less fortunate, then those interest groups have to be stripped of their control over allocating resources. This could be achieved by empowering families with direct control over education resources via vouchers or a weighted student-based formula for allocating government funds.

For most of the book, Roza leans toward the ignorance explanation: “The most important answer is that they don’t know about real spending patterns … Bad information leads to mistaken assumptions and ultimately misguided strategic resource decisions.” Unfortunately, this explanation for misallocated school spending is unsatisfying and fails to yield compelling solutions, even according to Roza herself. She lists a variety of school-finance reforms and argues that they are all “guaranteed to fail” because they do not address the “entire package of incoherent, inefficient, and inequitable spending.”

The solution, she acknowledges in the final two chapters, requires a more comprehensive restructuring of the education system than just transparency measures. On the final page of the text, she reveals how that restructuring might take shape when she emphasizes “the need to separate the functions of allocating resources, setting standards, and defining accountability from the function of making decisions about resource use. If states could recognize that they play some role in the first three, they might be convinced that they should not also take on the fourth.” This sounds like vouchers or weighted student-based funding, where the government funds education and establishes accountability for results while decentralizing to the family or individual school the power to decide how money is spent.

The book would be stronger if the political restructuring of the education system were addressed earlier and more fully. As it stands, readers are likely to get the mistaken impression that ignorance is the primary cause of the failure of school funding systems and improved awareness the solution. Ignorance is a problem, but it is the willful ignorance of malicious indifference. No solutions are possible without addressing that.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute.

WWC makes mistakes, is subject to political manipulation, and applies arbitrary standards.  In short, what WWC says is not The Truth.  WWC is not necessarily less reliable than any other source that claims to adjudicate The Truth for you.  Everyone may make mistakes, distort results, and apply arbitrary standards.  The problem is that WWC has the official endorsement of the U.S. Department of Education, so many people fail to take their findings with the same grains of salt that they would to the findings of any other self-appointed truth committee.  And with the possibility that government money may be conditioned on WWC endorsement, WWC’s shortcomings are potentially more dangerous.

I could provide numerous examples of WWC’s mistakes, political manipulation, and arbitrariness, but for the brevity of a blog post let me illustrate my point with just a few.

First, WWC was sloppy and lazy in its recent finding that the Milwaukee voucher evaluation, led by my colleagues Pat Wolf and John Witte, failed to meet “WWC evidence standards” because “the authors do not provide evidence that the subsamples of voucher recipients and public school comparison students analyzed in this study were initially equivalent in math and reading achievement.” WWC justifies their conclusion with a helpful footnote that explains: “At the time of publication, the WWC had contacted the corresponding author for additional information regarding the equivalence of the analysis samples at baseline and no response had been received.”

But if WWC had actually bothered to read the Milwaukee reports they would have found the evidence of equivalence they were looking for.  The Milwaukee voucher evaluation that Pat and John are leading has a matched-sample research design.  In fact, the research team produced an entire report whose purpose was to demonstrate that the matching had worked and produced comparable samples. In addition, in the 3rd Year report the researchers devoted an entire section (see appendix B) to documenting the continuing equivalence of the matched samples despite some attrition of students over time.

Rather than reading the reports and examining the evidence on the comparability of the matched samples, WWC decided that the best way to determine whether the research met their standards for sample equivalence was to email John Witte and ask him.  I guess it’s all that hard work that justifies the multi-million dollar contract Mathematica receives from the U.S. Department of Education to run WWC.

As it turns out, Witte was traveling when WWC sent him the email.  When he returned he deleted their request along with a bunch of other emails without examining it closely.  But WWC took Witte’s non-response as confirmation that there was no evidence demonstrating the equivalence of the matched samples.  WWC couldn’t be bothered to contact any of the several co-authors.  They just went for their negative conclusion without further reading, thought, or effort.

I can’t prove it (and I’m sure my thought-process would not meet WWC standards), but I’ll bet that if the subject of the study was not vouchers, WWC would have been sure to read the reports closely and make extra efforts to contact co-authors before dismissing the research as failing to meet their standards.  But voucher researchers have grown accustomed to double-standards when others assess their research.  It’s just amazingly ironic to see the federally-sponsored entity charged with maintaining consistent and high standards fall so easily into their own double-standard.

Another example — I served on a WWC panel regarding school turnarounds a few years ago.  We were charged with assessing the research on how to successfully turnaround a failing school.  We quickly discovered that there was no research that met WWC’s standards on that question.  I suggested that we simple report that there is no rigorous evidence on this topic.  The staff rejected that suggestion, emphasizing that the Department of Education needed to have some evidence on effective turnaround strategies.

I have no idea why the political needs of the Department should have affected the truth committee in assessing the research, but it did.  We were told to look at non-rigorous research, including case-studies, anecdotes, and our own experience to do our best in identifying promising strategies.  It was strange — there were very tight criteria for what met WWC standards, but there were effectively no standards when it came to less rigorous research.  We just had to use our professional judgment.

We ended up endorsing some turnaround strategies (I can’t even remember what they were) but we did so based on virtually no evidence.  And this was all fine as long as we said that the conclusions were not based on research that met WWC standards.  I still don’t know what would have been wrong with simply saying that research doesn’t have much to tell us about effective turnaround strategies, but I guess that’s not the way truth committees work.  Truth committees have to provide the truth even when it is false.

The heart of the problem is that science has never depended on government-run truth committees to make progress.  It is simply not possible for the government to adjudicate the truth on disputed topics because the temptation to manipulate the answer or simply to make sloppy and lazy mistakes is all too great.  This is not a problem that is particular to the Obama Administration or to Mathematica.  My second example was from the Bush Administration when WWC was run by AIR.

The hard reality is that you can never fully rely on any authority to adjudicate the truth for you.  Yes, conflicting claims can be confusing.  Yes, it would be wonderfully convenient if someone just sorted it all out for us.  But once we give someone else the power to decide the truth on our behalf, we are prey to whatever distortions or mistakes they may make.  And since self-interest introduces distortions and the tendency to make mistakes, the government is a particularly untrustworthy entity to rely upon when it comes to government policy.

Science has always made progress by people sorting through the mess of competing, often technical, claims.  When official truth committees have intervened, it has almost always hindered scientific progress.  Remember that  it was the official truth committee that determined that Galileo was wrong.  Truth committees have taken positions on evolution, global warming, and a host of other controversial topics.  It simply doesn’t help.

We have no alternative to sorting through the evidence and trying to figure these things out ourselves.  We may rely upon the expertise of others in helping us sort out competing claims, but we should always do so with caution, since those experts may be mistaken or even deceptive.  But when the government starts weighing in as an expert, it speaks with far too much authority and can be much more coercive.  A What Works Clearinghouse simply doesn’t work.

That may be, but as a graduate student observed to me today, choice (in the form of vouchers) may have lost Sol Stern, but choice (in the form of charters) just gained Oprah, the Today Show, and the Democratic Party platform.    Overall, he thought that was a pretty good trade, especially since he had to look up who Sol Stern was.

Let’s review.  It is now commonly accepted among mainstream elites — from Oprah to Matt Lauer to Arne Duncan — that simply pouring more money into the public school system will not produce the results we want.  It is now commonly accepted that the teacher unions have been a significant barrier to school improvement by protecting ineffective teachers and opposing meaningful reforms.  It is now commonly accepted that parents should have a say in where their children go to school and this choice will push traditional public schools to improve.  It is now commonly accepted that we have to address the incentives in the school system to recruit, retain, and motivate the best educators.

These reform ideas were barely a twinkle in Ronald Reagan’s eye three decades ago and are now broadly accepted across both parties and across the ideological spectrum.  This is a huge accomplishment and rather than being all bummed out that everyone else now likes the band that I thought was cool before anyone ever heard of it, we should be amazed at how much good music there is out there.

We won!  At least we’ve won the war of ideas.  Our ideas for school reform are now the ones that elites and politicians are considering and they have soundly rejected the old ideas of more money, more money, and more money.

Now that I’ve said that, I have to acknowledge that winning the war of ideas is nowhere close to winning the policy war.  As I’ve written before, the teacher unions are becoming like the tobacco industry.  No one accepts their primary claims anymore, but that doesn’t mean they don’t continue to be powerful and that people don’t continue to smoke.  The battle is turning into a struggle over the correct design and implementation of the reform ideas that are now commonly accepted.  And the unions have shown that they are extremely good at blocking, diluting, or co-opting the correct design and implementation of reforms.

Rick Hess correctly demonstrated how important design and implementation are almost two decades ago in his books, Spinning Wheels and Revolution at the Margins.   And it is always useful for him and others to remind reformers of the dangers that lurk in those union-infested waters.  But for a moment can’t we just bask in the glow of our intellectual victory — even if our allies are a new crop of naive reformers?

The WSJ piece by Barbara Martinez is entitled “Private-School Tuitions Burden DOE.”  The DOE in this case is the Department of Education in New York City, which the article points out “last year spent $116 million on tuition and legal expenses related to special-education students whose parents sued the DOE on the grounds that the public-school options were inadequate. That’s more than double the number of just three years ago, and the costs are expected to continue to rise in coming years.”

As I’ve pointed out before, the trick to writing an article blaming special education is to mention a high cost for educating certain special education students (or even a high-sounding aggregate figure) without putting in perspective how much money that is relative to the entire school budget.  True to form, this article states: “The tuition payouts range between $20,000 and more than $100,000 per child and have been used for schools as far away as Utah.”  Wow, that sounds like a lot of money.  And going all the way to Utah sounds extravagant.

But let’s put this issue in perspective, which even a minimal amount of effort by the reporter could have done.  If private school tuition really is a “burden” as the title asserts, the cost of private-placement should be a significant portion of the New York City school budget.  It isn’t.  If you look at the NYC education budget you see that schools spent a total of $17.9 billion in 2009.  The total cost of private placement is only $116 million, which is about .6% of total spending.  This is close to rounding error for NYC.

To put it further in perspective, the NYC education system spent $151 million last year on pollution remediation to address lead paint, asbestos, and contaminated soil at its properties.  Imagine if there had been a news article entitled “Pollution Clean-Up Burdens DOE.”  People would have dismissed that as ridiculous, noting that the total amount spent on pollution is a very small part of the total budget and could hardly be considered a burden.  What’s more, people would have acknowledged that cleaning up pollution is important and the schools need to do it.

But this article on private tuition for special education “burdens” is even worse because the burden on the district isn’t the total cost, but the cost for private placement in excess of what the district would have spent if they had served these disabled students in traditional public schools.  We know from the article that there were 4,060 students who sought private placement for an aggregate cost of $116 million.  That works out to $28,571 per student.

We also know from the NYC DOE budget that schools spent a total of $17.9 billion for about 1.1 million students, which works out to $16,263 per student.  But wait, NYC spends more on its special education students than on the average student.  If we look at the NYC DOE budget (which any education reporter worth his or her salt could easily do), they identify additional costs associated with special education.  From that we can calculate that NYC spends an average of $24,773 on its special education students.

The “burden” on NYC DOE from paying private school tuition is the difference between the average tuition and legal costs associated with private placement ($28,571) and the average cost for a disabled student in the traditional public schools ($24,773), which works out to $3,798 per student.  An extra $3,798 per privately placed student over 4,060 students constitutes an additional expense of $15.4 million for NYC DOE.  That amounts to less than .09% of the NYC DOE education budget.

Calling this a “burden” on the district is irresponsible and just distracts people from the true and large areas of waste burdening the school system.

I raised doubts about the rigor and soundness of the proposed national standards, citing the fact that many credible experts have denounced them as lousy.  His response is simply to repeat that Fordham has given the standards good grades and thinks the latest revisions have been positive.  This is not a substantive response; it is simply a reiteration of their initial position.

Why should we find Fordham’s grading of the proposed national standards any more credible than that of the experts who have denounced the standards?  The fact that Fordham issued a report with letter grades is just a marketing exercise for Fordham’s opinion.  There is nothing scientific or rigorous about Fordham hand-picking their friends experts to repeat the opinion Fordham already holds — especially when we know from past experience that Fordham might exclude experts or change the grades if it does not come out the way they want.

Yes, the national standards may be better than those in some states, but everyone seems to agree that they are also worse than the standards in some states.  Why should we hurt the excellent standards in MA or CA to improve the standards in AR or MS?  Wouldn’t it be smarter to focus our energies on pressuring states with bad standards to improve them?

It is true that the Edublob dominates the standards and assessment process in many states, but the existence of choice and competition among the states places constraints on their ability to impose nonsense through that machinery.  If the standards and assessment process is centralized at the national level, the Edublob will be able to impose nonsense on everyone with no “exit power” to constraint them.

Rather than rely on market mechanisms to constrain nonsense, Mike places his trust in devising national political systems that he thinks can develop and maintain good national standards and assessments.  In particular, Mike thinks that it is “more likely that the good guys will stay in charge at the national level, where all of this stuff will operate under the bright lights of the national media, than in the states, where decisions get made behind closed doors.”  The national government also regulates off-shore drilling and the financial system.  How well did those bright lights work at ensuring a sensible regulatory framework?

The hard reality is that regulation tends to be captured by the regulated industry (unless there are competing, well-organized interests, which in education there are not).  Education regulations, like national standards and assessments, are at least as likely to be captured by the Edublob as the oil industry is to capture off-shore drilling regulations or the banking industry is to capture financial regulations.

The answer is not to have bigger, more centralized regulations.  The answer is to maintain the proper incentives by empowering market forces, which also serve to keep the regulatory framework honest.  I’m not advocating against all regulations.  I’m saying that there need to be market checks and balances to keep regulatory frameworks reasonable.  If we centralize the standards and assessment process, we have eliminated some of the few market checks and balances we have in education.  The fact that Linda Darling-Hammond is part of the leading bid to develop national assessments to go along with these national standards should make clear the dangers of nationalizing this process.

And make no mistake.  The Obama administration has signaled that it intends to link federal money to adoption of a Linda Darling-Hammond test or whatever other nonsense this centralized process may produce.  Just because Mike thinks  ”the Administration erred and gave national standards opponents an opportunity to raise concerns about federal overreach” doesn’t mean that they aren’t going to do precisely what they have declared they will do even if he thinks it is mistaken.

But the most telling comment of Mike’s faulty thinking on national standards was when he asked: ”Does Jay oppose voucher programs because they might get hijacked by shady for-profit providers who just want to make money off the backs of poor kids?”  The fundamental difference between the potential for “hijacking” of national standards and assessments and the “hijacking” of a voucher school is the mechanism by which one can control (or hijack) them.

Voucher schools are controlled primarily by the market choices of parents.  You can’t “hijack” a voucher school because parents can choose to go to another school if they dislike what the school tries to do.  But you can “hijack” national standards and assessments because they are controlled politically and not by market forces.  People who dislike what the national standards and assessments do are still compelled to send their children to schools operating under that national system.  You don’t need parental or even popular buy-in to hijack national standards and assessments.  You just have to be better politically organized and motivated to dominate the process by which those standards and assessments are developed and maintained.

This all leads to my question that Mike never answered:

“If there really were one true way to educate all children, why stop at national standards? Why not have global standards with a global curriculum?

We would oppose global standards for the same reasons we should oppose national standards. Making education uniform at too high of a level of aggregation ignores the diversity of needs of our children as well as the diversity of opinion about how best to serve those needs. And giving people at the national or global level the power to determine what everyone should learn is dangerous because they will someday use that power to promote unproductive or even harmful ideas.”

The reason Mike and other supporters of national standards and assessments don’t advocate for global standards and assessments (even though the logic for doing so is essentially the same as national standards and assessments) is that they imagine that they’ll be the ones controlling the national process.  Someone else would dominate the global one and that would have to be bad.

As much as I like Mike, I don’t want him or (more likely) the Edublob dominating national standards and assessment, which would have profound effects on how every classroom in the country operates.  Even though it is messy and imperfect, we need to decentralize power in education rather than centralize it.  We need to do so for the same reason the Constitution decentralizes power — to prevent abuses and tyranny that inevitably arise when power is unchecked and concentrated.  We need to decentralize power in education to allow market mechanisms to operate.  We need to decentralize power to recognize the legitimate diversity of needs and approaches that exist in our educational system.

Benevolent dictatorships are always attractive on paper but the benevolent part never works out in practice.