There’s still an increment, it turns out, but it’s been shrinking: from two years more schooling (by age thirty) for those born in 1955 down to just eight months more for those born in 1980. The implication, quoth the Journal reporters: “Without better educated Americans, economists say, the U.S. won’t be able to maintain high-wage jobs and rising living standards in a competitive global economy.”

This isn’t exactly news. Nor is the Goldin-Katz analysis the first time we have observed that the U.S. is no longer leading the planet when it comes to the quantity of education that its population receives. But is more education—more hours and days, more years and degrees—the cure for what ails us? Or are we already pigging out on the educational equivalent of fast food—fattening but not nutritious—and will supersizing our portions just make matters worse?

If we accept the Goldin-Katz view of what’s wrong with U.S. education, we will inevitably demand more preschool, more full-day Kindergarten, more high school graduations, more college attendance, more college and postgraduate degrees, etc. Supersizing is the standard American response. Indeed, it’s already on the election-year menu with both parties demanding that student-loan interest rates be made to stay low so that more people can afford more tertiary education. Not much talk about quality, though.

Singapore is one of those places that’s been going a mile a minute in boosting both the quality and the quantity of formal education that its population receives. For example, the proportion of Singaporeans aged twenty-five to thirty-nine that completed secondary school (meaning tenth grade) jumped from 25 percent in 1980 to 96 percent in 2010. At the same time, Singapore students beat almost everyone else in the world on international assessments of math and science knowledge.

To an American, however, it’s surprising how little rush there is to supersize Singaporean education. Kindergarten is optional. (The primary schools start at age six or seven.) And only about one in four young Singaporeans currently qualifies for the “junior colleges” (basically grades eleven and twelve) that are the usual path into the country’s four universities. Government policy is headed toward placing 30 percent of the age cohort in public universities; for now, as many as 40 percent of secondary graduates head into career-oriented “polytechnics” that resemble the best of American community colleges and some 20 percent attend the Institute of Technical Education, which emphasizes “hands-on” training.

There is, to be sure, public pressure to increase the number who can enter Singapore’s universities—and some private and non-Singaporean institutions have opened to accommodate some of that demand. (Other students travel overseas for their tertiary education.) But basically nobody is saying that every young person should go to university. And remember: this in an education-obsessed country with no other resources to speak of save its highly skilled populace.

Nor are they going to take the easy path (as England and Hong Kong have done in recent decades and as the U.S. started to do long ago) and put fancier labels onto existing institutions. They are not, for example, going to pretend that their polytechnics are really universities, as we have done with hundreds of ex-teachers colleges and quondam “normal schools.” They regard that kind of maneuver as both an affront to quality control at the university level and damaging to the real-world job-preparation work that the polytechnics specialize in.

The United States, of course, tends to reject both the benefits and the detriments of Singapore-style central planning in the education space, at least when it comes to planning from Washington. But the new Goldin-Katz data, combined with OECD trend data, make clear that our system (or non-system) isn’t doing a very good job of propelling more people onward to get more education than their parents got. And we know from plenty of other data (TIMSS, PISA, etc.) that the quality of much of what they’re getting isn’t so great, either, especially when viewed in international perspective.

Any number of reform initiatives are seeking to tackle one or the other problem. Some are focused on raising academic standards, others on keeping more people in the education system longer and seeing that they emerge with credentials. Some insist that the two goals are complimentary—and the mantra that “everyone should emerge from high school both college and career-ready” tends to blur the distinction and terminate the discussion.

But what will we do when we face hard trade-offs, such as the likely discovery that higher graduation standards will lead to a higher failure (and dropout) rate? Our track record in this regard leaves much to be desired. Even much-envied Massachusetts, which has done a commendable job of getting almost all who stay in school over the medium-high bar set by MCAS, has worrisome dropout rates, particularly among minority youngsters, and has been loath to raise its high school exit-bar to the level of true college readiness.

Are our presidential candidates crazy to yammer about cheap loans to make college more affordable for all? I understand that nobody (except maybe Rick Santorum) is going to campaign for the White House by urging fewer young Americans to go to college. But if more do in fact go and stay, will they really be getting a good education there? Or just a bigger bag of fries? What if, instead, more of them simply emerged career ready from our secondary schools, which already last two years longer than the norm in Singapore? Wouldn’t a whole lot of time and money be saved and a lot of heartache and dashed aspirations avoided?

I don’t expect these dilemmas to be resolved in Washington—though it would be fascinating to hear them discussed by Messrs. Obama and Romney in an upcoming debate. But it’s something our states had better come to grips with—including how they finance their P-20 education systems. It’s clear that rising tertiary education costs paid by consumers—and heavy debt burdens on many who enter and persist in college—are part of the problem. But only part. Maybe more attention to quality would do greater good.

-Chester E. Finn, Jr.

This post originally appeared in the Fordham Institute’s Education Gadfly Weekly.

Surpassing Shanghai: An Agenda for American Education Built on the World’s Leading Systems

As reviewed by Jay P. Greene

“Best practices” is the worst practice. The idea that we should examine successful organizations and then imitate what they do if we also want to be successful is something that first took hold in the business world but has now unfortunately spread to the field of education. If imitation were the path to excellence, art museums would be filled with paint-by-number works.

The fundamental flaw of a “best practices” approach, as any student in a half-decent research-design course would know, is that it suffers from what is called “selection on the dependent variable.” If you only look at successful organizations, then you have no variation in the dependent variable: they all have good outcomes. When you look at the things that successful organizations are doing, you have no idea whether each one of those things caused the good outcomes, had no effect on success, or was actually an impediment that held organizations back from being even more successful. An appropriate research design would have variation in the dependent variable; some have good outcomes and some have bad ones. To identify factors that contribute to good outcomes, you would, at a minimum, want to see those factors more likely to be present where there was success and less so where there was not.

“Best practices” lacks scientific credibility, but it has been a proven path to fame and fortune for pop-management gurus like Tom Peters, with In Search of Excellence, and Jim Collins, with Good to Great. The fact that many of the “best” companies they featured subsequently went belly-up—like Atari and Wang Computers, lauded by Peters, and Circuit City and Fannie Mae, by Collins—has done nothing to impede their high-fee lecture tours. Sometimes people just want to hear a confident person with shiny teeth tell them appealing stories about the secrets to success.

With Surpassing Shanghai, Marc Tucker hopes to join the ranks of the “best practices” gurus. He, along with a few of his colleagues at the National Center on Education and the Economy, has examined the education systems in some other countries with successful outcomes so that the U.S. can become similarly successful. Tucker coauthors the chapter on Japan, as well as an introductory and two concluding chapters. Tucker’s collaborators write chapters featuring Shanghai, Finland, Singapore, and Canada. Their approach to greatness in American education, as Linda Darling-Hammond phrases it in the foreword, is to ensure that “our strategies must emulate the best of what has been accomplished in public education both from here and abroad.”

But how do we know what those best practices are? The chapters on high-achieving countries describe some of what those countries are doing, but the characteristics they feature may have nothing to do with success or may even be a hindrance to greater success. Since the authors must pick and choose what characteristics they highlight, it is also quite possible that countries have successful education systems because of factors not mentioned at all. Since there is no scientific method to identifying the critical features of success in the best-practices approach, we simply have to trust the authority of the authors that they have correctly identified the relevant factors and have properly perceived the causal relationships.

But Surpassing Shanghai is even worse than the typical best-practices work, because Tucker’s concluding chapters, in which he summarizes the common best practices and draws policy recommendations, have almost no connection to the preceding chapters on each country. That is, the case studies of Shanghai, Finland, Japan, Singapore, and Canada attempt to identify the secrets to success in each country, a dubious-enough enterprise, and then Tucker promptly ignores all of the other chapters when making his general recommendations.

Tucker does claim to be drawing on the insights of his coauthors, but he never actually references the other chapters in detail. He never names his coauthors or specifically draws on them for his conclusions. In fact, much of what Tucker claims as common lessons of what his coauthors have observed from successful countries is contradicted in chapters that appear earlier in the book. And some of the common lessons they do identify, Tucker chooses to ignore.

For example, every country case study in Surpassing Shanghai, with the exception of the one on Japan coauthored by Marc Tucker, emphasizes the importance of decentralization in producing success. In Shanghai the local school system “received permission to create its own higher education entrance examination. This heralded a trend of exam decentralization, which was key to localized curricula.” The chapter on Finland describes the importance of the decision “to devolve increasing levels of authority and responsibility for education from the Ministry of Education to municipalities and schools…. [T]here were no central initiatives that the government was trying to push through the system.” Singapore is similarly described: “Moving away from the centralized top-down system of control, schools were organized into geographic clusters and given more autonomy…. It was felt that no single accountability model could fit all schools. Each school therefore set its own goals and annually assesses its progress toward meeting them…” And the chapter on Canada teaches us that “the most striking feature of the Canadian system is its decentralization.”

Tucker makes no mention of this common decentralization theme in his conclusions and recommendations. Instead, he claims the opposite as the common lesson of successful countries: “students must all meet a common basic education standard aligned to a national or provincial curriculum… Further, in these countries, the materials prepared by textbook publishers and the publishers of supplementary materials are aligned with the national curriculum framework.” And “every high-performing country…has a unit of government that is clearly in charge of elementary and secondary education…In such countries, the ministry has an obligation to concern itself with the design of the system as a whole…”

Conversely, Tucker emphasizes that “the dominant elements of the American education reform agenda” are noticeably absent from high-performing countries, including “the use of market mechanisms, such as charter schools and vouchers….” But if Tucker had read the chapter on Shanghai, he would have found a description of a system by which “students choose schools in other neighborhoods by paying a sponsorship fee. It is the Chinese version of school choice, a hot issue in the United States.” And although the chapter on Canada fails to make any mention of it, Canada has an extensive system of school choice, offering options that vary by language and religious denomination. According to recently published research by David Card, Martin Dooley, and Abigail Payne, competition among these options is a significant contributor to academic achievement in Canada.

There is a reason that promoters of best-practices approaches are called “gurus.” Their expertise must be derived from a mystical sphere, because it cannot be based on a scientific appraisal of the evidence. Marc Tucker makes no apology for his nonscientific approach. In fact, he denounces “the clinical research model used in medical research” when assessing education policies. The problem, he explains, is that no country would consent to “randomly assigning entire national populations to the education systems of another country or to certain features of the education system of another country.” On the contrary, countries, states, and localities can and do randomly assign “certain features of the education system,” and we have learned quite a lot from that scientific process. In the international arena, Tucker may want to familiarize himself with the excellent work being done by Michael Kremer and Karthik Muralidharan utilizing random assignment around the globe.

In addition, social scientists have developed practices to observe and control for differences in the absence of random assignment that have allowed extensive and productive analyses of the effectiveness of educational practices in different countries. In particular, the recent work of Ludger Woessmann, Martin West, and Eric Hanushek has utilized the PISA and TIMSS international test results that Tucker finds so valuable, but they have done so with the scientific methods that Tucker rejects. Even well-constructed case study research, like that done by Charles Glenn, can draw useful lessons across countries. The problem with the best-practices approach is not entirely that it depends on case studies, but that by avoiding variation in the dependent variable it prevents any scientific identification of causation.

Tucker’s hostility to scientific approaches is more understandable, given that his graduate training was in theater rather than a social science. Perhaps that is also why Tucker’s book reminds me so much of The Music Man. Tucker is like “Professor” Harold Hill come to town to sell us a bill of goods. His expertise is self-appointed, and his method, the equivalent of “the think system,” is obvious quackery. And the Gates Foundation, which has for some reason backed Tucker and his organization with millions of dollars, must be playing the residents of River City, because they have bought this pitch and are pouring their savings into a band that can never play music except in a fantasy finale.

Best practices really are the worst.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute.

American education has problems, almost everyone is willing to concede, but many think those problems are mostly concentrated in our large urban school districts. In the elite suburbs, where wealthy and politically influential people tend to live, the schools are assumed to be world-class.

Unfortunately, what everyone knows is wrong. Even the most elite suburban school districts often produce results that are mediocre when compared with those of our international peers. Our best school districts may look excellent alongside large urban districts, the comparison state accountability systems encourage, but that measure provides false comfort. America’s elite suburban students are increasingly competing with students outside the United States for economic opportunities, and a meaningful assessment of student achievement requires a global, not a local, comparison.

We developed the Global Report Card (GRC) to facilitate such a comparison. The GRC enables users to compare academic achievement in math and reading between 2004 and 2007 for virtually every public school district in the United States with the average achievement in a set of 25 other countries with developed economies that might be considered our economic peers and sometime competitors. The main results are reported as percentiles of a distribution, which indicates how the average student in a district performs relative to students throughout the advanced industrialized world. A percentile of 60 means that the average student in a district is achieving better than 59.9 percent of the students in our global comparison group. (Readers can find all of the results of the Global Report Card at http://globalreportcard.org. The web site contains a full description of the method by which we calculated the results. For a summary, see the methodology sidebar.)

For the purposes of this article, we focus on the 2007 math results, although the GRC contains information for both math and reading between 2004 and 2007. We focus on 2007 because it is the most recent data set, and we focus on math because it is the subject that provides the best comparison across countries and is most closely correlated with economic growth. Readers should feel free to consult the GRC web site to find reading results as well as results for other years.

The Example of Beverly Hills

It is critically important to compare exclusive suburban districts against the performance of students in other developed countries, as these districts are generally thought to be high-performing. The most wealthy and politically powerful families have often sought refuge from the ills of our education system by moving to suburban school districts. Problems exist in large urban districts and in low-income rural areas, elites often concede, but they have convinced themselves that at least their own children are receiving an excellent education in their affluent suburban districts.

Unfortunately, student achievement in many affluent suburban districts is worse than parents may think, especially when compared with student achievement in other developed countries. Take for example Beverly Hills, California. The city has a median family income of $102,611 as of 2000, which places it among the top 100 wealthiest places in the United States with at least 1,000 households. The Beverly Hills population is 85.1 percent white, 7.1 percent Asian, and only 1.8 percent black and 4.6 percent Hispanic. The city is virtually synonymous with luxury. A long-running television show featured the wealth and advantages of Beverly Hills high-school students (as well as their overly dramatic personal lives). If Beverly Hills is not the refuge from the ills of the education system that elite families are seeking, it’s not clear what would be.

But when we look at the Global Report Card results for the Beverly Hills Unified School District, we don’t see top-notch performance. The math achievement of the average student in Beverly Hills is at the 53rd percentile relative to our international comparison group. That is, one of our most elite districts produces students with math achievement that is no better than that of the typical student in the average developed country. If Beverly Hills were relocated to Canada, it would be at the 46th percentile in math achievement, a below-average district. If the city were in Singapore, the average student in Beverly Hills would only be at the 34th percentile in math performance.

Of course, people don’t think of Beverly Hills as a school district with mediocre student achievement. This is partly because people assume that affluent suburbs must be high achieving and partly because state accountability results inflate achievement by comparing affluent suburban school districts with large urban ones. According to California’s state accountability results, the average student in Beverly Hills is at the 76th percentile in math achievement relative to other students in the state. But outperforming students in Los Angeles, which is only at the 20th percentile in math relative to a global comparison group, should provide little comfort to Beverly Hills parents.

Los Angeles Unified is not the main source of competitors for Beverly Hills students, so the state accountability system encourages the wrong comparison. If Beverly Hills graduates are to have the kinds of jobs and lifestyles that their parents hope for them, they will have to compete with students from Canada, Singapore, and everywhere else. Beverly Hills students have to be toward the top of achievement globally if they expect to get top jobs and earn top incomes.

Results from Affluent Suburbs Nationwide

We can repeat the story of Beverly Hills all across the country. Affluent suburban districts may be outperforming their large urban neighbors, but they fail to achieve near the top of international comparisons (see Figure 1). White Plains, New York, in suburban Westchester County, is only at the 39th percentile in math relative to our global comparison group. Grosse Point, Michigan, outside of Detroit, is at the 56th percentile. Evanston, Illinois, the home of Northwestern University outside of Chicago, is at the 48th percentile in math. The average student in Montgomery County, Maryland, where many of the national government leaders send their children to school, is at the 50th percentile in math relative to students in other developed countries. The average student in Fairfax, Virginia, another suburban refuge for government leaders, is at the 49th percentile. Shaker Heights, Ohio, outside of Cleveland, is at the 50th percentile in math. The average student in Lower Merion, Pennsylvania, near Philadelphia, is at the 66th percentile. Ladue, Missouri, a wealthy suburb of St. Louis, is at the 62nd percentile. And the average student in Plano, Texas, near Dallas, is at the 64th percentile in math relative to our global comparison group.

All of these communities are among the wealthiest in the United States. All are overwhelmingly white in their population. All of them are thought of as refuges from the dysfunction of our public school system. But the sad reality is that in none of them is the average student in the upper third of math achievement relative to students in other developed countries. Most of them are barely keeping pace with the average student in other developed countries, despite the fact that the comparison is to all students in the other countries, some of which have a per-capita gross domestic product that is almost half that of the United States. In short, many of what we imagine as our best school districts are mediocre compared with the education systems serving students in other developed countries.

Pockets of Excellence

While many affluent suburban districts have lower achievement than we might expect, some districts are producing very high achievement even when compared with that of students in other developed countries. For example, the average student in the Pelham school district in Massachusetts is at the 95th percentile in math. That means that if we were to relocate Pelham to another developed country in our comparison group, the average student in Pelham would outperform 95 percent of the students in math. That’s very impressive.

Of course, Pelham is a small district that is home to Amherst College, among other institutions of higher learning, and serves a rather select group of students. But not all college-town school districts are equally high achieving. As we have already seen, Evanston, Illinois, is at the 48th percentile in math in a global comparison. Palo Alto, California, the home of Stanford University, is at the 64th percentile. And the average student in Ann Arbor, Michigan, home to the University of Michigan, is at the 58th percentile in math relative to students in other developed countries. So, the 95th percentile math achievement in Pelham is outstanding, even for college towns.

Spring Lake, New Jersey, has a similarly impressive record of having the average student at the 91st percentile in math. It is a very small and affluent community on the New Jersey shore that has somehow escaped the influence of Snooki and The Situation. Waconda, Kansas, a small rural community, also is at the 91st percentile. Highland Park, Texas, an affluent community near Dallas, is at the 88th percentile.

Interestingly, of the top 20 U.S. public-school districts in math achievement, 7 are charter schools (some states treat charter schools as separate public-school districts). And most of the 13 traditional districts remaining are in rural communities rather than in a large suburban “refuge” from urban education ills.

Pools of Failure

In total, only 820 of the 13,636 public-school districts for which we have 2007 math results had average student achievement that would be among the top third of student performance in other developed countries. That is, 94 percent of all U.S. school districts have average math achievement below the 67th percentile. There aren’t that many truly excellent districts out there.

Of the 13,636 districts, 9,339, or 68 percent, have average student math achievement that is below the 50th percentile compared with that of the average student in other developed countries. Most of our large school districts are well below the 50th percentile. This is especially alarming, because these lower-performing large districts comprise a much greater share of the total student population than do the relatively small higher-performing districts.

The average student in the Washington, D.C., school district is at the 11th percentile in math relative to students in other developed countries. In Detroit, the average student is at the 12th percentile. In Milwaukee, the average student is at the 16th percentile. Cleveland is at the 18th percentile. The average student in Baltimore is at the 19th percentile in math relative to students in other developed countries. In Los Angeles, the average student is at the 20th percentile. The average student in Chicago is at the 21st percentile in math. Atlanta is at the 23rd percentile. The average student in New York City is at the 32nd percentile in math. And in Miami-Dade County, the average student is at the 33rd percentile in math.

Not 1 of the largest 20 school districts is above the 50th percentile in math relative to other developed countries. Those districts contain almost 5.2 million students or more than 10 percent of the country’s schoolchildren. The rare and small pockets of excellence in charter schools and rural communities are overwhelmed by large pools of failure.

Previous Research

The Global Report Card is not the first analysis to compare the performance of U.S. students to international peers. Eric A. Hanushek, Paul E. Peterson, and Ludger Woessmann (see “Teaching Math to the Talented,” features, Winter 2011) used a very similar method to compare the performance of students in each state to students in other countries and arrived at similarly gloomy conclusions. Using state NAEP results for 8th-grade students and PISA results for 15-year-olds internationally, the researchers focused on the percentage of students performing at an advanced level in math. In almost every state, they found that we had far fewer advanced students than most of the countries taking PISA. They also narrowed the comparison to white students in the U.S. and to students whose parents had a college education to show that even advantaged students in the U.S. failed to achieve at an advanced level in math relative to their international peers. More recently, Hanushek et al. updated their analysis to examine the percentage of students in each state and across countries performing at the proficient level in math and reading.  The results were similarly disappointing.

The main difference between the GRC and the Hanushek et al. analyses is that in our study we push the comparison down to the district level. By focusing on white students and children of college-educated parents, Hanushek et al. clearly mean to convey that even students in elite suburban districts have mediocre achievement. Our contribution with the GRC is to name the districts so that people do not indulge the fantasy that their suburb’s record is somehow different from the disappointing performance of others with advantaged students in their state.

There are other important differences between the GRC and the Hanushek et al. analyses. We incorporate test results for U.S. students in all available grades (typically grades 3 through 8 and grade 10) rather than focusing on the grade closest to the 15-year-olds in the PISA sample. We could have focused only on 8th-grade results, as Hanushek et al. did, but in doing so we would have greatly reduced the number of test results on which we were doing the calculations for school districts. We preferred to gain precision in estimating the achievement in each district by increasing our sample size rather than restricting the sample to 8th graders in order to gain comparability in the age of the students under review.

The GRC analysis also differs from those of Hanushek et al. in that the latter focus on students performing at the advanced or proficient level, while we focused on the average student performance in both math and reading. Hanushek et al. concentrated on advanced or proficient performance because they were trying to compare our best students with the best abroad to show that even our best are mediocre. We did the same by highlighting the results for elite suburban school districts. Focusing on the average also avoids any dispute about how “advanced” or “proficient” are defined across different tests.

Gary Phillips at the American Institutes for Research has also conducted a series of analyses comparing state achievement on NAEP to international performance on a different international test, the Trends in International Mathematics and Science Study (TIMSS). Phillips arrives at somewhat less gloomy conclusions about U.S. performance, but that is because the countries included in TIMSS differ from those covered by PISA. Hanushek et al. rightly note that PISA provides a much more appropriate comparison for the U.S.: “Put starkly, if one drops from a survey countries such as Canada, Denmark, Finland, France, Germany, and New Zealand, and includes instead such countries as Botswana, Ghana, Iran, and Lebanon, the average international performance will drop, and the United States will look better relative to the countries with which it is being compared.”

This has sparked a debate among researchers about whether TIMSS or PISA provides a better set of countries against which we should compare the U.S. The Global Report Card circumvents this dispute by developing its own set of countries against which we compare U.S. students. The comparisons provided by TIMSS and PISA depend on which countries decide to take each test each time it is administered. And PISA scales its scores against the results for members of the OECD, which excludes countries like Singapore while including countries like Mexico. Our comparison group depends on PISA results, but it is also based on objective criteria, like per-capita GDP, to identify a set of developed economies that can be reasonably compared with that of the U.S. Our comparison group is a significant improvement on the self-selection of countries that choose to take a test as well as an improvement upon arbitrary membership in an organization like the OECD.

No Refuge

The elites, the wealthy families that have a disproportionate influence on politics, clearly recognize the dysfunction of large urban school districts and have sought refuge in affluent suburban districts for their own children. But the reality is that there are relatively few pockets of excellence to which these families can flee.

In four states, there is not a single traditional district with average student achievement above the 50th percentile in math. In 17 states, there is not a single traditional district with average achievement in the upper third relative to our global comparison group. And apart from charter school districts,  in over half of the states, there are no more than three traditional districts in which the average achievement would be in the upper third.

The elites in those states have almost nowhere to find an excellent public education for their children. But state accountability systems and the desire to rationalize the lack of quality options have encouraged the elites to compare their affluent suburban districts to the large urban ones in their state. These inappropriate comparisons have falsely reassured them that their own school districts are doing well.

This false reassurance has also perhaps undermined the desire among the elites to engage in dramatic education reform. As long as the elites hold onto the belief that their own school districts are excellent, they have little desire to push for the kind of significant systemic reforms that might improve their districts as well as the large urban districts. They may wish the urban districts well and hope matters improve, but their taste for bold reform is limited by a false contentment with their own situation.

But the elites should not take comfort from the stronger performance of affluent suburban districts relative to large urban districts. As the Global Report Card reveals, even our best public-school districts are mediocre when compared with the achievement of students in a set of countries with developed economies.

Of course, the Global Report Card does not isolate the extent to which schools add or detract from student performance. Factors from student backgrounds, including their parents, communities, and individual characteristics, have a strong influence on achievement. But the GRC does tell us about the end result for student achievement of all of these factors, schools included. And that end result, even in our best districts, is generally disappointing.

Jay P. Greene is professor of education reform at the University of Arkansas and a fellow at the George W. Bush Institute. Josh B. McGee is vice president for public accountability initiatives at the Laura and John Arnold Foundation.

Are America’s highest achieving students being left behind?

A trio of recent studies and articles raises troubling questions about America’s “Achievement-Gap Mania.” Are we leaving our highest performing students behind in the quest to raise the test scores of students at the bottom? If so, what will this mean for our future international competitiveness?

Learn about the recent studies–Fordham’s Do High Flyers Maintain their Altitude? and the George W. Bush Institute’s Global Report Card—as well as Frederick M. Hess’s new National Affairs essay, “Our Achievement-Gap Mania.” And join a conversation about whether our focus on raising the bottom is blinding us to trouble at the top.

Panelists:

• Ulrich Boser, Senior Fellow at the Center for American Progress
• John Cronin, Director of the Kingsbury Center for Research on Academic Growth at the Northwest Evaluation Association
• Frederick M. Hess, Resident Scholar and Director of Education Policy Studies at American Enterprise Institute
• Josh McGee, Vice President for Public Accountability Initiatives at the Laura and John Arnold Foundation

Moderator:

• Chester E. Finn, Jr., President, Thomas B. Fordham Institute

Find more information on student achievement and the global report card here.

The study on which the Global Report Card is based, “When the Best is Mediocre: Developed countries far outperform our most affluent suburbs,” by Jay Greene and Josh McGee, will appear in the Winter 2012 issue of Ed Next and is now available online.

The rankings of 13,636 U.S. school districts can be found in the Global Report Card, available on the website of the George W. Bush Institute, where readers can see how students in each school district compare to students in 25 other nations.

A detailed explanation of the methods used to conduct the analysis is available here.

The study, “When the Best is Mediocre,” by Jay Greene and Josh McGee, will appear in the Winter 2012 issue of Education Next, and is now available online.

Readers can check out the rankings of 13,636 U.S. school districts, and see how students in each district compare to students in 25 other nations, in a Global Report Card available on the website of the George W. Bush Institute. There readers can also find a detailed explanation of the methods used to conduct the analysis.

—Chester E. Finn, Jr.

American 15-year-olds continue to perform no better than at the industrial-world average in reading and science, and below that in mathematics. According to the results of the 2009 Program for International Student Assessment (PISA) tests, released in December 2010 by the Organisation for Economic Co-operation and Development (OECD), the United States performed only at the international average in reading, and trailed 18 and 23 other countries in science and math, respectively. Students in China’s Shanghai province outscored everyone.

Many have identified variations in teacher quality as a key factor in international differences in student performance and have urged policies that will lift the quality of the U.S. teaching force. To that end, President Barack Obama has called for a national effort to improve the quality of classroom teaching and repeatedly indicated his support for policies that would provide financial rewards for outstanding teachers.

In a March 2009 speech to the Hispanic Chamber of Commerce, he explained,

Good teachers will be rewarded with more money for improved student achievement, and asked to accept more responsibilities for lifting up their schools. Teachers throughout a school will benefit from guidance and support to help them improve.

In the administration’s Race to the Top initiative, the U.S. Department of Education encouraged states to devise performance pay plans for teachers in the hope that such an intervention could have a significant impact on student performance.

But is there anything in the data the OECD has accumulated to give policymakers reason to believe that merit pay works? Do the countries that pay teachers based on their performance score higher on PISA tests? Based on my new analysis, the answer is yes. A little-used survey conducted by the OECD in 2005 makes it possible to identify the developed countries participating in PISA that appear to have some kind of performance pay plan. Linking that information to a country’s test performance, one finds that students in countries with performance pay perform at higher levels in math, science, and reading. Specifically, students in countries that permit teacher salaries to be adjusted for outstanding performance score approximately one-quarter of a standard deviation higher on the international math and reading tests, and about 15 percent higher on the science test, than students in countries without performance pay. These findings are obtained after adjustments for levels of economic development across countries, student background characteristics, and features of national school systems.

I draw these conclusions cautiously, as my study is based on information on students in just 27 countries, and the available information on the extent of performance pay in a country is far from perfect. Further, the analysis is based on what researchers refer to as observational rather than experimental data, making it more difficult to make confident statements regarding causality.

It is possible that what I have observed is the opposite of what it seems: countries with high student achievement may find it easier to persuade teachers to accept pay for performance, thereby making it appear that merit pay is lifting achievement. More generally, both performance pay and higher levels of achievement could be produced by some set of factors other than all of those taken into account in the analysis. For example, performance pay could be more widely used in places where, as in Asia, cultural expectations for student performance are high, making it appear that performance pay systems are effective, when in fact both performance pay plans and student achievement are the result of underlying cultural characteristics. But even if my findings are not indisputable, I did carry out a variety of checks to see if any observable factor, such as Asian-European differences, could account for the conclusion. Thus far, I have been unable to find any convincing evidence that the findings are incorrect. Given that, let us take a closer look at what can be learned about the impact of performance pay from PISA data.

Prior Research

Standard economic theory predicts that workers will exert more effort when monetary rewards are tied to the amount of the product they produce. Not only does performance pay stimulate individual effort on the job, it is theorized, but jobs where rewards are tied to effort attract energetic, risk-taking employees who are likely to be more productive. This latter consideration, says Stanford economist Edward Lazear, “is perhaps the most important” way in which a merit pay plan can influence worker performance. But if economists expect positive results from merit pay, many educators believe that teachers are motivated primarily by the substantive mission of the teaching profession and that they do not respond to—indeed, they may resent and resist—monetary incentives that tie salary levels to performance indicators.

To see whether the education sector is an exception to general economic theory, a number of performance pay experiments have been carried out, and in Israel and India such studies have shown positive impacts on student achievement. Experimental studies have tracked only the short-term impact of merit pay, however, and so have not identified any long-term effects that might come from changes in the kinds of people who choose to go into this line of work. Conceivably, a merit pay system could discourage entry into the profession of potentially excellent teachers reluctant to subject themselves to the requirements of a pay-for-performance scheme. Alternatively, if performance pay makes teaching more attractive to talented workers, short-term evaluations could understate its benefits.

One way to capture the long-term effects of teacher performance pay, including changes in the characteristics of those choosing to become a teacher, is to compare countries with performance pay systems to those without. This is now possible because the OECD in 2005 administered a separate survey to each of its member countries concerning the teacher compensation systems in place during the 2003–04 school year, when the 2003 PISA study was conducted. The 2003 PISA provides test score results in math, reading, and science for representative samples of 15-year-olds within each country, or nearly 200,000 students altogether. (The relative performance of countries on the PISA changed only slightly between the 2003 and 2009 tests. For example, in no subject did the scores for the United States differ significantly between 2003 and 2009.)

The PISA study is particularly useful, because it also includes information on a wide variety of family, school, and institutional factors that are likely determinants of student achievement. My analysis adjusts, at the level of the individual student, for such characteristics as the student’s gender and age, preprimary education, immigration status, household composition, parent occupation, and parent employment status. Nine measures of school resources and location are available, including class size, availability of materials, instruction time, teacher education, and size of community. Country-level variables included in the analysis were per capita GDP, teacher salary levels, average expenditure per student, external exit exams, school autonomy in budget and staffing decisions, the share of privately operated schools, and the portion of government funding for schools.

The PISA sampling procedure ensured that a representative sample of 15-year-old students was tested in each country. The student sample sizes in the OECD countries range from 3,350 students in 129 schools in Iceland to 29,983 students in 1,124 schools in Mexico. I therefore use weights when conducting my analysis so that each country contributes equally to the estimated effect of performance pay on student achievement.

Measuring Teacher Performance Pay

The measure of performance pay available from the OECD survey is less precise than one would prefer. It simply asks officials in participating countries whether the base salary for public-school teachers could be adjusted to reward teachers who had an “outstanding performance in teaching.” While the survey asked about many other forms of salary adjustments, the study protocol reports that this was the only one that “could be classified as a performance incentive.”Among the 27 OECD countries for which the necessary PISA data are also available, 12 countries reported having adjustments of teacher salaries based on outstanding performance in teaching. The form of the monetary incentive and the method for identifying outstanding performance varies across countries. For example, in Finland, according to the national labor agreement for teachers, local authorities and education providers have an opportunity to encourage individual teachers in their work by personal cash bonuses on the basis of professional proficiency and performance at work. Outstanding performance may also be measured based on the assessment of the head teacher (Portugal), assessments performed by education administrators (Turkey), or the measured learning achievements of students (Mexico). Unfortunately, the coding of the measure does not allow my analysis to consider variation in the scope, structure, and incentives of performance-related pay schemes.

As an example of the limitation of this measure, note that the United States is coded as a country where teacher salaries can be adjusted for outstanding performance in teaching on the grounds that salary adjustments are possible for achieving the National Board for Professional Teaching Standards certification or for increases in student achievement test scores. That policy, however, affects only a few teachers in selected parts of the country. Given such weaknesses in the survey measure, it is all the more remarkable that I was able to detect impacts on student achievement.

Main Results

As noted above, my main analysis indicates that student achievement is significantly higher in countries that make use of teacher performance pay than in countries that do not use it. On average, students in countries with performance-related pay score 24.8 percent of a standard deviation higher on the PISA math test; in reading the effect is 24.3 percent of a standard deviation; and in science it is 15.4 percent (see Figure 1). These effects are similar to the impact identified in the experimental study conducted in India and about twice as large as the one found in a similarly designed Israeli study.

Figure 2 depicts the math result graphically. The figure’s vertical axis displays the average math test scores of students in each country after adjusting for all of the control variables in the model, with the exception of the variable measuring the use of performance pay. The horizontal axis in turn shows the performance pay variable, also after adjusting for those same control variables. The solid line on the figure shows the estimated relationship between these two variables across the 27 countries included in the analysis. It shows a clear positive association between the variation in country-average test scores and the variation in teacher performance pay that cannot be attributed to the other factors included in the analysis.

A lingering concern, however, is that the analysis may be contaminated by the fact that the very cultures that introduce merit pay are those that set high expectations for student achievement. The countries represent widely different cultures, including Asian ones, where expectations for students are often much higher than in Europe and North America. The best way to account for cultural differences among the continents of the world is to control in the analysis for the average effect of living on a particular continent, a strategy known to statisticians as continental fixed effects. Figure 1 thus also shows results based on models that include a fixed effect for each of the four continents with OECD countries: Europe, North America, Oceania, and East Asia. In these models, the effects of pay for performance are shown to be even larger than the results based on comparisons across continents. In other words, the findings cannot be attributed to cultural differences among the major regions of the world, because they are even larger when one looks only at patterns within these regions.

As a further test, I estimated the impact of performance pay for only the 21 participating European countries. Once again, the results showed even larger positive effects than those obtained for the full sample.

Other Sensitivity Tests

When findings are based on small samples, it is important to ascertain whether a conclusion is sensitive to the particular analysis being conducted. Even after conducting a preferred analysis that maximizes use of the information available and best conforms to underlying economic theory, it is important to make sure that the pattern that one has identified is not a statistical accident that readily disappears if a slightly different analysis is conducted. For this reason, I performed a variety of sensitivity tests for math achievement because the reliability of the math test across countries and cultures is usually considered higher than it is for reading or science. Remarkably, the relationship between performance pay and math achievement remained essentially unchanged, regardless of the sensitivity test that I ran.

My first sensitivity check focused on cultural differences among countries that were not captured by the continental fixed effects analysis. In this sensitivity check, I excluded two countries, Mexico and Turkey, which have particularly low levels of GDP per capita. Since it is known that the level of GDP is strongly correlated with educational performance, it may be that the inclusion of these two countries is producing misleading results. But dropping these countries hardly affects results.

The second sensitivity test excluded the level of educational attainment of the teachers, on the grounds that teacher quality might itself be affected by a country’s performance pay policies and therefore should not be used as a control variable. Excluding this variable did not materially change the results from those reported in Figure 1. In a third series of sensitivity tests, I excluded from the analysis one country at a time to make sure that the situation in no one country was driving the overall pattern of results. I found no evidence that that was happening.

The incidence of performance pay is, to some extent, clustered in two regions: Scandinavia (Denmark, Finland, Norway, and Sweden) and Eastern Europe (Czech Republic and Hungary). In a fourth set of sensitivity tests, I separately excluded the countries from these two regions from the analysis to see whether results were highly dependent on one or the other cluster. The results remained unchanged, indicating that neither of these regional clusters is solely responsible for the main result.

A fifth set of sensitivity tests was possible because I have information on other policies that lead to differential pay among teachers. Salaries may vary depending on 1) the teaching conditions and responsibilities (such as taking on management responsibilities, teaching additional classes, and teaching in particular areas or subjects), 2) teacher qualifications and training, or 3) a teacher’s family status and/or age. Since it is possible that student achievement is higher whenever pay schedules are flexible, regardless of the connection to teacher classroom effectiveness, I estimated the impact of each of these three sets of factors on math achievement. None showed a significant impact on performance, and the effect of performance pay remained large and significant, even when these other possible salary adjustments were included in the analysis.

In sum, the main results shown in Figure 1 survive a wide variety of sensitivity tests. That the results are robust to multiple model specifications provides strong evidence that performance pay helps to explain the variation in student performance on the PISA tests.

Differential Effects

With one exception (immigrants benefited less than native-born students from a performance pay regime), I found only small differences in the impact of performance pay on the math achievement of subgroups in the population. Since important differential effects were identified for only one subgroup, one cannot infer that the impact of performance pay on student math learning is concentrated on any particular group of students.

I did, however, find a surprising difference in the way in which a teacher’s education background affects math learning, depending on the presence of a pay-for-performance system. In countries with performance pay, teachers who have an advanced degree in pedagogy do not outperform those without such a degree (the only measure of a teacher’s education available in the PISA data base). However, in countries without performance pay, students learn more in math if they have a pedagogically trained teacher. Perhaps an incentive system washes out any differences that may be caused by variations in teacher training.

Conclusions

The analysis presented above represents the first evidence that, all other observable things equal, students in countries with teacher performance pay plans perform at a higher level in math, reading, and science. The differences in performance are large, ranging from 15 percent (in science) to 25 percent (in math and reading) of a standard deviation. Since one-quarter of a standard deviation is roughly a year’s worth of learning, it might reasonably be concluded that by the age of 15, students taught under a policy regime that includes a performance pay plan will learn an additional year of math and reading and over half a year more in science. However, this conclusion depends on the many assumptions underlying an analysis based on observational data.

Although these are impressive results, before drawing strong policy conclusions it is important to confirm the results through experimental or quasi-experimental studies carried out in advanced industrialized countries. Nothing in the PISA data allows us to identify crucial aspects of performance pay schemes, such as the way in which teacher performance is measured, the size of the incremental earnings received by higher-performing teachers, or very much about the level of government at which or the manner in which decisions on merit pay are made. Studies of such matters are probably better performed within countries, taking advantage of variation in policies within those countries. The study design also does not allow one to tease out the relative importance of the incentive to existing teachers of a performance pay plan as compared to the changes that may take place in teacher recruitment when compensation depends in part on merit rather than just on a standardized pay schedule. Since much more work needs to be done on all of these questions, a wit might insist that performance pay apply to scholars as well.

Ludger Woessmann is professor of economics at the University of Munich and head of the department of Human Capital and Innovation at the Ifo Institute for Economic Research.

An unabridged version of this article is available here.

Unfortunately, each of these common arguments is either terribly misleading or wrong.  Simply looking for blue sky in the test results ignores a substantial body of scientific research.  While many people want to be reassured that things are going just fine, ignoring the real message of these tests actually imperils our economic future.

Let’s start at the top.  The recently released PISA results that compare 15-year-olds around the world in reading, math, and science place U.S. students above the developed-country average in reading, at the average in science, and below average in math.  If one focuses just on reading, perhaps we are not doing so badly, even though we still trail 17 countries.  But reading is very difficult to assess accurately in the international tests.  And reading scores have proven less important than math and science for both individual and national success.  In math, we place 31^st in the world rankings.

Research has shown that international performance on these tests is very closely related to the economic growth of nations.  Does the difference between 550 points (roughly Finland) and 500 points (roughly the U.S.) make a difference?  By the historical record of growth, such a difference is consistent with one percent per year in the growth of per capita income.  If we project this out over the lifetimes of children born today, the present value of economic gains from the U.S. reaching the level of Finland would be $100 trillion!  These potential economic gains from improved schools should be compared to the huge political fights in the U.S. over a stimulus package of one trillion dollars, or one hundredth of the magnitude of the gains we are leaving on the table from ignoring the achievement in our schools.

The challenge to the U.S. is clearest when one looks at the proportion of students achieving at the advanced level in math.  Presumably our scientists, engineers, and innovators are drawn from these high performers.  Paul Peterson, Ludger Woessmann, and I assessed not only how well our best states were doing but also how well our white students and our children of college educated parents were doing in advanced skills.

The performance of U.S. students of the Class of 2009 as a whole trails 29 countries.  Sixteen countries actually produce twice the proportion of advanced math students that we do.  And there are more highly talented math students in the whole population of 18 countries than in U.S. families with a college educated parent.

Yes, the U.S. has had advantages that have covered up the poor performance of our schools.  The free and open labor and product markets of the U.S. along with the generally limited intrusion of the government and respect for individual property rights have promoted an innovative society and have attracted the brightest from abroad.  But our relative advantages in these areas are swiftly eroding as other countries emulate our economic institutions and as other countries attract their bright and well-trained students back to work at home.

The feel-good message offers solace to those who counsel maintaining the current course.  It is, however, a bad message that truly threatens our economic future.   To be sure, it will not show up very clearly for some time, maybe even a decade or two.  By then, recovery will at least be much more difficult, if not impossible.

- Eric A. Hanushek

Too melodramatic? Maybe you’d prefer “Sixty-three years after Sputnik caused an earthquake in American education by giving us reason to believe that the Soviet Union had surpassed us, China delivered the aftershock.”

It came via yet another wonky study, The PISA 2009 Results: What Students Know and Can Do, reporting that on a test of math, reading and science given to fifteen year olds in sixty-five countries in 2009, Shanghai’s 15-year-olds topped those in every other jurisdiction in ALL THREE SUBJECTS. What’s more, Hong Kong ranked in the top four on all three assessments.

Though Hong Kong took part in earlier rounds of the OECD’s PISA (Programme for International Student Assessment), the 2009 testing cycle marked the first time that youngsters in China proper participated. To be sure, it was only Shanghai, the country’s flagship city in so many ways, a single megalopolis on which Beijing has lavished much investment and attention, many favorable policies and even (for China) a relatively high degree of freedom. But Americans—and the rest of the world—would make a big mistake to suppose for one second that this Shanghai result is some sort of aberration or unique case.

I have the gravest misgivings about China and the threat that it poses to U.S. interests in the years ahead, but I have the utmost respect for that nation’s capacity to accomplish its own ends and attain its goals, however ruthless it must be. If they can produce top PISA scorers in one city in 2009—keep in mind that Shanghai’s population of 20 million is roughly that of Florida, New York State or one-third of France—they can do this in ten cities in 2019 and fifty in 2029. Or maybe faster.

I admit to misgivings about PISA, too, about how it defines knowledge, what it tests, and how it tries to divorce itself from school curriculum. But its international rankings are widely trusted as a reliable barometer of how young people in different countries compare in core academic subjects. And what the 2009 results show is that China, what it sets out to do well on PISA, is fully capable of doing so.

How did Shanghai accomplish this? The OECD folks offer some explanations, terming Shanghai a “leader in reform” and citing in particular its near-universal education system, its competitiveness (including admission both to universities and to the best secondary schools), a very high level of student engagement, a modernized assessment system, an ambitious new curriculum, and a program of intervention into weak schools.

Most of China isn’t doing those things today. Tomorrow, however, is apt to be a very different story.
Also near the top on PISA in 2009 were a half dozen countries that we’re used to seeing there: Singapore, Taipei, Finland, Korea, Japan, etc. In reading, Canada, New Zealand, Australia, and the Netherlands also did well. The United States was, once again, in the middle of the pack in reading and science and a bit below the international average in math. We didn’t do badly. We’re not getting worse. But we’re not getting better, either, and other countries are.
Plenty of people have been pointing this out for a long time now. Our trend lines are essentially flat while others are rising. But until this week we could at least pretend that China wasn’t one of those countries that was a threat in education. We could treat Hong Kong as a special case—the British legacy, you know, combined with prosperity. We could believe that China was only interested in building dams, buying up our currency, making fake Prada bags, underselling everybody else, and coating our kids’ toys with toxic paint, while neglecting its education system. Yes, we knew they were exporting Chinese teachers to teach Mandarin (and who knows what else) in our schools while importing native English speakers to instruct their children in our language. But we could comfort ourselves that their curriculum emphasized discipline and rote learning, not analysis or creativity.

Today that comfort has been stripped away. We must face the fact that China is bent on surpassing us—and everyone else—in K-12 education, too, and that they are accomplishing precisely that goal, today in Shanghai but tomorrow in many more parts of that vast land.

Will this be the wake-up call that America needs to get serious about educational achievement? Will it be the Sputnik of our time? Will it stir us out of our torpor and get us beyond our excuse-making, our bickering over who should do what, our prioritizing of adult interests and our hang-ups about the very kinds of changes that China is now making while we dither?

I surely hope so. You should, too. This is serious.

– Chester E. Finn, Jr.

In Vancouver last winter, the United States proved its competitive spirit by winning more medals—gold, silver, and bronze—at the Winter Olympic Games than any other country, although the German member of our research team insists on pointing out that Canada and Germany both won more gold medals than the United States. But if there is some dispute about which Olympic medals to count, there is no question about American math performance: the United States does not deserve even a paper medal.

Maintaining our productivity as a nation depends importantly on developing a highly qualified cadre of scientists, engineers, entrepreneurs, and other professionals. To realize that objective requires a system of schooling that produces students with advanced math and science skills. To see how well schools in the United States do at producing high-achieving math students, we compared the percentage of U.S. students in the high-school graduating Class of 2009 with advanced skills in mathematics to percentages of similarly high achievers in other countries.

Unfortunately, we found that the percentage of students in the U.S. Class of 2009 who were highly accomplished in math is well below that of most countries with which the United States generally compares itself. No fewer than 30 of the 56 other countries that participated in the Program for International Student Assessment (PISA) math test, including most of the world’s industrialized nations, had a larger percentage of students who scored at the international equivalent of the advanced level on our own National Assessment of Educational Progress (NAEP) tests. Moreover, while the percentage of students scoring at the advanced level on NAEP varies considerably among the 50 states, not even the best state does well in international comparison. A 2005 report from the National Academy of Sciences, Rising Above the Gathering Storm, succinctly put the issue into perspective: “Although many people assume that the United States will always be a world leader in science and technology, this may not continue to be the case inasmuch as great minds and ideas exist throughout the world.”

The Demand for High Achievers

The gap between the burgeoning business demand for a highly accomplished workforce and a lagging education system has steadily widened. Even as the United States was struggling with a near 10 percent unemployment rate in the summer of 2010, businesses complained that they could not find workers with needed skills. New York Times writer Motoko Rich explained, “The problem…is a mismatch between the kind of skilled workers needed and the ranks of the unemployed.”

Skill shortages have severe consequences for a nation’s overall productivity. Two of the authors of this report have shown elsewhere that countries with students who perform at higher levels in math and science show larger rates of increase in economic productivity than do otherwise similar countries with lower-performing students (see “Education and Economic Growth,” research, Spring 2008).

Public discourse has tended to focus on the need to address low achievement, particularly among disadvantaged students. Both federal funding and the accountability elements of No Child Left Behind (NCLB) have stressed the importance of bringing every student up to a minimum level of proficiency. As great as this need may be, there is no less need to lift more students, no matter their socioeconomic background, to high levels of educational accomplishment. In 2006, the Science, Technology, Engineering, and Mathematics (STEM) Education Coalition was formed to “raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace for the 21st Century.” In the words of a National Academy of Sciences report that jump-started the coalition’s formation, the nation needs to “increase” its “talent pool by improving K–12 science and mathematics education.”

A Focus on Math

We give special attention to math performance because math appears to be the subject in which accomplishment in secondary school is particularly significant for both an individual’s and a country’s economic well-being. Existing research, though not conclusive, indicates that math skills better predict future earnings and other economic outcomes than other skills learned in high school. The American Diploma Project estimates that “in 62 percent of American jobs over the next 10 years, entry-level workers will need to be proficient in algebra, geometry, data interpretation, probability and statistics.”

There is also a technical reason for focusing our analysis on math. This subject is particularly well suited to rigorous comparisons across countries and cultures. There is a fairly clear international consensus on the math concepts and techniques that need to be mastered and on the order in which those concepts should be introduced into the curriculum. The knowledge to be learned remains the same regardless of the dominant language spoken in a culture.

Data and Methodology

Our analysis relies on test-score information from NAEP and PISA. NAEP, the National Assessment of Educational Progress, is often called the nation’s report card. It is a large, nationally representative assessment of student performance in public and private schools in mathematics, reading, and science that has been administered periodically since the early 1970s to U.S. students in 4th grade and 8th grade, and at the age of 17. PISA, the Program for International Student Assessment, is an internationally standardized assessment of student performance in mathematics, science, and reading established by the Organisation for Economic Co-operation and Development (OECD). It was administered in 2000, 2003, and 2006 to representative samples of 15-year-olds in all 30 OECD countries (which include the most developed countries of the world) as well as in many others.

We focus on performance of the international equivalent of the U.S. high-school graduating Class of 2009 at the time when this population was in the equivalent of U.S. grades 8 and 9. NAEP was administered to U.S. 8th graders in 2005, while PISA 2006 was given one year later to students at the age of 15, the year at which most American students are in 9th grade.

In 2005, NAEP tested representative samples of 8th-grade public and private school students in each of the 50 states in math, science, and reading. For each state, NAEP 2005 calculates the percentage of students who meet a set of achievement standards: a “basic” level, a “proficient” level, and an “advanced” level of achievement. The focus of this report is the top performers, the percentage of students NAEP found at the advanced level of achievement (subsequently referred to as “advanced”).

Only 6.04 percent of the students in the United States in 8th grade in 2005 scored at the advanced level in math on the NAEP. Some critics feel that the standard set by the NAEP governing board is excessively stringent. However, the 2007 Trends in International Math and Science Study (TIMSS 2007), another international test that has been administered to students throughout the world, appears to have set a standard very similar to NAEP 2005, as only 6 percent of U.S. 8th graders scored at the advanced level on that test as well.

We use the NAEP 2005 advanced standard to compare U.S. performance with that in other countries. Because U.S. students took both NAEP 2005 and PISA 2006, it is possible to find the score on PISA that is tantamount to scoring at the advanced level on NAEP, i.e., the score that will yield the same percentage of students as the percentage of U. S. students who scored at the advanced level on the NAEP.

A score on PISA 2006 of 617.1 points is equivalent to the lowest score attained by anyone in the top 6.04 percent of U.S. students in the Class of 2009. (The PISA assessment has an average score of 500 among OECD students and a standard deviation of 100.) It is assumed that both NAEP and PISA tests randomly select questions from a common universe of mathematics knowledge. Given that assumption, it may be further assumed that students who scored similarly on the two exams will have similar math knowledge, i.e., students who scored 617.1 points or better on the PISA test would have been identified at the advanced level had they taken the NAEP math test. Inasmuch as a score of 617.1 points is more than one standard deviation above the average student score on the PISA, it is clear that a group of highly accomplished students has been isolated. (For more methodological details, see sidebar.)

Because representative samples of student performance on NAEP 2005 are available for each state, it is possible to compare the percentages of students in the Class of 2009 who were at the advanced level for each state to the percentage of equally skilled students in countries from around the globe.

In short, linking the scores of the Class of 2009 on NAEP 2005 and PISA 2006 provides us with the opportunity to assess from an international vantage point how well the country as well as individual states in the United States are doing at lifting students to high levels of accomplishment.

U. S. Math Performance in World Perspective

We begin with an overall assessment of the relative percentages of young adults in the United States and other countries who have reached a very high level of mathematics achievement. It is frequently noted that the United States has a very heterogeneous population, with large numbers of immigrants. Such a diverse population, with students coming to school with varying preparation, may handicap U.S. performance relative to that of other countries. For this reason, we also examine two U.S. subgroups conventionally thought to have better preparation for school—white students and students from families where at least one parent is reported to have received a college degree—and compare the percentages of high-achieving students among them to the (total) populations abroad.

Overall results. The percentage of students in the U.S. Class of 2009 who were highly accomplished is well below that of most countries with which the United States generally compares itself. While just 6 percent of U.S. students earned at least 617.1 points on the PISA 2006 exam, 28 percent of Taiwanese students did. (See Figure 1 for these results as well as for the international rank of each U.S. state.)

Click to enlarge

It is not only Taiwan that did much, much better than the United States. At least 20 percent of students in Hong Kong, Korea, and Finland were similarly highly accomplished. Twelve other countries had more than twice the percentage of advanced students as the United States: in order of math excellence, they are Switzerland, Belgium, the Netherlands, Liechtenstein, New Zealand, the Czech Republic, Japan, Canada, Macao-China, Australia, Germany, and Austria.

The remaining countries that educate a greater proportion of their students to a high level are Slovenia, Denmark, Iceland, France, Estonia, Sweden, the United Kingdom, the Slovak Republic, Luxembourg, Hungary, Poland, Norway, Ireland and Lithuania.

The 30-country list includes virtually all the advanced industrialized nations of the world. The only OECD countries producing a smaller percentage of advanced math students than the United States are Portugal, Greece, Turkey, and Mexico. The performance levels of students in Spain and Italy are statistically indistinguishable from those of students in the United States, as are those of students in Latvia, which has subsequently joined the OECD.

State-level performance. The percentage of students scoring at the advanced level varies among the 50 states. Massachusetts, with over 11 percent of its students at the advanced level, does better than any other state, but its performance trails that of 14 countries. Its students’ achievement level is similar to that of Germany and France. Minnesota, with more than 10 percent of its students at the advanced level, ranks second among the 50 states, but it trails 16 countries and performs at the level attained by Slovenia and Denmark. New York and Texas each have a percentage of students scoring at the advanced level that is roughly comparable to the United States as a whole, Lithuania, and the Russian Federation.

Just 4.5 percent of the students in the Silicon Valley state of California are performing at a high level, a percentage roughly comparable to that of Portugal. The lowest-ranking states—West Virginia, New Mexico, and Mississippi—have a smaller percentage of the highest-performing students than Serbia or Uruguay, although they do edge out Romania, Brazil, and Kyrgyzstan.

In short, the percentages of high-achieving students in the United States—and in most of its individual states—are shockingly below those of many of the world’s leading industrialized nations. Results for many states are at a level equal to those of third-world countries. (Click the image below for an interactive map providing specific information for each state.)

Click to find specific information for each state

White students. The overall news is sobering. Some might try to comfort themselves by saying the problem is limited to large numbers of students from immigrant families, or to African American students and others who have suffered from discrimination. For example, the statement by the STEM Coalition that we “encourage more of our best and brightest students, especially those from underrepresented or disadvantaged groups, to study in STEM fields” suggests that the challenges are concentrated in nonwhite segments of the U.S. population.

Without denying that the paucity of high-achieving students within minority populations is a serious issue, let us consider the performance of white students for whom the case of discrimination cannot easily be made. Twenty-four countries have a larger percentage of highly accomplished students than the 8 percent achieving at that level among the U.S. white student population in the Class of 2009. Looking at just white students places the U.S. at a level equivalent to what all students are achieving in the United Kingdom, Hungary, and Poland. Seven percent of California’s white students are advanced, roughly the percentage for all Lithuanian students.

Children of parents with college degrees. Another possibility is that schools help students reach levels of high accomplishment if parents are providing the necessary support. To explore this possibility, we assumed that students who reported that at least one parent had graduated from college were likely to be given the kind of support that is needed for many to reach high levels of achievement. Approximately 45 percent of all U.S. students reported that at least one parent had a college degree.

The portion of students in the Class of 2009 with a college-graduate parent who are performing at the advanced level is 10.3 percent. When compared to all students in the other PISA countries, this advantaged segment of the U.S. population was outranked by students in 16 other countries. Nine percent of Illinois students with a college-educated parent scored at the advanced level, a percentage comparable to all students in France and the United Kingdom. The percentage of highly accomplished students from college-educated families in Rhode Island is just short of 6 percent, the same percentage for all students in Spain, Italy, and Latvia.

The Previous Rosy Gloss

Many casual observers may be surprised by our findings, as two previous, highly publicized studies have suggested that—even though improvement was possible—the U.S. was doing all right. This was the picture from two reports issued by Gary Phillips of the American Institutes for Research, who compared the average performance in math of 8th-grade students in each of the 50 states with the average scores of 8th-grade students in other countries. These comparisons used methods that are similar to ours to relate 2007 NAEP performance for U.S. students to both TIMSS 2003 and TIMSS 2007. His findings are more favorable to the United States than those shown by our analyses. While our study using the PISA data shows U.S. student performance in math to be below 30 other countries, Phillips found the average U.S. student to be performing better than all but 14 other countries in his 2007 report and all but 8 countries in his 2009 report. (Oddly, the 2007 report takes a much more buoyant perspective than the 2009 report, though the data suggest otherwise.) Phillips also finds that individual states do much better vis-à-vis other countries than we report.

Why do two studies that seem to be employing generally similar methodologies produce such strikingly different results?

The answer to that puzzle is actually quite simple and has little to do with the fact that Phillips compares average student performance while our study focuses on advanced students: many OECD countries, including those that had a high percentage of high-achieving students, participated in PISA 2006 (upon which our analysis is based) but did not participate in either TIMSS 2003 or TIMSS 2007, the two surveys included in the Phillips studies. In fact, 19 countries that outscored the U.S. on the PISA 2006 test did not participate in TIMSS 2003, and 22 higher-scoring countries did not participate in TIMSS 2007. As a report by the U.S. National Center for Education Statistics has explained, “Differences in the set of countries that participate in an assessment can affect how well the United States appears to do internationally when results are released.”

Put starkly, if one drops from a survey countries such as Canada, Denmark, Finland, France, Germany, and New Zealand, and includes instead such countries as Botswana, Ghana, Iran, and Lebanon, the average international performance will drop, and the United States will look better relative to the countries with which it is being compared.

Did NCLB shift the focus away from the best and the brightest?

Some attribute the comparatively small percentages of students performing at the advanced level to the focus of the 2002 federal accountability statute, No Child Left Behind, on the educational needs of very low performing students. That law mandates that every student be brought up to the level a state deems proficient, a standard that most states set well below NAEP’s proficient standard, to say nothing of the advanced level that is the focus of this report.

In order to comply with the federal law, some assert, schools are concentrating all available resources on the educationally deprived, leaving advanced students to fend for themselves. If so, then we should see a decline in the percentage of students performing at NAEP’s advanced level subsequent to the passage of the 2002 federal law. In mathematics, however, the opposite has happened. The percentage performing at the advanced level was only 3.7 percent in 1996 and 4.7 percent in the year 2000. But the percentage performing at an advanced level climbed steadily to the 7.9 percent attained in 2009.

Perhaps NCLB’s passage in 2002 dampened the prior rate of growth in the achievement of high-performing students. To ascertain whether that was the case, we compared the rate of change in the NAEP math scores of the top 10 percent of all 8th graders between 1990 and 2003 (before NCLB was fully implemented) with the rate of change after NCLB had become effective law. Between 1990 and 2003, the scores of students at the 90th percentile rose from 307 to 321, an increment of 14 points, or a growth rate of 1.0 points a year. Between 2003 and 2009, the shift upward for the 90th percentile was another 8 points, or a change of 1.3 points a year. Our results are confirmed by a more detailed study of NCLB’s impact on high-performing students conducted by economists Brian Jacob and Thomas Dee.

In short, the incapacity of American schools to bring students up to the highest level of accomplishment in mathematics is much more deepseated than anything induced by recent federal legislation.

Conclusions

The economic and technological demand for a talented, well-educated, highly skilled population has never been greater. Not only must everyday workers have a set of technical skills surpassing those needed in the past, but a cadre of highly talented professionals trained to the highest level of accomplishment is needed to foster innovation and growth. In the words of President Barack Obama, “Whether it’s improving our health or harnessing clean energy, protecting our security or succeeding in the global economy, our future depends on reaffirming America’s role as the world’s engine of scientific discovery and technological innovation. And that leadership tomorrow depends on how we educate our students today, especially in math, science, technology, and engineering.”

Unfortunately, the United States trails other industrialized countries in bringing a large proportion of its students up to the highest levels of accomplishment. This is not a story of some states doing well but being dragged down by states that perform poorly. Nor is it a story of immigrant or disadvantaged or minority students hiding the strong performance of better-prepared students. Comparatively small percentages of white students are high achievers. Only a small proportion of the children of our college-educated population is equipped to compete with students in a majority of OECD countries.

Major policy initiatives within the United States have in recent years focused on the educational needs of low-performing students. Such efforts deserve commendation, but they can leave the impression that there is no similar need to enhance the education of those students the STEM coalition has called “the best and brightest.” Yet, with rapidly advancing technologies in an increasingly integrated world economy, no one doubts the extraordinary importance of highly accomplished professionals.

Admittedly, the United States could simply ignore the needs of its own young people and continue to import highly skilled scientists and engineers who were prepared by better-performing schools abroad. But even such a heartless, irresponsible strategy relies on both the nature of immigration policies and the absence of better opportunities abroad, two things on which we might not want the future to depend. It seems much more prudent to encourage the most capable of our own people to reach high levels of academic accomplishment.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University. Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance and senior fellow at the Hoover Institution. Ludger Woessmann is professor of economics at the University of Munich.

But who’s counting? (No, apparently, Everett Dirksen did not say “a billion here, a billion there, and pretty soon you’re talking real money.”)

What exactly an education dollar is worth is not an inconsequential topic and I started gathering string for a blog post about the newest federal bailout for teachers when I was hit by a stray thought:  James Tooley.

One of the more striking stories I have read on the subject of money and education is one I helped edit at Education Next called “Private Schools for the Poor,” written by Tooley, a one-time mathematics teacher in Zimbabwe and currently professor of education policy at the University of Newcastle upon Tyne, England.

I recall almost gasping at sections of the manuscript, as Tooley  took the reader on a tour of some of the poorest neighborhoods on earth.

“In the slums of Nairobi, Kenya, private schools are made from the same materials as every other building,” he wrote.  “[C]orrugated iron sheets or mud walls, with windows and doors cut out to allow light to enter. Floors are usually mud, roofs sometimes thatched. Children will not be in uniform and will usually be sitting on homemade wooden benches. In the dry season, the wind will blow dust through the cracks in the walls; in the rainy season, the playground will become a pond, and the classroom floors mud baths. Teaching continues, however, through most of these intemperate interruptions.”

Clive Crook of the Atlantic called Tooley’s work “something of an embarrassment to the official aid and development industry” because he had “demonstrated something that many development professionals would rather not know—and would prefer that you not know, either.” Tooley and his research team found that “private education is a principle lifeline for the abjectly poor” and that it worked. “On the whole,” says Crook, “dime-a-day for-profit schools are doing a better job of teaching the poorest children than the far more expensive state schools

Tooley turned the research into a book, The Beautiful Tree, which came out in 2009, and should be on every policymaker’s beach blanket – a reminder not only of our own good fortune to be living in America, but of the need to constantly reconsider education’s relationship to money.

A new article by Mark Schneider, the former commissioner of NCES, warns that states may not get much bang for their buck from participating in international tests.  The article, posted today on the Ed Next website, explains that there are much cheaper ways to find out how the students in individual states compare to students in other countries (such as the statistical linking performed by Gary Phillips of AIR, who has estimated TIMSS scores for all states based on the states’ performance on NAEP).

Schneider also warns that while the purveyors of international tests can be very aggressive about developing policy recommendations based on the test results, the recommendations often include advice built on flimsy data and flawed analysis.

A video in which Schneider talks with Ed Next about his article can be found here.

“countries with more confident students who enjoy the subject matter–and with teachers who strive to make mathematics relevant to students’ daily lives–do not do as well as countries that rank lower on indices of conficence, enjoyment, and relevance.”

It turned out that some of the highest-confidence eighth-graders were some of the worst performers, and the contrary case held as well.  U.S. students rated themselves much more highly than did students in Korea, Japan, Hong Kong, Singapore, the Netherlands, and Chinese Taipei, but they scored well behind that insecure group.  While 93 percent of U.S. eighth-graders failed to achieve an advanced score on the test, only 5 percent of them “Disagreed a lot” with the statement that they “do well in math.”

Another report that just came out suggests that the same discorrelation between confidence and performance may hold in reading.  It appears as “Calibration of reading self-concept and reading achievement among 15-year-olds: Cultural differences in 34 countries” in the September issue of Learning and Individual Differences (volume 19, issue 3, pages 372-386).  Authors Ming Ming Chiu and Robert M. Klassen took data from many countries, examined “self-concept” and “students achievement” in reading, and found that students with under-confidence scored higher than the average in their country, while students with over-confidence scored lower.  This leads them to a firm conclusion.

“Higher reading achievement is linked to not only calibration accuracy but also calibration modesty (i.e., underestimations of performance levels). Students that were modest in their self-appraisals relative to their reading achievement – who were, in fact, underconfident – usually had reading scores that exceeded their country mean while overconfident students often had reading scores below the mean. In this regard, optimistic self-beliefs may signal a developmental lag in cognitive self-awareness.”

While the authors agree that self-confidence may be important for long-term endeavors in which motivation has to carry through from beginning to end, for “specific academic tasks . . . an optimistic or overconfident approach might be dysfunctional.”  While deep-seated doubts can, indeed, be disabling, deep-seated confidence can sometimes “mask real academic difficulties.”  In other words, over-confidence can be a sign not of prior superior achievement, but of inferior achievement, a defense mechanism against poor performance and skill level.

How are teachers in English Language Arts supposed to tell the difference?

As reviewed by Mark Bauerlein

Last spring, in Fairburn, Georgia, officials in two schools piloted a startling attendance program. If struggling 8th and 11th graders showed up for study hall, they could earn $8 an hour, and if their grades and test scores rose significantly, they would receive a bonus. An Associated Press story termed the policy a “bribe,” and a Georgia State University professor on National Public Radio declared it “morally bankrupt.” But Ben Chavis, then principal of American Indian Public Charter School in East Oakland, California, had started paying students for attendance years ago with steady results, doubling math scores in the school over time. “Poor people love money,” he explains, so why not let it motivate the kids? He even met with drug dealers off campus and offered them $5 for every truant they brought back. The cash came from creative budgeting, for instance, no computers for the kids. (“They can’t read,” he declares, “they don’t need a computer!”)

Chavis is one of a handful of school mavericks profiled in Flunked, a 45-minute documentary narrated by actor Joe Mantegna. The film reviews 50 years of public school investment, from Sputnik to No Child Left Behind, and derives a simple lesson: the claim “more money makes more success” is a myth, “the tallest tale of them all.” In spite of massive investment and however you measure it, one commenter says, academic achievement “looks like somebody just died—it’s just a flat line.” Success lies not in raising dollars but in changing the organization.

The “all-stars” in Flunked illustrate how it can happen. They are “entrepreneurial principals,” headstrong heroes who rescue failing schools, run charters, tighten discipline, and lower dropout rates. Steve Barr runs Green Dot Public Schools in Los Angeles, which divides dysfunctional high schools into small charter schools. His first principle: get every dollar into classrooms. He pays teachers well and grants them wide latitude in the classroom in exchange for a “dismissal-for-cause” condition in their contracts. Howard Lappin, who took on a high school in L.A.—“1,600 kids, out-of-control school, violent, terrible test scores”—recites his message for kids: “if you’re not in class you’re in trouble—your parents are gonna be in—we’re gonna talk to you—you’re not gonna be here—you got to do what you got to do because this is a school—this is not a playground.”

The ingredients are plain and they don’t include “Give us more money.”

• Provide strict discipline, longer hours, high expectations
• Give teachers high pay and discretion in the classroom, but hold them to professional standards
• Reduce bureaucracy

A sound approach for these schools, but on the evidence of another recent school documentary, the lessons of Flunked may not apply as we move up the U.S. public school ladder. Two Million Minutes profiles two high schoolers in Bangalore, India, two in Shanghai, China, and two in Carmel High School outside Indianapolis. Ranked in the top 5 percent of U.S. public schools, Carmel has loads of money and top-notch facilities. No need to fire any teachers or collar truants. But, as the film unfolds, a striking deficiency among the American students emerges, one that no in-school policy can address—the drive to compete with their peers.

“Competitiveness,” of course, has become a touchstone of education debate. Two years ago in the Washington Post, Bill Gates warned that unless Americans hit the workplace with math and science skills, they will sink in the knowledge economy and take their nation with them. But American students appear unaffected by what one commenter after another says in Two Million Minutes: We are in a global competition, and we’re losing. From 1985 to 2004, the proportion of bachelor’s degrees awarded in math or science in this country fell from 21.7 to 15.8 percent. Engineering went from 9.8 to 6.2 percent, and the numbers won’t improve soon. On the 2006 American Freshman Survey, only 0.8 percent of entering college students intended to major in math, 0.5 percent in physics. These fields are a micro-niche.

For Asian students, though, math and science degrees are the way to prosperity. These students live with “economic uncertainty,” the film explains, and view math and science study as a form of “economic opportunism,” a “passport out of poverty.” The girl from India wants to be rich, and she terms engineering the “safest” field. She attends a two-hour math tutorial that starts at 7:45 each Saturday morning, and after a break, three more hours of class follow. The boy from India aims to be a physicist (as are his father and sister), and he spends 12 hours a week in evening sessions preparing for the Indian Institute of Technology entrance exam. A half million take the test and only 5,000 win admission. The Chinese boy took his first standardized test in 1st grade, and his regular school day lasts nine hours. He doesn’t claim to be number one, but he loves to win and is, in fact, the top math student in his school. It’s not all math and science. Though the Chinese girl wants a biology career, along with her full school schedule she studies ballet and violin.

And the suburban American kids? The boy is senior class president and a National Merit semifinalist, and the girl ranks in the top 3 percent of her class. He admits, though, “Occasionally, I do homework,” and for a big class project due on Monday he starts preparing a day earlier. She claims to “set high expectations,” but adds, “I’m not that 9-to-5 kind of girl.” She favors medicine because “you get an awesome feeling, it’s really a rewarding experience, I think, being able to, um, save lives.” She’s “well-rounded,” which means doing homework with friends while watching Grey’s Anatomy. He doesn’t “ever want a cubicle,” so he works 20 hours a week in a pasta joint and does graphics for the school paper.

Teachers reflect the same laxity. When handing out an exam, the Carmel teacher assures his students that on one question, “I will accept three of the four answers.” In the Indian classroom, the teacher explains the steps in a calculation and concludes, “Nobody should say ‘I don’t know how to find the tangent.’” When the students pause, she blurts out, “Why are you simply standing there?” Nobody chides the American kids like that.

The American boy wins early admission and a full ride to Purdue, while the American girl gets into Indiana University. They are accepted into top universities, so why work any harder? The policies advocated in Flunked are not the answer here. More money in the classroom and less bureaucracy in the schools will make no difference, nor will stricter discipline or higher expectations as long as the college acceptances come through.

Not one of the Asian kids gets into the first-choice college. That outcome explains the relative efforts, and it puts the American high performers in a dismaying light. Asian kids don’t talk about their “awesome” feelings of helping people. They talk about how it feels to beat the kids sitting next to them. If the American boy and girl landed in an Asian classroom, they would sink to the bottom in a week. Call it classroom Darwinism with predictable results. It’s a survival-of-the-smartest world, with few survivors.

Mark Bauerlein is professor of English at Emory University.

Scholars have attempted to discern the effects of competition between the public and private sectors within the United States and in other countries, but no study, to our knowledge, has attempted to measure systematically the causal impact of competition by looking at variation across countries. Until now, research has been stymied by the fact that any simplistic statistical correlations between the extent of competition and student achievement that might be found are suspect. Countries where more people choose to invest in private schools may have other attributes, such as more income or a greater commitment to education, that lead to higher levels of achievement. If this is the case, any positive correlation between private schooling and student achievement could reflect a country’s income or educational commitment rather than any beneficial effects of competition. Or it may be the case that low-quality public schools increase the demand for private schooling. If so, then it could appear that competition lowered the quality of public schooling when in fact the causal connection was in the opposite direction.

In this study, we solve this conundrum by taking advantage of the historical fact that the amount of competition in education today varies from one country to another for reasons that have little to do with contemporary school quality, or national income, or commitments to education. The extent of private schooling stems in large part from the Catholic Church’s decision in the 19th century to build an alternative system of education wherever they were unable to control the state-run system.

Nineteenth-century Catholic doctrine strongly opposed Catholic attendance at state-run schools that were not controlled by the Church. In the United States, for example, Catholics perceived government-operated “common schools” to be Protestant-dominated institutions that were only ostensibly nonsectarian. Local parishes responded by establishing separate schools in which children received Catholic-infused instruction. The United States was not the only country where this happened. Catholic school systems developed in many other countries, but their size depended on the percentage of Catholics living in that country during this critical period (see sidebar). (In countries where Catholicism was the state religion, there was no perceived need for private schools, however.) As a result, even today the size of the private education sector—and thus the amount of competition between public and private schools—is related to the size of the Catholic population in 1900.

To connect the historical past to competition’s effect on achievement today requires two analytic steps. We first estimate the statistical relationship between the size of the Catholic population in 1900 and the extent of private schooling today in order to capture only that share of the private sector’s size that can be attributed to 19th-century Catholic policies—policies we assume to be otherwise unrelated to contemporary student achievement. Having estimated this relationship between Catholicity in the past and competition in the present, we then use that estimate to isolate the causal effect of private school competition on the achievement of individual students across 29 countries.

Our results confirm that countries with larger shares of Catholics but without an official Catholic state religion in 1900 have significantly larger shares of privately operated schools in 2003. More important, private school competition attributable to past Catholic policies generates higher student achievement in mathematics, reading, and science today. We also show that competition between the public and private sector positively affects the achievement of students attending public schools. Spending on education is also reduced, suggesting that school systems are more productive if they are more competitive.

PISA 2003

For the information on contemporary student achievement we rely on the well-regarded data sets compiled by the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA) in 2003. Working closely with official government agencies, PISA gathered information on the mathematical, scientific, and reading literacy of nationally representative student populations in all 30 OECD countries. The term “literacy” signifies that the PISA measured not only the students’ knowledge of math, reading, and science, but also their ability to use that knowledge to meet real-life challenges. In 2003, PISA made a special effort to measure math literacy, allocating 70 percent of testing time to questions in this subject. PISA assessed the achievement of 15-year-old students in each country, regardless of the grade they attended. This means that, in most participating countries, PISA tested students nearing the end of compulsory schooling.

For purposes of this analysis, we constructed a data set that contained pupil-level test scores for about 220,000 students. We also were able to obtain from PISA student reports of their background characteristics and administrator reports on the characteristics of each student’s school, including such things as school resources and whether the school was public or private. All that information was available from 29 of the 30 OECD countries. (France had to be dropped from the analysis because it did not supply any information on the characteristics of its participating schools.)

We defined a school as private if the principal reported that it was managed directly or indirectly by a nongovernment organization (e.g., a church, trade union, business, or other private entity). A public school was defined as one being managed directly or indirectly by a public education authority, government agency, or governing board appointed by government officials or elected by public franchise. We used these definitions to calculate the share of private schools in a country. Throughout our study, this figure serves as our measure of the extent of contemporary private school competition in each country.

The size of the private sector so defined ranges widely across countries. In the Netherlands, more than three-quarters of 15-year-old students attend privately operated schools. Private school shares in Belgium, Ireland, and Korea are also well above one-half. By contrast, the share of students attending privately operated schools in Greece, Iceland, Italy , New Zealand, Norway, Poland, Sweden, and Turkey is below 5 percent. Just over 6 percent of the American 15-yearolds sampled by PISA attended private schools, a figure that corresponds closely to official estimates of private enrollment at the secondary level from the U.S. Department of Education (see Figure 1).

ESTIMATING COMPETITIVE EFFECTS

Recall that our analysis involves two steps. First, we estimate the amount of contemporary private school competition across our 29 countries that can be accounted for by the share of each country’s population that was Catholic in 1900. Where Catholicism was the official state religion, we assign a value of zero for this variable (even though the size of the Catholic population was quite large). That decision is not as odd as it sounds, as we are interested in Catholicism only insofar as it was a factor contributing to the creation of a private sector, something that clearly was not the case in those countries where Catholicism was the state religion and Catholics had no reason to object to the education provided in state-run schools.

The second step uses the connection between past Catholicism and the contemporary size of the private sector to estimate the impact of competition on student achievement. Specifically, we measure the relationship between Catholic-induced private school competition in a country and the PISA test scores of individual students in math, reading, and science.

In taking this approach, we assume that the density of Catholics in 1900 is not directly related to student achievement today, independent of effects that may occur via school competition. While this assumption cannot be proven, there are good reasons to believe it is well founded. Protestant Christians have historically placed a greater emphasis than have Catholics on the value of education, because Protestants thought individual

Bible reading helped one along the road to salvation. Catholics placed greater emphasis on remaining connected to the traditions and practices of the Church. Interestingly enough, in those 22 majority-Christian countries for which data on literacy in 1900 are available, one finds a strong negative association between Catholic population shares and literacy rates. This strong negative correlation exists even after accounting for the lower gross domestic product (GDP) per capita, which is associated with lower literacy rates, in countries with larger Catholic population shares. So to the extent that we find any beneficial effect of Catholic-induced private school competition, its size is probably depressed by cultural values related to Catholicism. In other words, our approach is more likely to yield underestimates than overestimates of competitive effects.

Of course, the historical prevalence of Catholicism could also have had other consequences, apart from a greater reliance on private schooling, that indirectly affect student achievement. For example, the share of Catholics in a country could have an effect on current GDP per capita or education spending per student. We therefore account for the effect of both of these factors in all of our analyses.

In estimating the effect of private school competition on student achievement, we also adjust for the effects of a host of other factors that can affect individual student performance. In addition to the country-level factors of per capita GDP and education spending per student, we include in our analysis information on the presence or absence of external exit exams (which research suggests are associated with higher achievement) and information on whether the country had a Communist government in 1970 (which may have affected both the size of the private sector and achievement). Student and family background characteristics used in the analysis include a student’s gender, immigration status, exposure to early childhood education, the number of books in the home, and parental occupation and work status. Finally, we account for school resources such as class size, availability of materials, teacher certification and preparation, and amount of time for instruction.

PRIVATE SCHOOL COMPETITION AND STUDENT ACHIEVEMENT

The first step of our analysis confirmed the existence of a statistically strong relationship between the extent of private school competition in 2003 and a country’s Catholic population in 1900, much as the historical record would suggest. A 10-point increase in the percentage of Catholics in 1900 is associated with a 4.7-percentage-point increase in the share of students enrolled in privately operated schools in 2003 (see Figure 2). These results support our basic reasoning that as long as Catholics could not be sure that the emerging public school systems of the 19th century would provide education in line with their church’s demands, they tended to resist state schooling and establish their own private schools alongside the state sector. The consequences of historical differences in denominational shares across countries persist to this very day.

The results from the second step of our analysis are equally striking. Let us begin with the results related to student achievement in mathematics, the subject most extensively assessed in PISA 2003. A 10-percentagepoint increase in the share of national student enrollment in private schools attributable to a historically larger share of Catholics induces an improvement in PISA math scores of 9.1 percent of a standard deviation (see Figure 3). As a benchmark for interpreting the magnitude of this effect, note that the difference in average mathematics test scores between the two grades with the largest share of 15-year-olds (9th grade and 10th grade) in the PISA study was 22.1 percent of a standard deviation. This “grade-level equivalent” provides a rough idea of how much a typical student learns during one school year. By this metric, our estimate of the effect of a 10-percentagepoint increase in private school enrollment is equivalent to 41 percent of a year’s worth of learning in high school.

Because we are able to draw on evidence from a relatively small sample of only 29 countries, the statistical precision of our estimate is not very high. That is, we can say with 95 percent confidence that the effect of a 10-percentage-point increase in the private school share is between 3.9 and 14.2 percent of a standard deviation in test scores. Still, this means we have a very high degree of confidence that the real effect is larger than zero. The bottom line is that students in countries whose larger shares of Catholic population in 1900 induced them to have larger shares of privately operated schools today performed significantly better on the PISA 2003 math test.

As an additional step to address any lingering concerns about Catholicism’s direct influence on student achievement, we conducted both stages of our analysis again, this time accounting for the relationship between contemporary differences in the share of Catholic adherents in a country and student achievement. We found that historical Catholic shares continue to be a strong predictor of the extent of private school competition in a country. In addition, the estimated effect of Catholic-induced private school shares on student achievement increases relative to our first version of the analysis, which did not account for contemporary Catholic adherence. There is now a 12.2 percent of a standard deviation increase in test scores for each 10-percentage-point increase in the private school share in a country. This larger estimate suggests that the true effect may be closer to the upper bound of the interval we identified above.

Why the stronger relationship between private school competition and student achievement? We reason that this may be because, in the latter approach, the Catholic-induced school share was reflecting the slightly negative direct effect of contemporary Catholic adherence on student achievement, a relationship that reveals itself in this version of the analysis, although the estimated effect is just shy of statistical significance. Considered together, these results increase our confidence that we are describing a real, causal relationship between private competition and student performance, rather than effects of cultural differences related to religious adherence.

The estimated effects of the private school share on student achievement are somewhat smaller in science and reading than in math, but they remain substantial, positive, and statistically significant (see Figure 2). A change in the historical Catholic population share that produces a 10-percentage-point increase in the extent of contemporary private school competition generates an improvement of about 5.5 percent of a standard deviation in both science and reading—or more than one-fifth of a grade-level equivalent in these subjects.

To gain additional insights, we also re-ran both stages of our analysis while accounting for the average share of funding that private schools receive from the government. The inclusion of this variable hardly affects our results, suggesting that our findings reflect competitive effects stemming from the private operation of schools and not from differences in funding policies.

EFFECT ON PUBLIC SCHOOL STUDENTS

The previous portions of our study investigated the impact of private competition on student achievement in the educational system as a whole. But what about the effect of private school competition on public schools? To answer this question, we removed all students attending a privately operated school from the sample in each country and analyzed only the academic achievement of students in the public sector.

These results are somewhat more difficult to interpret than our findings above, as they combine the effects from competition with the consequences of student sorting. In other words, some of what we find may be due to high-ability students (and their parents) being more likely to choose private schools, leaving the weaker students in the public sector.

Nonetheless, the results suggest that public school students profit nearly as much from increased private school competition as do a nation’s students as a whole. While our estimates of the effects are somewhat smaller than the estimates for students in both the private and public sectors, the results are not statistically distinguishable. It therefore appears that much of the increased performance of education systems with higher levels of private school competition accrues to students who attend public schools.

EDUCATION SPENDING

The analysis so far has been limited to educational outcomes, estimating the effect of private school competition on students’ achievement. In doing so, we have controlled for possible effects of differences in educational inputs such as class sizes, availability of materials, and aggregate expenditure per student in the country. We wondered, though, whether private school competition also affects the input side of the educational process, specifically educational spending per student.

We again used a two-stage process, with the first stage using historical Catholicism to predict the Catholic-induced share of current private school competition in each country. Then, in a second stage, we measured the relationship across countries between the Catholic-induced share of competition and the cumulative educational expenditure per student up to age 15—a measure that includes both public and private spending. We continued to account for a range of country- and student-level characteristics when making these comparisons, but we now excluded measures of school resources that are likely to be affected by spending levels.

Our results show that private school competition, in addition to raising student achievement, substantially reduced the average spending level of the educational system. Changes in historical shares of Catholics in the population that are associated with a 10-percentage-point increase in the private school share today lead to a $3,209 reduction in cumulative spending per student, or 5.6 percent of the average OECD spending level of $56,947 (see Figure 3).

CONCLUSION

Our findings from an international study of 29 countries speak quite clearly. Competition from private schools improves student achievement, and appears to do so for public school as well as private school students. And it produces these benefits while decreasing the total resources devoted to education, as measured by cumulative educational spending per pupil. Under competitive pressures from private schools, the productivity of the school system measured as the ratio between output and input increases by even more than is suggested by looking at educational outcomes alone. Ironically, although Catholics historically placed less emphasis on education than did adherents of many other religions, their resistance to state-run schooling in many countries helped create institutional configurations that continue to spur student achievement.

Martin R. West is assistant professor of education at Brown University and an executive editor of Education Next.
Ludger Woessmann is professor of economics at the University of Munich and heads the Department of Human Capital and Innovation of the Ifo Institute for Economic Research.

The explanation for this odd fact: since 1981 Chile has had a more comprehensive school choice system than any other country in the world, as well as a system of publicly available information on student test performance. Scholars have thus seen Chile as a place to test theories of school choice. Do students with vouchers learn more in private schools or in those run by municipalities? What is the impact of a voucher system on equality of educational opportunity? The answers to these and related questions have been just about as varied as the number of scholars who have inquired into the matter. On balance, the bulk of the research shows a small educational advantage for students who attend privately operated voucher schools rather than municipal ones. But hardly any study looks at differences among the voucher schools, and none has examined differences between private schools in networks and those that operate on a stand-alone basis. Yet interest in school networks has escalated since many operators of charter schools in the United States have begun to expand their operations beyond a single school. Some have argued that this is the ideal way for connecting school choice to school improvement. If effective schools can expand either by setting up or by “franchising” other schools, school quality can gradually improve. But others say that the formation of networks of schools will lead to a standardization that will undermine the vitality of individual school communities.

Chile is an ideal place for exploring these questions. In 2002, only 53 percent of students were still being educated in municipally run schools, which nonetheless received a good deal of their funding from the vouchers paid for by the national government. Another 9 percent of students attended fee-based private schools that were independently operated and received no government assistance whatsoever. For the most part, these were schools with well-established reputations that served the country’s upper class. The remaining students attended what might be called voucher schools, because the schools, while private, had been since 1981 heavily dependent on the subsidy that the schools received from the national government for each student they enrolled. This sector is the fastest growing segment of the Chilean educational system.

Like American charter schools (see “Brand-Name Charters,” features), Chile’s privately run voucher schools may be part of a larger organization or school network, or operate on their own. Most schools are of the stand-alone or “mom-and-pop” variety: 25 percent of all students in Chile attend such schools. But another 13 percent of students attend schools that are part of a network of two or more schools.

The schools, inside and outside of networks, vary from one another in many ways. Some are operated by teachers who once worked in municipal schools. Others are run by business entrepreneurs. Fifty-nine percent of the network schools are run by nonprofit entities, either religious or secular. For-profit organizations operate the remainder. Some schools as well as some networks are religious. Most networks, and especially those in rural areas, consist of just two or three schools. Only about 20 percent of primary (K–8) private voucher school students attend schools that belong to networks that have more than three schools (see sidebar).

Proponents of school networks say that the networks facilitate the flow of information (such as research on best practices) among schools and provide political benefits, credibility, and legitimacy in the eyes of the community. They argue that larger schooling operations have more access to private investments and loans to expand than smaller operations do. Supporting this view, research on public charter schools in the United States indicates that well-established charter school networks can build credibility for fund-raising more easily than stand-alone charter schools can. The underlying hypothesis is that, all else bring equal, the more a schooling organization facilitates transactions between members of a school’s community, the better the school’s performance. The research conducted so far shows that stand-alone charter and brand-name schools, like their district counterparts, vary widely in quality.

Critics of school networks fear unintended negative consequences. They argue that large centralized operations create hard-to-manage bureaucracies and make it difficult to maintain order and create a sense of community among students, parents, teachers, and administrators. Opponents also claim that large schooling operations grant too much authority to administrators and other professionals far removed from the classroom. Some critics are concerned that consolidation encourages standardization. For instance, they maintain that school networks must establish a brand to be successful, which necessitates relatively uniform operations and services from site to site. They argue that this branded approach to education stifles innovation.

Very little factual information is available to sort out the credibility of these claims and counterclaims. It is thus of interest to examine the Chilean experience, where both network and stand-alone voucher-subsidized schools have been operating for several decades. Information on more than one-quarter million students who were 4th graders in 2002 allows us to compare Spanish language and mathematics achievement in network and stand-alone voucher-subsidized schools. Our findings suggest that network schools in Chile are more effective than stand-alone schools, and that larger networks tend to outperform smaller networks. While we cannot be certain whether the higher performance of network schools is because good schools were the ones to expand or whether networking, by itself, had a positive impact, our results nonetheless add considerably to the sparse information currently available on a question of substantial policy interest.

School Reform in Chile

During the 1980s, the Chilean government decentralized the administration of schools, transferring responsibility for public school management from the Ministry of Education to municipalities (recognized neighborhoods in Chile around which municipal services are organized). The government also changed the education financing scheme. Municipalities began to receive funding from the central government according to the number of students who chose to attend the municipal schools. Any enrollment loss had a direct effect on their education budgets. Equally important, privately run schools that had not charged tuition began receiving the same per-student voucher as the public schools. Tuition-charging (elite) private schools mostly continued to operate without public funding.

Despite the parity in funding, significant differences remained between municipally run schools and privately run voucher schools. First, starting in 1994 municipal elementary schools were not allowed to charge parents fees, while all privately run voucher schools could. Second, municipal schools were required by law to accept all who applied. Private voucher schools, in contrast, were allowed to consider results from admissions tests and parent interviews when making admission decisions. Third, municipal schools had to comply with labor laws that made it virtually impossible to fire a low-performing teacher. Privately run voucher schools had greater freedom to terminate employment. In addition, municipal school teachers received salary increases incrementally, based on years of experience. There were no rules with regard to incremental salary increases in the private school sector.

The policies sparked a movement of students from municipal to previously existing private schools as well as the establishment of new institutions. In 1981, 15 percent of the nearly 2.9 million Chilean K–12 students had been attending private schools that received some public subsidy, and another 7 percent attended elite, unsubsidized private schools. By 1990, 34 percent of students attended privately run voucher schools; by 2002, enrollment in such schools reached 38 percent of the roughly 3.4 million in total enrollment (see Figure 2). The trend continued after 2002, the year in which the data for this study were collected. By 2005, 43 percent of students were enrolled in privately run voucher schools. As indicated above, about one-third of the voucher schools belong to networks, while the remaining two-thirds operate independently.

Beginning in 2003, after our data were collected, the Chilean government sought to alter several features of the system, although not all of the changes have been fully implemented: Rather than providing vouchers at a flat rate, voucher amounts are to be tied to family income. Private voucher schools can no longer select students in primary school; for secondary school admission, they can administer tests, but they cannot conduct parent interviews. In addition, Congress recently passed legislation that will hold schools accountable for student achievement and improvement over time.

The political debate continues. In 2006, widespread student protests of inequalities in the education system prompted debate over whether entrepreneurs should be able to own and run private voucher schools for profit. Proposed legislation, which initially prohibited for-profit education organizations, now would require that such entities make available to the public information on their profitability as well as their use of voucher funds.

Data and Methodology

Our study is based on student-level data from Chile’s national standardized test, Sistema de Medición de la Calidad de la Educación (Educational Quality Measurement System—SIMCE), which assesses students in grades 4, 8, and 10 in language, mathematics, history and geography, and natural sciences. In 2002, SIMCE evaluated 274,863 4th graders. Complementing student test scores are parent and teacher questionnaires, which include socioeconomic and environmental information on the students, their families, their peers, and their schools.

Because we lacked complete data for some schools, our study includes 252,202 students. Fifty-eight percent of the students were attending municipal schools, 24 percent were attending stand-alone schools, and 18 percent were attending network schools. In addition to separating out municipal and stand-alone private schools, our analysis subdivides the network schools into five groups—those that are in networks of two, three, four, five, and more than five schools—for a total of seven categories. (We have excluded information from the elite, independent schools that receive no government subsidy.) We compare the test scores of students in each of the seven categories, taking into account differences in the students’ socioeconomic characteristics, including parent schooling, self-reported household income, the number of non-school books in the home, and the quality of the peer groups (calculated by averaging family background and home resources for all students in the classroom). We also included some school-level control variables—whether or not the school was located in a rural area, the total number of students per school, and the average monthly tuition a school charges.

If these variables fully account for differences in student and peer demographics across the various categories of schools, then this strategy will provide unbiased evidence on the relative effectiveness of municipal, stand-alone, and network schools. We cannot account for other factors that could be significant. For example, the average student attending a privately run voucher school, whether network or stand-alone, may have parents who place a higher value on education than those of the average student attending a municipal school. Because we do not have a measure of parent commitment to education, we may confuse the effect of having a committed parent with that of attending a private school. Similarly, the “brand name” value attached to network schools may enable them to select more-qualified students, on average, than their independent counterparts.

As a result, simple comparisons of student outcomes in municipal, stand-alone, and network schools might give misleading estimates of the impact of schools on student achievement, even after adjusting for the measured characteristics of the students who attend each type of school. In order to correct for this selection bias, we restrict our analysis to differences across students in the type of school they attend that result from the types of schools available to them. More specifically, we assume that an individual’s probability of choosing a given school type is affected by the school density (that is, the number of schools per square kilometer) of each type in her municipality. All else being equal, students are more likely to choose schooling alternatives that are more densely concentrated in their municipalities. The crucial assumption made by our method is that school choice is influenced by local school supply, but school densities at the community level do not directly influence student achievement.

Though every precaution has been taken to make the comparison exact, it is still possible that our results overestimate the benefits of privately subsidized schools over municipal ones. For example, it is possible that voucher schools are to be found in greater density in higher income areas, where more parents are willing to pay additional fees for their children to attend higher-quality schools. To the extent that is happening, our results could be biased toward finding greater voucher benefits than is actually the case. For that reason, our comparisons between private and municipal schools should be interpreted cautiously. However, that potential source of bias is unlikely to affect comparisons between stand-alone private schools and network ones, the main focus of this analysis.

Results

We report our results in terms of standard deviations of student test scores. The difference in performance between American 4th and 8th graders on the National Assessment of Educational Progress is about one full standard deviation, suggesting that students improve by one quarter of a standard deviation each year. Although comparable measures of the rate of student learning are not available for Chile, researchers studying the Chilean school system typically consider a difference in student achievement of 10 percent of one standard deviation to be a small to moderate effect. Without accounting for any differences in students’ socioeconomic status, the Spanish language and mathematics test scores of students who attend network schools are considerably higher than the scores of those attending stand-alone schools. After controlling for student and peer attributes and for selection bias, we still find a substantial positive and statistically significant effect of attending a network school on student achievement. Students at network schools score 19 percent and 25 percent of a standard deviation higher than students at stand-alone schools in Spanish language and math, respectively. We also find that students at municipal schools do significantly worse than students at stand-alone schools on achievement tests (19 percent and 16 percent of a standard deviation in Spanish language and math, respectively), although, as discussed above, we are less confident in these results because of the difficulties of accounting for the selection of students into and by private schools.

Although these results provide some evidence of the effectiveness of school networks, a more precise analysis is needed to understand the optimal size of a network. We examined whether larger networks are more effective than smaller ones and found that, both with and without correcting for student and peer socioeconomic characteristics and selection bias, students at schools that are part of networks of three or more schools consistently outperform students at schools in networks of only two schools.

Figure 3 shows the results from our estimations. Students in schools in larger networks generally learned more than students in stand-alone schools. The results for Spanish language achievement show students in schools in networks with three schools learn 24 percent of a standard deviation more, those in networks of five schools learned 50 percent of a standard deviation more, and those in networks of more than five schools learned 23 percent of a standard deviation more. The effects on mathematics achievement are similar. Students who attend schools in networks with three schools learn 37 percent of a standard deviation more than students in stand-alone schools. The percentages for those in networks of five and more than five schools are 36 and 34, respectively.

Prior research in Chile and in the United States has demonstrated that, all else being equal, Catholic schools outperform public schools and other private schools. Since some of the network schools were affiliated with Catholic churches, that fact could be the explanation for the apparent positive benefits that come from networking. To determine whether that was the case, we checked whether the school owners were Catholic. Only 13 percent of the students attended such schools, however. And after adjusting for Catholic affiliation, the differences between network and stand-alone schools remained large and significant. In other words, the superior performance of network schools is not driven by the number of them that are Catholic.

Policy Implications

This paper compares the academic achievement of 4th graders in municipal schools, stand-alone schools, and network schools. Controlling for individual and peer characteristics as well as selection bias, we find that students in network schools outperformed those in stand-alone schools in both Spanish language and math. The stand-alone schools outperformed municipal schools but not by as large a margin. It also is of interest that students generally performed better in networks of large size. Most clearly, those in networks that contained three or more schools generally outperformed those in networks with only two schools.

Possible explanations for the positive school network effect include the substantial benefits of scale for employing education professionals and administrators, the bulk purchases of supplies and equipment, and the costs of implementing innovations in the curriculum. School networks may also benefit from greater access to credit and private investment than that extended to small individual schools in Chile. In addition, it may be that operating within a larger communal organization reduces agency problems; encourages interactions between parents, teachers, administrators, and students; and influences the development of professional school communities.

Of course, it is also possible that good schools are invited to join networks, while weaker schools are left on their own. In a competitive schooling environment, low-quality schools may be unable to attract students and additional resources needed to expand operations.

The results of this paper add evidence to the debate in the United States over the desirability of creating networks of charter and voucher schools. The findings provide some ground for optimism about the effects of networking on student achievement. Policies that provide incentives for schools to establish a network or to be managed by an organization that runs a network of schools may have the potential to increase educational outcomes.

Gregory Elacqua is professor at the Universidad Diego Portales in Santiago, Chile, and former policy advisor to the Minister of Education of Chile. Dante Contreras is senior researcher at the United Nations Development Program (UNDP) Chile and associate professor, Universidad de Chile. Felipe Salazar is researcher at the Universidad Diego Portales.

Why aren’t schools an issue in the 2008 election? Results from the latest international tests arrived just as Education Next was going to press. In math and science, the United States again trailed the average international score achieved by students in the 57 test-taking nations that together comprise 87 percent of the world economy. Embarrassingly, the United States now lags behind Poland, which lifted its scores more than any other nation. Meanwhile, Finland, Taiwan, Hong Kong, South Korea, and our next-door neighbor, Canada, won high marks.

So why hasn’t the condition of the nation’s schools become a top issue in the 2008 election? Why is lagging student performance going unnoticed so far, undiscussed by candidates, questioners, and commentators alike?

In this issue, Eric Hanushek and Ludger Woessmann (“Education and Economic Growth”) demonstrate the critical contribution to economic growth that good schools can make: If the United States were to join the world leaders in math and science this coming year, the country’s Gross Domestic Product, within a couple of decades, could be expected to rise by an extra 5 percentage points—enough to cover the full cost of its education system.

Ah, there’s the rub. Most of the economic payoff does not fall within the four- to eight-year horizon of our duly elected public officials. If a country fails to educate its young, the nation does not suffer until all those candidates are writing their memoirs or become subjects of posthumous biographies.

Candidates must worry about the present—and the present requires that one pay close attention to interest groups, especially to powerful teachers unions that pour vast sums into political campaigns. Between 1989 and 2006, the National Education Association (NEA) came in fourth among all entities contributing to national campaigns, right behind the National Association of Realtors. With the NEA opposed to meaningful accountability, genuine school choice, and anything resembling merit pay, politicians have little to gain from trumpeting reforms that might get schools back on track.

When the special interests get control of policy, the consequences can be disastrous, as the housing credit morass reveals. Realtors loved the free-flowing credit, so politicians had every reason to ignore the risks.

It was not always this way. The United States once led the world in its commitment to education. From the earliest days of the Republic, the United States invested heavily in its human capital, more so than any other nation. Those investments contributed to the extraordinary growth rate that propelled the nation to the world’s pinnacle by World War II.

Around 1970, the educational-industrial complex, long under construction, was finally hammered into place. Legislatures gave teachers collective bargaining rights, the courts began instructing the schools on disciplinary procedures, regulations multiplied, the United States gained a national department of education, and state and federal dollars poured into the system.

Despite the cash flow, education itself was put on pause. Grades inflated, learning faltered, graduation rates stagnated.

Forty years later, the impact on the well-being of the country is becoming increasingly obvious. As the world becomes “flatter,” the importance of human capital escalates, say Hanushek and his co-authors. The nations of South and East Asia are on the march. Corporations move operations offshore in order to find appropriately educated workers at the going price. Universities are finding it easier to recruit top-level scientists and sophisticated social scientists from abroad rather than try to grow them at home.

At least New Orleans’s new superintendent, Paul Vallas, and New York’s mayor, Michael Bloomberg, are beginning to address the issues (see “The Vallas Effect” and “New York City’s Education Battles”). After jump-starting schools in Chicago and Philadelphia, Vallas is giving new hope to the hurricane-battered city on the Mississippi. Meanwhile, Bloomberg’s education boss, Joel Klein, is deconstructing New York City’s educational-industrial complex, while giving merit pay, charter schools, and student accountability a chance. If Vallas, Bloomberg, and Klein have made mistakes, at least they are trying. May their spirit catch on.

According to the latest international math and science assessment conducted by the Organisation of Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA) (see Figure 1), the United States remains a long distance from that target. Rather than worrying about the consequences, some have begun to question what all the fuss was about. Education researcher Gerald Bracey, for example, has argued that no one has “provided any data on the relationship between the economy’s health and the performance of schools. Our long economic boom suggests there isn’t one—or that our schools are better than the critics claim.”

Truth be told, the Bracey critique is not entirely misplaced. Most commentators rely more on the commonsense understanding that countries must have good schools to succeed economically rather than presenting conclusive empirical evidence that connects what students learn in school to what subsequently happens in a nation’s economy. Even economists, the people who think the most systematically about the way in which “human capital” affects a nation’s economic future, have skirted the heart of the question by looking only at “school attainment,” namely the average number of years students remain in school.

Using average years of schooling as an indicator of a country’s human capital has at least two major drawbacks. First and foremost, the approach assumes that students in diverse school systems around the world receive the same educational benefits from a year of schooling. A year of schooling in Papua New Guinea and a year of schooling in Japan are treated as equally productive. Second, this measure does not account for learning that takes place outside the classroom—within families, among peers, or via the Internet, for example.

A more direct measure of a country’s human capital is the performance of students on tests in math and science, something that might be called the average level of “cognitive skills” among those entering a country’s work force. At one time, internationally comparable information on student performance was not available for a sufficient number of countries over a long enough period of time to allow for systematic study, which is why economists relied upon the less informative measures of school attainment. Now that test-score data for many countries over an extended period of time are readily available, it is possible to supplement measures of educational attainment with these more direct measures of cognitive skills.

In a series of studies conducted over several years, the four of us have explored the role of both school attainment and cognitive skills in economic growth. Beginning in the mid-1960s, international agencies started conducting tests of students’ performance in mathematics and science at various grade levels. We used performance on 12 of these standardized tests as rough measures of the average level of cognitive skill in a given country. With this information, we could assess how human capital relates to differences in economic growth for 50 countries from 1960 to 2000, more countries over a longer period of time than any previous study. We were also able to pay close attention to institutional factors that influence economic growth, such as openness of the economy and protection of property rights.

What we discovered gives credence to the concerns expressed in A Nation at Risk. The level of cognitive skills of a nation’s students has a large effect on its subsequent economic growth rate. Increasing the average number of years of schooling attained by the labor force boosts the economy only when increased levels of school attainment also boost cognitive skills. In other words, it is not enough simply to spend more time in school; something has to be learned there.

We also discovered that the size of the impact of cognitive skills depends on whether a nation’s economy is open to outside trade and other external influences. The more open the economy, the more important it is that a country’s students are acquiring high levels of cognitive skills. As the world becomes increasingly interdependent or “flat,” to use New York Times columnist Thomas Friedman’s familiar terminology, enhancing human capital will become increasingly critical. As the world continues to change, the United States can ill afford to rest easily on its past accomplishments.

Measuring Cognitive Skills

Reaching these conclusions required a multistep analysis. The first step was to use the 12 PISA and other international math and science assessments, dating back to 1964, to construct an index of cognitive skill levels for a large sample of countries at various points in time. Because the number of countries participating in the 12 test administrations changed from one administration to the next, and because testing agencies have made no attempt to link their results to one another, we needed to develop comparable scores for each test. This required a norm against which each test could be calibrated. Fortunately, we could construct that norm by using information from tests in the United States, the country that has had the earliest, most sophisticated, and most comprehensive system of testing. The United States has participated in all of the international tests since 1964, and it has also maintained a separate longitudinal testing system of its own, the National Assessment of Educational Progress (NAEP). With that information in hand, it was possible to calibrate scores on each of the separate international tests to one another via the connection of those tests to the NAEP. To obtain further precision, we used the variation in scores across a subset of the more-advanced developed countries to obtain an estimate of the spread in scores across countries. By following these two steps, we were able to aggregate all available scores for each country into measures of average cognitive skill levels for each country.

The 50 countries for which we were able to develop a comparable measure of cognitive skill levels include the 30 democracies that have market economies and have been accepted as members of the OECD, most of which are at a relatively high level of economic development. The other 20 countries are at lower levels of economic development. In Figure 2, you can identify top performers like Finland and Japan, average performers such as the United States and Germany, and low performers that include Albania, the Philippines, and South Africa.

Impact on Economic Growth

We wanted to use this new information to compare the economic benefits of higher levels of just school attainment with the benefits of higher levels of cognitive skills. We therefore took measures of average educational attainment and average cognitive skill levels for as many countries as possible and examined their relationship to the average annual growth rate in the country’s gross domestic product (GDP) per capita from 1960 through 2000.

First, we looked just at the impact of average school attainment on the economic growth rate. (An adjustment was made for the initial level of GDP because it is “easier” to grow if you are starting out at a lower level; that is, it is easier to copy more productive technologies than to initiate progress on your own.) When we performed this analysis, we found, as other economists before us, that when the average number of years of schooling in a country was higher, the economy grew at a higher annual rate over subsequent decades. Specifically, we found that, across the 50 countries, each additional year of average schooling in a country increased the average 40-year growth rate in GDP by about 0.37 percentage points.

That may not seem like much, but consider the fact that since World War II, the world economic growth rate has been around 2 to 3 percent of GDP annually. Lifting it by 0.37 percentage points is a boost to annual growth rates of more than 10 percent of what would otherwise have occurred, a significant amount.

But the impact of improved cognitive skills, as measured by the performance of students on math and science tests, is considerably larger. When we performed the analysis again, this time also including the average test-score performance of a country in our model, we found that countries with higher test scores experienced far higher growth rates. If one country’s test-score performance was 0.5 standard deviations higher than another country during the 1960s—a little less than the current difference in the scores between such top-performing countries as Finland and Hong Kong and the United States—the first country’s growth rate was, on average, one full percentage point higher annually over the following 40-year period than the second country’s growth rate. Further, once the impact of higher levels of cognitive skills are taken into account, the significance for economic growth of school attainment, i.e., additional years of schooling, dwindles to nothing (see Figure 3). A country benefits from asking its students to remain in school for a longer period of time only if the students are learning something as a consequence.

Another indication of the importance of education quality to economic growth lies in our ability to explain global variation in GDP growth. When we tried to account for economic growth with information only about school attainment levels and the level of a country’s GDP in 1960, we were able to explain only one-quarter of the differences we saw among countries. But when we also included cognitive skills in our statistical models of economic growth, we were able to attribute nearly three-quarters of the differences among countries to these three factors. In other words, higher levels of cognitive skill appear to play a major role in explaining international differences in economic growth.

Of course, the initial level of economic development, schooling attainment, and cognitive skills are not the only factors that affect economic growth. Could it be that some other factor we have overlooked is responsible for the close connection between test scores and economic growth?

Other economic research has identified two additional factors that affect a country’s economic growth rate: the security of its property rights and its openness to international trade. When those two factors are taken into account, the positive effect of cognitive skills on annual economic growth becomes somewhat smaller, but is still 0.63 percentage points per half of a standard deviation of test scores. This is the best available estimate of the size of the impact of cognitive skills on economic growth.

Other commonly discussed determinants of economic growth are fertility and geography. However, when we took into account the total fertility rate and common geographical proxies, such as latitude or the fraction of the land area of a country that is located in the tropics, neither of these additional variables was significantly associated with economic growth. Once again, the strong effect of cognitive skills remained clear.

We performed a variety of additional tests to assess the validity of these basic results. For example, we estimated the relationships over shorter periods of time, used different subsets of international tests, and compared smaller groups of the 50 countries.

One of our tests was particularly interesting. We thought it possible that the effect of cognitive skills could be the result of the presence in our sample of East Asian countries, most of which have both high levels of cognitive skill and rapidly growing economies. To see whether the inclusion of those countries in our study influenced our results, we excluded them from one of our models. The impact of cognitive skill remained very powerful, albeit diminished.

We also looked at cognitive skills as measured in the 1960s through the mid-1980s to see what their impact was on growth between 1980 and 2000, ensuring that the cognitive skills themselves were not caused by the economic growth. Again, our basic findings remained intact. Finally, we looked at whether a country’s estimated cognitive skills affected the earnings of immigrants working in the United States. Higher home country cognitive skills translated into higher earnings if the immigrants were educated in their homeland but not if educated in the United States.

Our commonsense understanding of the importance of good schools can thus be documented quite precisely. A highly skilled work force can raise economic growth by about two-thirds of a percentage point every year.

More Rocket Scientists or Basic Skills for All?

To gain additional insight into the relationship between cognitive skills and economic growth, we examined the separate impact of improvements at different levels of a nation’s distribution of skills. Loosely speaking, is it a few “rocket scientists” at the very top of the distribution who spur economic growth, or is it “education for all” that is needed?

To address this question, we measured the share of students in each country who reach a threshold of basic competency in mathematics and science, as well as the share of students who perform at very high levels. To estimate the importance of basic competency, we identified the share of students performing at least at a very basic level, that is, no more than one standard deviation below the international average of all OECD countries. In the average OECD country in our study, 89 percent of the students achieved at least at this very basic level. The share of students with at least basic skills ranged widely among countries, from as low as 18 percent in Peru to 97 percent in the Netherlands and Japan. To show a country’s ability to develop a large cadre of high-performing students, we identified the share of students performing at very high levels—at or above one standard deviation over the OECD average. On average across all countries, 6 percent of students performed at that high level. Once again, countries varied enormously in this respect, the percentage ranging from as low as 0.1 percent in Colombia and Morocco to 18 percent in Singapore and Korea and 22 percent in Taiwan.

Which is more important for growth—having a substantial cadre of high performers or bringing everyone up to a basic level of performance? The answer, it seems, is not one or the other but both! When we estimated the importance of each within the same model, we found each of them to be separately important to economic growth. That is, both the performance of countries in ensuring that almost all students achieve at basic levels and their performance in producing high-achieving students seem to matter.

The reasons that a substantial cadre of highly skilled citizens and near-universal basic skills matter are not difficult to imagine. Even if a country is simply making use of new technologies developed elsewhere, as is often the case in developing parts of the world, the more workers that have at least basic skills, the easier it will be for them to make use of those new technologies. Some workers need a high level of skill so they can help adapt the new technologies to their countries’ particular situation. In countries on the technological frontier, substantial numbers of scientists, engineers, and other innovators are obviously needed. But so is a labor force that has the basic skills needed to survive in a technologically driven economy.

But even if the results seem intuitively correct, they should be taken as suggestive rather than definitive, because the two measures of cognitive skills are closely related to one another and our models have difficulty in separating out the precise impact of each individually. Most countries that have a high percentage of students with very high cognitive skills also are ones in which basic skills are near universal. Conversely, countries with a substantial percentage of students lacking even basic skills tend to be those that have only a small percentage of highly skilled students. Still, that pattern is not a perfect one, so we are able, at least tentatively, to identify the impact of each type of human capital, and we are quite confident that we can recommend that countries both concentrate resources on their “best and brightest” and make sure that “no child is left behind.”

The Impact of Becoming a World Leader

What would it mean for economic growth, then, if a country like the United States, currently performing somewhat below the average of OECD countries, managed to increase its performance by 50 points (or 0.5 standard deviations) so that it would score alongside the world leaders? (On average on the PISA 2006 math and science exams, countries such as Canada and Korea scored about 50 points higher than the U.S., Hong Kong and Taiwan about 60 points higher, and Finland as many as 74 points higher.) That increase of 50 test points is exactly what George H. W. Bush and the nation’s governors together promised in 1989 the United States would achieve by the year 2000.

Unfortunately, no such gains were realized. But had the promise been fulfilled by 2000, our results suggest that GDP would by 2015 be 4.5 percent greater than in the absence of any such gains (see Figure 4). That 4.5 percent increment in GDP is equal to the total the U.S. currently spends on K–12 education. In other words, had that money effectively raised cognitive skills by the 50 test points that would have brought the United States close to world leadership, the economic returns to the country would probably have been enough to cover the entire cost of education in 2015 and after.

Figure 4 shows that the benefits of successful reform grow even more vivid when we look farther out. Over 75 years, even a reform that takes effect in 20 years (instead of the governors’ 10 years) yields a real GDP that is 36 percent higher than it would be if there was no change in the level of cognitive skills.

None of this is meant to suggest that schooling is the only factor contributing to a society’s cognitive skill development. Family, individual ability, and health combine with school quality to determine a student’s level of achievement. Yet there is every reason to believe that the single best route to higher levels of cognitive skill is strengthening a country’s education system. After all, most people think that is the system’s primary purpose.

An American Exception?

The United States has never done well on international assessments of student achievement. Instead, its level of cognitive skills is only about average among the developed countries. Yet the country’s GDP growth rate has been higher than average over the past century. If cognitive skills are so important to economic growth, how can we explain the puzzling case of the U.S.?

Part of the answer is that the United States may be resting on its historic record as the world’s leader in educational attainment. In addition, the United States has other advantages, some of which are entirely separate and apart from the quality of its schooling. The U.S. maintains generally freer labor and product markets than most countries in the world. There is less government regulation of firms, and trade unions are less powerful than in many other countries. Put more broadly, the U.S. has generally less intrusion of government in the operation of the economy, including lower tax rates and minimal government production through nationalized industries. Taken together, these characteristics of the U.S. economy encourage investment, permit the rapid development of new products and activities by firms, and allow U.S. workers to adjust to new opportunities.

Those economic institutions seem to matter on their own and in conjunction with cognitive skills. Our analyses suggest that the value of a high-quality education system is substantially diminished in closed economies. We estimate that the effect of a one-standard-deviation improvement in cognitive skills on annual economic growth is 0.9 percentage points per year in closed economies, identified by heavy restrictions on international trade, but 2.5 percentage points in open economies. It may be that rich human capital combines with a laissez-faire economy to foster robust economic growth.

It is also the case that, over the 20th century, the expansion of the U.S. education system outpaced the rest of the world. The U.S. pushed to open secondary schools to all citizens. Higher education expanded with the development of land grant universities, the GI Bill, and direct grants and loans to students. The extraordinary U.S. higher-education system is a powerful engine of technological progress and economic growth in the U.S. not accounted for in our analysis. By most evaluations, U.S. colleges and universities rank at the very top in the world.

Although the strengths of the U.S. economy and its higher-education system offer some hope for the future, the situation at the K–12 level should spark concerns about the long-term outlook for the U.S. economy, which could eventually have an impact on the higher-education system as well. The U.S. higher-education system may also be challenged by improvements in higher education across the world. Other countries are doing more to secure property rights and open their economies, which will enable them to make better use of their human capital. Most obviously, the historic advantage of the U.S. in school attainment has come to an end, as half of the OECD countries now exceed the U.S. in the average number of years of education their citizens receive. Those trends could easily accelerate in the coming decades.

Not Just a Matter of Money

Our evidence of a clear, strong relationship between cognitive skills and economic growth should encourage continued reform efforts. Improvements in mathematics performance called for by No Child Left Behind would matter, contrary to what critics sometimes suggest. Yet reformers should bear in mind that money alone will not yield the necessary improvements. Many expensive attempts around the world to improve schooling have failed to yield actual improvements in student achievement.

Economic growth flows only from reforms that bring actual improvements in cognitive skills. Identifying what works and how to implement it on a society-wide scale remains a challenge, not only for the U.S. but also for many nations across the globe. But, if we are to remain economically competitive, we need to solve the puzzle of our schools and meet the governors’ challenge. We should not, simply because we have failed to meet them in the past, decide that the goals were not legitimate or important.

Eric A. Hanushek is a senior fellow at the Hoover Institution of Stanford University. Dean T. Jamison is professor of health economics in the School of Medicine at the University of California, San Francisco. Eliot A. Jamison is an investment professional at Babcock & Brown. Ludger Woessmann is professor of economics at the University of Munich and heads the Department of Human Capital and Innovation of the Ifo Institute for Economic Research. The opinions expressed in this article are those of the authors and do not necessarily reflect those of their employers.

Within a country’s educational system, the relevant institutions and policies include the ways in which a society finances and manages its schools, how a society assesses student performance, and who is empowered to make basic educational decisions, such as which curricula to follow, which teachers to hire, and what textbooks to purchase. If resources are to be used effectively, policies must create incentives that encourage school personnel to behave in ways that do not necessarily further their own interests. For instance, without the right incentives, teachers may avoid using the most promising teaching strategies, preferring to use the techniques they find most convenient. In terms of policy, one might speculate that if a nation assesses the performance of students with some sort of national exam and uses this information to monitor teachers, teachers will put aside their other interests and focus mainly on raising student achievement.

International Evidence

This study asks two basic questions: Do policy and institutional variation help to explain variation in student performance? If so, which policies and institutions are most conducive to student performance? To answer these questions, I turn to the international evidence on student achievement. This is because the institutions within a country do not vary enough to test how different institutions affect student achievement. Only the international evidence, which encompasses many education systems with a wide variety of institutional structures, has the potential to show which institutions heavily affect student performance. My working hypothesis is that differences in educational institutions explain more of the international variation in student performance than differences in the resources nations devote to schooling.

A large body of empirical evidence on the effects of resources on student achievement already exists. It overwhelmingly shows that, at given spending levels, an increase in resources does not generally raise educational performance. Studies summarized by Eric Hanushek of the Hoover Institution have shown the lack of a strong, systematic relationship between resources and performance within the United States, within developing countries, and among countries. Likewise, studies by Erich Gundlach and myself at the Kiel Institute of World Economics have found no systematic relationship between resources and performance across time within most countries in the Organization for Economic Cooperation and Development (OECD) and within some countries in East Asia.

Data from the Third International Mathematics and Science Study (TIMSS) again show that differences from country to country in per-pupil spending do not help in understanding differences in educational performance. The simple correlation between spending per student and average TIMSS test scores is 0.13 in primary school and 0.16 in middle school, on a scale where 1.0 denotes an absolute positive correlation between the two variables and 0 signals no correlation (see figure 2). This means that school productivity, the ratio of educational performance to the level of spending, differs widely across schooling systems.

There is no consensus on the lack of a strong positive relationship between educational resources and performance, however. In the within-country literature, some scholars have questioned the use of meta-analyses, while others have suggested the use of alternative measures of school performance, such as students’ future labor-market performance. Still others point to controlled and quasi-controlled empirical experiments that have shown that more resources can lead to higher achievement. Notwithstanding this debate, the international variation in student performance levels in mathematics and science is a fact, and it is generally accepted that differences in the amount of resources given to the education sector do not fully explain why performance levels vary.

Data

This study uses data from 39 countries to analyze how various institutions affect educational performance at the student level. I constructed a student-level database that combines data from TIMSS with data on education systems from the OECD. TIMSS is the latest, largest, and most extensive international student achievement test ever conducted. In 1994-95, representative samples of students in more than 40 countries were tested (for various reasons, data files were available for only 39 countries for this study). Countries participating in the study were required to administer tests to students in the middle-school years, but could choose whether or not to participate in the primary and final school years. This paper focuses on the middle-school years, where students enrolled in the two adjacent grades containing the largest proportion of 13-year-old students (7th- and 8th-graders in most countries) were tested. This data set includes data on more than 250,000 individual students, who form a representative sample of a population of more than 30 million students in the 39 countries.

TIMSS contains student-level data on achievement and family background and various institutional data: class-level data on teachers, and school- and country-level data on the distribution of decision-making powers within the education system. TIMSS does not include data on spending, so current national public spending per student in secondary education in international dollars was calculated on the basis of UNESCO and World Bank data. Further country-level data on institutional features of the education system–mainly concerning the distribution of decision-making powers and the size of the private-schooling market–come from the OECD educational indicators.

I performed the analysis at the level of the individual student (not the class, school, district, or country) because this directly links student performance to the teaching environment. This allowed me to control for the influence of each individual student’s background, assess the influence of the actual resource level and teacher characteristics each student faces, and look at the institutional features that are relevant to individual students. Previous international studies have used country-level data to analyze what influences student performance. These macro approaches cannot control for individual influences on a student’s performance. Country-level analyses are also limited because they can analyze only institutions that work at the country level, such as centralized exams, and not institutional features that work at lower levels, such as teachers’ influence over the curriculum.

The trouble with performing the analysis at the individual level is that there are no independent, individual observations for many variables. Within the TIMSS data set, the primary sampling unit (or PSU) is the school, not the student or classroom. Individual students who attend the same school may share some characteristics that are not captured by survey data; the individual observations are not wholly independent of one another. Also, several of the resource and institutional variables, such as the school’s decision-making responsibility and the existence of national examinations, are measured at the school or country level, further decreasing the independence of individual observations and reducing the number of independent observations on these variables. For instance, in comparing students in countries with centralized exams with those with no centralized exams, there were only 39 independent observations (the number of countries in the TIMSS sample). Unless the econometric method is adjusted to account for the lack of variation in some of the independent variables, the findings will appear more robust than they are. I use a statistical method known as robust linear regression with countries as strata and schools (or countries where appropriate) as the primary sampling unit to calculate appropriate standard errors for my findings and to adjust for this potential bias.

Results

This study deals with five main institutional features of a nation’s educational system: 1) centralized exams; 2) the distribution of decision-making power between schools and their governing bodies; 3) the level of influence that teachers and teacher unions have on school policy; 4) the distribution of decision-making power among levels of government, from local to national; and 5) the extent of competition from the private-school sector (see Figure 3).

Before we can test hypotheses, we must control for the effects of family background and the level of resources devoted to education. In this study, the educational level achieved by the students’ parents was strongly positively related to students’ educational performance. Students of parents who completed secondary school (or higher) achieved considerably more than students of parents who finished only primary school. The effect of a family’s having more books at home, shorthand for the educational and social background of the family, was even stronger than that of the highest educational level achieved by the parents. The perform-ance of students increases steadily as you go from students having fewer than 10 books at home to those having more than 200 books. Students scored 54 points better in math and 57 in science (on a range with an international average of 500 and an international standard deviation of 100) when they had more than 200 books at home compared with students who had fewer than 10. Just how big are these effects? Quite large. Consider that the average test-score difference between 7th- and 8th- graders is 40 points in math and 47 in science.

The results for school spending are consistent with the literature: no strong positive relationship exists between spending and student performance. When other factors are taken into account, higher spending and smaller class sizes seem to correspond to inferior mathematics and science results, though the overall effect is relatively small. Nevertheless, providing schools with the proper instructional materials and supplies seems to have a positive effect on performance. Students in schools whose principals reported that they do not suffer from inadequate instructional materials scored 7 points higher in math and science relative to students in schools whose principals reported that they were somewhat limited by inadequate materials. Students in schools with a great shortage of materials scored 6 points worse in math and 12 in science. Both of these findings should be interpreted with care, however: inadequate supplies may have led to poor achievement, or principals of low-achieving schools may tend to blame their poor achievement on inadequate supplies.

The quality of a nation’s teaching force also affects student performance and therefore must be controlled for. If teachers’ age is held constant, then more years of experience are positively related to student performance. But if teachers’ experience is held constant, teachers’ age is negatively related to student performance. This may reflect the positive effects of having more-experienced teachers combined with the negative effects of large age differences between teachers and students. Aging teachers may not understand a younger generation as well as younger teachers, and their motivation levels may be in decline as well.

Teachers who finished secondary school plus some teacher training added 16 points to students’ math scores and 24 to science (compared with teachers who did not complete secondary education). Having a bachelor’s degree added 11 points to students’ math scores, 12 points to science. Possessing a master’s or doctorate added 26 points in math, 32 in science. Overall, the effects of teachers’ educational levels were larger in science than in math.

Altogether, the relationship between school resources and student perform-ance is ambiguous. Per-pupil spending and smaller class size do not have positive effects, while having decent instructional materials and experienced, well-educated teachers do show positive effects.

Centralized exams. Of the 39 countries in this study, 15 have some kind of centralized exams, in the sense that an administrative body beyond the schooling level writes and administers the exams to all students. This can profoundly alter the incentive structure within the educational system by measuring student performance against an external standard, making performance comparable across classes and schools. It makes it easier to tell whether a given student’s poor performance is an exception within a class or whether the whole class is doing poorly relative to the country as a whole. In other words, centralized exams make it obvious whether it is the student or the teacher who is to blame. This reduces the teachers’ leeway and creates incentives to use resources more effectively. It makes the whole system transparent: parents can assess the performance of children, teachers, and schools; heads of schools can assess the performance of teachers; and the government and administration can assess the performance of different schools.

Centralized exams also alter the incentive structure for students by making their performance more transparent to employers and advanced educational institutions. Their rewards for learning thus should grow and become more visible. Without external assessments, students in a class looking to maximize their joint welfare will encourage one another not to study very hard. Centralized exams render this strategy futile. All in all, given this analysis, we should expect centralized exams to boost student performance.

And they seem to. All things being equal, students in countries with centralized exams scored 16 points higher in math and 11 points higher in science, although the science finding is not statistically significant due to the small number of countries in the sample (see Figure 3 for results). Furthermore, students in schools where external exams or standardized tests heavily influence the curriculum scored 4 points higher in math, though there appears to be no effect in science. This suggests that science tests may lend themselves less readily to standardization.

Decision-making between schools and their governing bodies. Some school systems are characterized by a high degree of centralization, where decisions on a wide range of issues are taken out of the schools’ hands. Other school systems are highly decentralized; most decisions are made at the local level. For instance, schools have a high degree of autonomy in the Netherlands, where 73 percent of decisions are made at the local level, according to the OECD. By contrast, Greece, Norway, and Portugal allow local school personnel to make fewer than 25 percent of the decisions. Here the question is, What is the division of decision-making powers between schools and the government in a country, and how do these divisions affect student achievement?

The effects of granting more autonomy to schools are hard to predict. On the one hand, schools need a high degree of autonomy in order to respond to the demands of parents–a prerequisite for competition. Also, the educators within a school should have more knowledge of effective teaching strategies for their students than central administrators. Likewise, individual teachers should know what are the best textbooks and supplies for their students. Heads of schools should also have more knowledge than central administrators about which teachers to hire and who deserves promotion or a raise in salary.

On the other hand, enhanced autonomy makes it easier for school personnel to reduce their workload, unless they are subject to external monitoring and evaluation. The more flexibility a school has, the more important it is to have external standards and assessments. Putting decisions on the size of the school budget in the hands of school personnel might also harm performance; it is clearly in their interest to garner additional funds for themselves or resources that lighten their workload.

We should expect, then, that giving schools the power to set their own budgets, performance goals, and standards of what to teach will have an adverse impact on student achievement. Such powers are probably best left to central authorities. By contrast, decisions on how to meet the goals and standards, such as the choice of teaching techniques and the purchase of supplies, are best left to schools, as long as an effective monitoring and assessment program is in place.

The first variable I analyze is whether having a centrally designed curriculum and a centralized list of approved textbooks is conducive to student performance. These are essentially decisions about what schools are expected to cover. Students in countries with centralized curricula scored 11 points better in math, 6 in science. Students in countries with centralized textbook approval scored 10 points better in math, 6 in science. These findings are not statistically significant due to the small number of independent observations, but they are nonetheless suggestive.

Students in schools that had primary responsibility for setting the school budget scored 6 points worse in math and 3 in science (the science effect, however, is statistically insignificant). By contrast, giving schools autonomy in purchasing their supplies goes hand in hand with superior achievement. This is also true for decisions on hiring teachers. Students in schools that hire their own teachers scored 13 points higher in math, 5 in science. Students in schools that determine their own teacher salaries scored 11 points higher in math, 15 in science. Centralized decision-making on curriculum issues seems to prevent schools from seeking to reduce their workload and thus raises student achievement. Conversely, local control of teacher recruitment and compensation may allow schools to retain a more effective staff.

The influence of teachers. Within schools, the incentives that teachers face and their ability to influence the process also affect student achievement. Besides a student’s family, teachers probably have the greatest impact on student achievement. Since they cannot be easily monitored, they also have a great deal of freedom to pursue their teaching in whatever way they wish. Often they face conflicting interests. They clearly have a genuine interest in increasing their income at a given workload or decreasing their workload at a given income. Nonetheless, seeing their students learning also gives teachers pleasure, which encourages them to work harder no matter what their income. Furthermore, teachers who perform poorly may face negative consequences from their heads of school or from parents. The institutional setting will influence them to behave more in one way than another.

It is important to emphasize the difference between teachers acting individually and as part of a union, for these settings may have very different consequences for student achievement. When teachers act collectively, they are a potentially powerful political interest group; their sheer numbers give them voting power that politicians cannot ignore. The aim of teacher unions is to promote the interests of teachers and to defend them against the interests of other groups. The unions, therefore, will focus on the interests that are not advanced by other interest groups–mainly, increasing teachers’ pay and decreasing their workload. They can also exert collective bargaining power. In doing so, they will advance the interest of the median teacher, favoring a leveling of salary scales instead of differentiation by merit. Other things equal, strong teacher unions should promote behavior that is detrimental to student performance. Harvard University economist Caroline Hoxby’s research has shown that teacher unionization helps explain why schools can perform worse when given more resources. Hoxby found that teacher unions act to increase school resources but reduce the productivity with which these resources are used. The cumulative effect is a reduction in school productivity (the ratio of student performance to spending).

By contrast, when teachers act individually, their deep, personal knowledge of their students and the students’ needs may increase their effectiveness and thus outweigh their interest in decreasing their workload. The effect of teacher influence may also differ among decision-making areas. A high degree of teacher leeway in making decisions about which textbooks to buy should be conducive to student learning, since the teachers know best how to teach their students. But a high degree of influence in determining salary levels or the amount of subject matter to be covered should be detrimental to student performance. Altogether, the predicted effect of increasing the power of individual teachers is uncertain.

Here the results are similar to those obtained in the analysis of decision-making divisions between schools and administration. Students in schools whose principals reported that teachers had primary responsibility for determining the school budget scored 13 points worse in math, 5 in science. Likewise, students of teachers who reported that they had a lot of influence on the subject matter to be taught performed worse in science, while the effect in math was statistically insignificant.

By contrast, students scored 14 points better in math and 7 points better in science if teachers had primary responsibility for buying supplies. Teachers’ influence on the curriculum needs to be divided according to the way they exercise it. Students in schools where each teacher individually had a lot of influence on the curriculum performed 12 points better in math, 11 points in science. Teachers’ being able to choose their specific textbooks also has a positive effect in math. But in schools where teachers acting collectively as a union had a lot of influence over the curriculum, students performed 32 points worse in math, 18 points in science.

Overall, these findings on schools’ and teachers’ influence give a clear picture. If schools and teachers can use their intimate knowledge of their students to choose the best teaching method, then they can teach more effectively. But if they can use their influence, whether acting collectively or individually, to reduce their workload, then students’ learning opportunities will suffer.

Decision-making among levels of government. Different levels of government–local, intermediate, and national–have varying degrees of control over school systems worldwide. In the United States most educational decision-making and basically all fund allocation take place at the local level. In Germany the responsibility for planning and purchasing educational resources lies mainly with the intermediate level of government, namely state authorities. In Greece almost all decisions and basically all funding take place at the national level.

Here, again, the predicted incentives are mixed. Local levels of government are more accountable to parents and possess more knowledge about the needs of their particular communities. At the same time, local officials will have closer ties to school personnel, making school-based interest groups more influential. Local officials and school personnel might collude in determining the level and use of funds. Giving the national level more power can make collusion harder to achieve, but national officers do not have enough information to make wise decisions on allocating resources among various needs. A self-interested central administration will also find it easier to develop excessive bureaucracy and to divert resources to the central level. All in all, the intermediate level may be better positioned to govern the schools. It may be far enough away to make lobbying difficult, yet close enough to effectively monitor the schools.

In this study, students in countries where schools have more decision-making powers in managing personnel, planning, choosing their instructional methods, and deciding how to use resources scored significantly higher in science and higher in math (though the effect in math is statistically insignificant). If the percentage of decisions made at the school level increased by 10 percentage points, students scored 8 points higher in science. Students in countries where a larger share of decisions was made at the national level scored lower in both math and science (the effect in math, however, was statistically insignificant).

The level at which schools are funded also affects student performance. More responsibility for purchasing educational resources at the national and local levels appears to correspond to lower student achievement, at least in mathematics. Students performed considerably better when responsibility for purchasing educational resources resided at an intermediate level of government. This suggests that an authority that is close enough to local schools to understand their needs, yet far enough away to avoid collusion between local officials and school employees, is the best place to rest responsibility for funding education.

Competition from private schools. The level of competition that public schools face from private schools is another important institutional feature. The existence of more private schools gives parents who want to raise their children’s achievement the opportunity to choose whether to send them to a particular private school or to a public school. Because the loss of students to private schools may have negative repercussions for the heads of public schools, increased competition from private schools should have a positive effect on the efficiency of resource use in the public schools. The existence of private schools should also increase a country’s overall achievement level. The heads of private schools have clear monetary incentives to use resources in ways that maximize student performance–thereby giving more parents reasons to choose their schools. Therefore, the more privately managed educational institutions there are in a nation, the higher student performance should be.

The degree of competition from private schools varies greatly worldwide. The Netherlands has by far the highest share of students attending privately managed schools (76 percent), followed by the United Kingdom (36 percent) and South Korea (35 percent). However, fewer than 1 percent of Dutch schools are financially independent in the sense that they receive less than half of their core funding from government agencies. Countries with the largest shares of students attending financially independent private schools are Japan (24 percent), South Korea (18 percent), and the United States (16 percent). Australia, Austria, the Czech Republic, Denmark, France, Germany, Hungary, Iceland, Norway, Spain, and Sweden have virtually no independent private schools. Students in countries with larger shares of their enrollment in privately managed schools scored significantly higher in both math and science. If the share of enrollment in privately managed schools was 10 percentage points higher, students scored 6 points better in math, 5 in science. The effect was even larger when only those private institutions that were financially independent were considered.

The Netherlands and Belgium are the countries with by far the largest share of public funds going to private institutions (75 percent and 63 percent, respectively). Meanwhile, less than half a percent of public funding goes to private schools in Austria, Greece, Ireland, New Zealand, the Russian Federation, and the United States. Countries with a higher share of public-education spending going to private institutions performed better in math and science (though the effect in science is statistically insignificant). The effect was even stronger when only those expenditures were counted that went to independent private institutions receiving less than half of their core funding from government. If the share of public funds going to independent private schools rose by 1 percentage point, there was a 12 point increase in math achievement. This suggests that student performance is higher in educational systems where private schools take over resource allocation from public decision-makers.

Institutions Do Matter

Taken together, the effects of all these institutional variables add up to more than 210 points in math and 150 in science. In other words, a student who faced institutions that were all conducive to student performance would have scored more than 200 points higher in math than a student who faced institutions that were all detrimental to student performance. In short, institutional variation across countries explains far more of the variation in student test scores than do differences in the resources devoted to education.

More specifically, having centralized exams and a large private-schooling sector seems conducive to student performance. Generally, school autonomy seems to have a positive impact–but only when schools are given extensive decision-making powers over the purchase of supplies, the hiring and rewarding of teachers, and the choosing of instructional methods. Giving schools power over designing the curriculum syllabus, approving textbook lists, and determining the school budget seems to be detrimental to student performance. The effect of teachers’ influence seems to depend on how it is exercised. Students seem to benefit from their teachers’ having influence over the curriculum, but only when they act as individuals and not as part of a union. It appears to be better for students if teachers significantly influence the choice of supplies, but worse if they have a strong say in the amount of material to be covered.

The only difference between the results for math and science is that the effects of standardization seem to be more positive in math than in science. This shows up in the fact that centralized exams, curricula, and textbook approval have stronger effects in math than in science. One can speculate that mathematics is easier to standardize, whereas science may require more creativity and initiative on the part of teachers.

For education policy, the results of this study suggest that the crucial question is not one of providing more resources but of improving the institutional environment in which schools function. Spending more money within an institutional system that sets poor incentives will not improve student performance. An institutional system in which all the people involved have an incentive to improve student performance is the only alternative that promises positive effects.

-Ludger Woessmann is a research associate at the Kiel Institute of World Economics in Kiel, Germany. To view his study in its entirety, log on to www.edmattersmore.org.

Reducing class sizes is one of today’s most popular education reform strategies. The Education Commission of the States estimates that such efforts cost states $2.3 billion during the 1999-00 school year alone. The federal government contributed another $1.6 billion in 2000-01 toward meeting the Clinton administration’s goal of decreasing class size nationwide in the early grades to no more than 18 students. During the past year or so, the deteriorating condition of state budgets and the Bush administration’s new emphasis on accountability have made class-size reduction less of a priority. Yet it remains popular among parents, teachers, and the teacher unions, which often promote it as an alternative to vouchers.

The motivation for reducing class size is intuitive: with smaller classes, teachers should be able to devote more time to each student, both in the classroom and in giving feedback on homework and tests. The concern is at least threefold. First, reducing class size is remarkably expensive, since it requires hiring more personnel. There may be less costly reforms that are at least as effective as class-size reduction. Second, hiring more teachers may dilute the quality of the workforce, thereby negating any gains among the students of good teachers. Finally, the intuitive relationship between class size and teachers’ effectiveness may not actually hold true—teachers may be no more successful with 18 students than with 23.

The most persuasive evidence of the benefits of class-size reduction has come from the Project STAR (Student/Teacher Achievement Ratio) experiment in Tennessee, where students were randomly assigned to classrooms of varying size. Smaller classes appeared to yield substantial gains among kindergartners and possibly 1st graders in the first year of the program—gains that were maintained throughout their school years. However, a large body of research literature on class-size reduction contradicts the findings from Project STAR.

To lend a fresh perspective on this issue, we use data from the Third International Mathematics and Science Study (TIMSS) to compare the effects of class size around the world. While Americans squabble over whether class size should be 18 or 25 students, teachers in Korean schools routinely face classrooms of more than 50 students. These and other differences, such as the quality of a nation’s teachers, can be valuable tools in discerning where, if ever, class-size reductions are likely to be beneficial.

Photograph by Getty Images.

Two Strategies

Ascertaining the effect of class size is less straightforward than it might appear. The central problem is that students are not assigned to classrooms randomly. For instance, schools often establish small remedial classes for lagging students or small enrichment classes for the so-called gifted and talented. In addition, school systems may direct students into schools with different average class sizes on the basis of their performance.

Parents also may influence their children’s class sizes. They may work hard to move their children to schools with smaller classes, where they are likely to receive more attention. Thus variation in class size may be simply the result, rather than the cause, of differences in student achievement. Estimating the true effect of class size on student performance requires a strategy that looks only at variations in class size that are unrelated to students’ previous achievement.

In principle, two such strategies are available. The first is to conduct a randomized field trial along the lines of Project STAR in Tennessee. Unfortunately, while it used a powerful research design, the Tennessee study was flawed in its implementation. For one thing, no data were collected on students’ performance before they were assigned to their classrooms, making it impossible to know whether the assignment was truly random. In addition, the teachers were aware of their participation in Project STAR, as in almost any true experiment. This has led some to question whether its findings can be expected to hold under more typical conditions. It is also worth noting that the evidence here comes from an experiment conducted in a single U.S. state during the mid-1980s, in which classes were reduced from 22-25 students to fewer than 17. In that sense, the findings may not apply to school systems in other parts of the world.

The second strategy, quasi-experimental research, relies either on special types of variation in class size or on econometric techniques to make appropriate comparisons. However, the conditions that must be met in order to use this approach make credible quasi-experimental studies possible for only a small number of school systems. For example, Princeton economists Anne Case and Angus Deaton used data on black students in South Africa during apartheid to measure the effects of class size. They argued that the black population of South Africa during this time lacked the power to influence class sizes, making the assumption that students were randomly assigned to classrooms of different size more plausible. But the South African school system under apartheid was obviously unique; in some districts, the average class size reached 80 students.

While Case and Deaton found that smaller classes were modestly beneficial, Harvard economist Caroline Hoxby’s careful quasi-experimental study of elementary schools in Connecticut suggests that Case and Deaton’s results may not be relevant for more developed countries. Hoxby analyzed variation in class size due to random fluctuations in the number of births and restrictions on maximum class sizes. She found no evidence of even trivial class-size effects. However, her approach requires a long panel of rich data and has yet to be applied in other contexts.

East Asian countries feature large classes, with an average of more than 30 students. Photograph by Getty Images.

International Evidence

Taking data from TIMSS, we used a quasi-experimental design to take a broader look at how class size affects student achievement in different nations around the world. Conducted in 1994-95, TIMSS was the largest international study of student performance ever, with more than 40 countries participating initially. Each country administered the test to a nationally representative sample of middle-school students, defined as those students enrolled in the two adjacent grades that contained the largest proportion of 13-year-old students at the time of testing (grades 7 and 8 in most countries).

Our strategy takes advantage of the fact that data were collected on both actual and average class sizes and on students’ performance and socioeconomic backgrounds for more than one grade level in each school. We looked at whether 7th graders in a particular school performed better than the same school’s 8th graders (relative to the national average for their respective grades) when, on average, the 7th-grade classes were smaller than the 8th-grade classes. With this strategy, the variation in class size we considered is strictly a consequence of fluctuations in the cohort size from one grade to the next. This excludes variation in class sizes within the same grade and from school to school, both of which can be subject to the influence of parents and school-system policies that tend to sort students into classrooms by their performance. The remaining differences should be essentially unrelated to student performance.

This approach forced us to restrict the sample to schools in which both a 7th-grade and an 8th-grade class were actually tested and in which data on the actual class sizes and average class sizes were available for each grade. We ultimately conducted our analysis on the 18 countries in which data for at least 50 schools in both mathematics and science remained after applying these criteria.

As shown in Figure 1, Portugal exhibits the lowest average combined test scores in math and science among the 18 countries in our sample, Singapore the highest. Iceland has the smallest average class size, with just 20 students per classroom. At nearly 53 students per class, Korea has by far the highest average. The other East Asian countries also feature large classes, with an average of more than 30 students. In general, the countries with the smallest classes tended to be the worst performers. The reverse is also true: high performers tend to have larger classes. While this does not say much about the effectiveness of reducing class sizes in various environments, it does demonstrate that it is possible to have a high-achieving school system with relatively large classes.

Results

Let’s look first at the results of a straightforward comparison that adjusts the data on student performance for students’ socioeconomic background and grade level (since 7th and 8th graders were tested), thereby attempting to isolate the effects of class size. This initial analysis is of interest primarily because it is analogous to the approach used in most research on class size. Comparing these results with those obtained by a more reliable strategy will provide an indication of what biases may exist in other studies.

In 11 of the 18 nations, the estimate of the effects of class size were positive and statistically significant, suggesting that students in larger classes perform significantly better than students in smaller classes. In other words, a naïve strategy that does not account for the ways in which students are sorted into classes of different size leads to the counterintuitive result that students fare better in larger classes. Moreover, this result seems universal: it emerges in western Europe (Belgium, France), eastern Europe (Czech Republic, Romania), Australia, and East Asia (Hong Kong, Japan). No country showed students in smaller classes outperforming their peers in larger classes.

Let’s turn now to the preferred strategy, which controls for the fact that students performing at different levels may be sorted into smaller or larger classes both between and within schools. The first notable feature of this approach is the disappearance of the counterintuitive result that students do better in larger classes. In 16 of the 18 countries, none of the results was statistically different from zero. In the other two countries, Greece and Iceland, smaller classes did appear to elicit superior student performance. Moreover, the benefits appear to be substantial: Students scored just over two points (or 2 percent of the international standard deviation) higher for every one student fewer in their class.

The evidence suggests that capable teachers are able to promote student learning equally well regardless of class size. Photograph by Getty Images.

Precision Testing

What can be learned from the 16 countries where the results were statistically insignificant? Does this suggest the lack of a causal relationship between class size and student performance? Or is it merely the result of statistical imprecision? In four of the countries, Australia, Hong Kong, Scotland, and the United States, the standard error of the estimated effects of class size was extremely large, indicating that little confidence should be placed in the results. The lack of precision in these cases seems to be a direct consequence of our research strategy’s rather demanding data requirements. These school systems simply exhibit little variation in average class size from one grade to the next—the type of variation on which our strategy relied.

The remaining 12 countries can be further distinguished by comparing their results with those from other studies. We chose first to compare our results with those reported by Princeton economist Alan Krueger in his reanalysis of the Project STAR data from Tennessee, which produced some of the highest estimates of class-size effects among credible studies. Krueger performed a very rough cost-benefit analysis, in which the economic benefits of class-size reduction, in terms of the increase in future earnings due to higher test scores, appeared to approximate the costs.

Krueger’s results indicate that students in kindergarten classrooms that had 7 to 8 fewer students than regular-sized classes performed about 3 percent of a standard deviation better for every one student fewer in their class. Converted to international scores on TIMSS, this is equivalent to three test-score points. This is greater than the two-point gain we found in Iceland and Greece, but it is within the standard error of these estimates, suggesting that the actual effect of reducing class size in Iceland and Greece could be as large as Krueger found in the United States.

For 11 of the 12 countries with relatively precise yet statistically insignificant estimates, the possibility of class-size effects of the same size as Krueger found can be rejected with at least 95 percent confidence. There could still be class-size effects in these nations, just not of the magnitude estimated by Krueger. Note, however, that Krueger’s effects were found in kindergarten and 1st grade, while these estimates are for students in 7th and 8th grades.

We further tested to see whether a one-student reduction in class sizes would increase TIMSS scores by just one point, or 1 percent of an international standard deviation. An effect of this size would be so small as to be essentially negligible from the standpoint of public policy; a one-point gain is too little to justify the expense of class-size reduction. Regardless, even the possibility of this small an impact can be rejected with at least 90 percent confidence in 6 of our 12 school systems with reasonably precise results.

In short, the effect of class size on student performance varies across the 18 countries in our sample (see Figure 1). We can rule out even a minimal relationship between class size and TIMSS scores in the middle grades in six school systems: those of Flemish Belgium, Canada, Japan, Portugal, Singapore, and Slovenia. In an additional five school systems, we can rule out the possibility of large class-size effects: French Belgium, the Czech Republic, Korea, Romania, and Spain. These results cast doubt on the desirability of class-size reduction in the middle grades as a reform strategy in many countries. In Greece and Iceland, by contrast, smaller classes were clearly beneficial. (In five countries—Australia, France, Hong Kong, Scotland, and the United States—our strategy led to inconclusive estimates that do not allow for any confident assertions about the effects of differences in class size.)

Quantity versus Quality

Why would class-size reduction elicit improvement in Greece and Iceland but not elsewhere? One might expect class-size effects to be related to such characteristics as a nation’s overall level of resources. For instance, it is feasible that countries with relatively large classes would glean substantial benefits from reducing class sizes. However, there is no clear pattern in countries’ average class sizes that distinguishes the two countries where substantial class-size effects exist from either the six countries where we ruled out any noteworthy class-size effects or from the five countries where we ruled out at least large class-size effects. Greece’s average class size is similar to the mean class size among the nations where no class-size effects were found, and Iceland’s average class size is substantially lower (see Table 1).

One possibility is that class-size reduction has a large impact in relatively ineffective school systems. Both Greece and Iceland performed considerably below the international average on TIMSS, while the countries where class-size reduction did not have even a small effect performed above the average. Also, even though Greece’s class sizes are roughly at the mean and Iceland’s were substantially lower than the mean, education spending per student in both countries is substantially below the average of the two comparison groups. This suggests that Greece and Iceland spend rather little per employed teacher, which is reflected in the data on teachers’ salaries. Teachers’ salaries in Greece and Iceland are below the mean of the other countries in absolute terms, in terms of salary per teaching hour, and relative to the country’s per capita GDP (see Table 1).

A low average salary for teachers suggests that a country may be drawing its teaching population from a pool of less-skilled workers. If this is the case, different countries appear to be making different tradeoffs between the quantity and quality of their teachers: with class sizes low, Greece and Iceland employ many teachers of low quality. The countries where class-size effects were not observed appear to employ relatively fewer teachers, but of higher quality.

This assumption is borne out by the available data on teachers’ educational attainment. In Greece, the highest level of education reached by the vast majority of teachers is the equivalent of a bachelor’s degree without any teacher training. In Iceland, about one-third of the teachers surveyed by TIMSS had not even completed secondary education, with only some basic teacher training. Meanwhile, about 60 percent of the teachers surveyed in the other countries held either a bachelor’s or a master’s degree in addition to their training as teachers.

This evidence suggests that capable teachers are able to promote student learning equally well regardless of class size (at least within the range of variation that occurs naturally among grades). Less capable teachers, however, do not seem to be up to the job of teaching large classes.

This interpretation is corroborated by teachers’ responses in TIMSS when they were asked to what extent their teaching was limited by a high student-to-teacher ratio in their classroom. In Greece and Iceland, 45 percent of teachers reported that their teaching was limited “a great deal” by a high student-to-teacher ratio. The comparable statistics averaged only 19 percent and 25 percent among countries where no class-size effects and no large class-size effects were found, respectively. This is despite the fact that average class sizes in Greece and Iceland were lower than in either comparison group.

In short, our evidence suggests that the existence of class-size effects is related to the quality of the teaching force. Smaller classes appear to be beneficial only in countries where average teacher quality is low. If teacher quality is a key input in education, this interpretation can explain why class-size effects exist in some countries but not in others and at the same time why the countries in our sample where we did find sizable class-size effects also exhibit poor overall performance. Greece and Iceland exhibit class-size effects and poor performance because they employ a population of relatively less capable teachers, while other countries exhibit no class-size effects but high overall performance because they employ good teachers. This suggests that it may be better policy to devote the limited resources available for education to employing more capable teachers rather than to reducing class sizes. The merits of this admittedly speculative conclusion are a promising topic for future research.

-Martin R. West is a research fellow at the Harvard University Program on Education Policy and Governance and the research editor of Education Next. Ludger Woessmann is a senior researcher at the Ifo Institute for Economic Research in Munich, Germany.

What has this flurry of reform achieved? And what lessons, if any, can be drawn from this huge investment of political energy and public funds?

The answer to the first question is “not enough.” A quarter of British 11-year-olds still leave primary school unable to read well enough to deal with the demands of the secondary-school curriculum. Results from the General Certificate of Secondary Education exam that students take at age 16 show improvement each year, but there is a general recognition that grade inflation makes the progress illusory. Only 43 percent of 16-year-olds pass the General Certificate exams in the core subjects of English, mathematics, and science. Thousands leave secondary school with no meaningful qualifications at all.

The lesson of this failure is simple: the top-down imposition of politically inspired education reforms does not work.

In the early 1990s, when I became chief executive of the Schools Curriculum and Assessment Authority, the body responsible for introducing the national curriculum into English schools and administering the tests in English, mathematics, and science that children were to take at ages 7, 11, and 14, I thought differently. I supported the idea of a national curriculum that defined core bodies of knowledge in English, mathematics, science, history, geography, art, music, physical education, information technology, design technology, and, in secondary schools, a modern foreign language. Such a curriculum would, I thought, challenge the child-centered orthodoxies and raise academic expectations.

National testing would give us crucial information on how individual students and schools were performing. At the time, in England, as in America, too many parents expected too little. They had no point of reference, no data on how their child’s school was performing compared with other schools. More often than not, mediocrity went unchallenged. With testing, successful schools could be rewarded and the unsuccessful helped to improve. Failing that, they could be closed.

I also supported, and still do, what were known as “grant-maintained” schools. Grant-maintained schools, which are similar to charter schools in the United States, were established by the 1988 reforms. They are funded directly by the government and run independently of local education authorities. The thinking here was that the school should be responsible for its own destiny. Resources should not be wasted on local and national bureaucracies. It is the principal who makes the difference, not the distant bureaucrat. So let principals decide how the curriculum is to be organized and pupils taught. Give them the resources to do the job and hold them responsible for their school’s performance. However, one of the very first things the Labor government did on coming to power in 1997 was to abolish grant-maintained schools. The policy was termed elitist and divisive because only a minority of schools, usually successful ones, had applied for and been given grant-maintained status. It therefore had to go.

I was also enthusiastic about the new school inspection system, especially after I was appointed chief inspector of schools in 1994. The new policy required that every school be inspected every four years. The idea was that expert inspectors would comment on standards of pupil achievement, the strength of the principal’s leadership, the use of school resources, and the quality of what we called, rather quaintly, “social, moral, spiritual and cultural education.” A report was to be published immediately following the inspection, and the failure of underperforming schools was to be made public.
In other words, from the beginning I supported nearly every change wrought by the Education Reform Act in 1988. That was before government officials and various education groups got into the act and diluted nearly every reform, rendering the changes impotent.

The National Curriculum

From day one, there was a battle over the content of the new national curriculum. Politicians felt that they had to involve the education “experts.” Predictably, these experts fought strenuously for their pet theories, which were usually progressive. Skills were deemed more important than knowledge. Strenuous attempts were made to introduce “cross curricular themes” that, it was hoped, would undermine the focus on separate subject disciplines. In an open letter published in the London Times Higher Education Supplement, 576 English professors and lecturers complained that the government’s “doctrinaire preoccupation” with grammar and spelling “betrayed a disastrously reductive, mechanistic understanding of English studies.” They objected to making the study of Shakespeare compulsory in secondary schools and rejected as “dictatorial” a plan to “impose” a “canon of supposedly great works,” including having all pupils read some Dickens and Wordsworth.

One of the results of all this lobbying has been the removal of the knowledge base from subjects like geography and history. The emphasis in geography is now on “sustainable development,” field-work techniques, and the teaching of general skills (such as how to “collect, record and analyze evidence”). In history, 7-year-olds are encouraged “to see the diversity of human experience, and understand more about themselves as individuals and members of society.” The fact that many students leave school at the age of 16 knowing next to nothing about the history of their own country does not appear to matter.

By all indications, officials in the Blair government support the progressivist tilt in the curriculum. “We need,” said Blair appointee Estelle Morris just before resigning in 2002 as secretary of state for education, “a shift in understanding away from the old model of teaching as transmission of facts and figures towards one which captures the teacher’s role as expert practitioners in advanced pedagogy.” David Hopkins, who Morris appointed to the key role of head of the Standards and Effectiveness Unit at the Department for Education and Skills, believes that “when students begin to take ownership of the learning behavior, you see something quite transformational taking place inside the school, because then it is the students who actually control learning rather than the teacher.”

The ideas about education and teaching that determine government policies and drive the spending of billions of pounds of public money are important. Yet the state relies on the “thought world” of the education establishment to define and implement its program of reform. The result is the waste of huge sums of money and the exhaustion and anger of teachers who want to get on with the job of teaching.

So how can a society establish a standards-based curriculum without having it hijacked by the education experts whose views are responsible for the mess we are trying to resolve? The key is a political will that remains resolute in the face of carefully orchestrated resistance and has the confidence to challenge the arguments of the professional lobby groups.

Inflated Results

Then there are the tests and examinations that are meant to be a robust, objective measure of students’ and schools’ performance. The problem is the fact that the national government, which desires that the electorate view its reforms favorably, controls both the national curriculum tests that pupils take at the ages of 7, 11, and 14 and the public examinations that are taken at ages 16 (the General Certificate of Secondary Education exams meant to assess skills and knowledge in the traditional academic subjects) and 18 (the Advanced Level exams for students hoping to attend university). This control is denied, but the organization that holds direct responsibility for the administration of the tests and the maintenance of standards in public examinations (the Qualifications and Curriculum Authority, formerly the Schools Curriculum and Assessment Authority) has, as I know from my experience when I was its chief executive, a very fragile independence. Meetings with government officials are held regularly. Department of Education officials attend all board meetings. We knew exactly what the ministers wanted, and they knew exactly what we were doing.

Consequently, results from these tests and examinations appear to improve each year, but endemic grade inflation, maladministration, and, in my view, political interference have undermined public and professional confidence in the entire examination system.

For instance, in 2002, the scores 11-year-olds needed to pass the national curriculum tests in English and mathematics were reduced by 4 percent and 5 percent, respectively. Was this done, as the Qualifications and Curriculum Authority argues, because 2002′s test was significantly harder than the previous year’s? I very much doubt it. In fact, the public examinations have been diluted across the board. David Burghes, a professor of mathematics in Exeter University’s education department, recently wrote, “It has become obvious that a [General Certificate of Secondary Education] tells you nothing. You can get a grade C in Mathematics (the pass mark) without being able to do long division and multiplication or anything to do with decimal fractions without a calculator.”

A biology question that was on the 1979 examination for 16-year-olds turned up 16 years later as an Advanced Level question for 18-year-olds. In 1989, the mark needed to achieve a grade C in one mathematics examination for the General Certificate of Secondary Education was 48 percent. In 2000 it was 18 percent. Students can get four-fifths of the answers wrong and still pass. Moreover, university admissions tutors report escalating worries about the knowledge base of applicants for their courses. Cambridge University has extended the term needed to earn a mathematics degree from three years to four because the tutors found that today’s students were simply not able to make enough progress in the time previously allocated to the degree.

It is hard, given such evidence, to have much, if any, confidence in England’s system of tests and examinations. A recent survey of ordinary classroom teachers found that they also think the examinations have become easier. But the government and the teacher unions remain in denial. “We have every reason,” a Qualifications and Curriculum Authority spokesman reassured the great British public last year, “to think improvements in grades are a consequence of hard work and better preparation of pupils and teachers.”

Self-Inspection

Would the system of school inspections make up for the problems with the national curriculum and testing? At first the inspection system showed real promise. Of the 24,000 state-supported schools in England, 1,500 have failed their inspection since the system of inspections began in 1992. Many of these schools had been failing for years, yet no one had bothered to do anything. With the establishment of inspections, however, the trauma of public humiliation, coupled with a change in senior staff and some extra resources, led most failing schools to improve. Done well, the inspections provided school managers with invaluable information about how their school was performing. The inspections also provided politicians and their bureaucrats with the information they needed to formulate relevant policies and to target resources intelligently. More generally, the prospect of inspection concentrated a large number of minds that needed concentrating.

But of course the unions and the education establishment hated the whole thing. Why, they asked plaintively, did the government want “to pillory and demoralize” hard-working teachers? They compared school inspections to the Spanish Inquisition. And in the end they won. During the past four years, the inspection process has been softened gradually in response to teachers’ criticism. Schools are now inspected less frequently, less extensively, and, some would say, less rigorously. In 2004 the idea is to make the inspections a check on the school’s own self-evaluation, thus destroying the whole idea of an independent appraisal.

Good Money Chasing Bad Ideas

My sense, seven years after the Labor Party took control, is that the wheel has more or less come full circle. The old socialist belief that problems can be solved by throwing money at them has replaced a commitment to real change. The secretary of state for education insists that the decision to spend more on education is dependent on the willingness of the profession to “modernize,” but the truth is that billions of pounds have been spent with precious little to show for it.

For instance, 230 million pounds were wasted on a policy known as Education Action Zones that was meant to encourage partnerships between business and schools in deprived areas. It had no discernible effect on standards and has now been abandoned. A National College of School Leadership has been established with a budget of 60 million pounds a year. Does it have to send groups of prospective principals to China for no apparent reason? Should it really be encouraging schools to believe that it is better to teach “emotional literacy” than reading and writing? The current secretary of state for education, Charles Clarke, wants every secondary school to become a “specialist” school focusing on a particular subject, such as science, technology, business, or foreign languages. The fact that the inspectorate has not been able to find any link between specialist school status and improved examination standards is, apparently, irrelevant.

Perhaps the best example of the waste of public money is the scheme to introduce a system of performance pay for teachers. In theory, this is a sensible idea. Good schools and successful teachers ought to be rewarded for their success. But in practice it has been a disaster. The absurdity is that once teachers have received their “bonus,” they will be paid that extra money for the rest of their professional life. There is no annual setting of targets, no review of performance, and no possibility, therefore, of the enhanced motivation that additional pay is meant to encourage. The government pretends that it has overseen a “radical modernization of the teaching profession.” Hogwash. Of teachers who applied for the extra pay, 95 percent crossed the threshold. Our teachers are being paid more. End of story.
In the past few years, the belief that centralized initiatives coupled with massive new “investment” will solve the problems of state education has triumphed over the drive to open the performance of state schools to public scrutiny. The bureaucracies, local and national, have grown in size and power as the number of initiatives has multiplied. Many of the new initiatives are based on the educational ideas that created the problems in our schools, such as the drive to teach “thinking skills” in a knowledge vacuum.

What is the alternative? The way forward is to return to and extend the concept of the grant-maintained schools that Labor abolished in 1997. Principals should be allowed to run their schools according to their own professional judgment and the wishes and aspirations of parents. Teachers should not have to waste 25 percent of their working day on bureaucratic tasks that add nothing to their effectiveness in the classroom. Forty percent of every pound spent on state education should not be squandered on the bureaucracies that are responsible for the miseducation of so many children.
It does not have to be like this. The state does not have to micromanage what happens in each of its schools. We could trust parents more and politicians and their bureaucrats less. But there is little sign that the current government really wants to involve the private sector in the running of state education or to find ways, through vouchers or tax credits, of genuinely empowering parents. Neither does our Conservative Party seem to have the courage of what ought to be its natural convictions. Sooner or later, however, because the present situation is untenable, change will come. Taxpayers will realize that they are paying more for public services that have not improved. Parents will grow more restive as they fail to secure a place for their children in a decent school. The government of the day will have to respond.

-Christopher Woodhead is a professor of education at the University of Buckingham. He served as chief inspector of schools in Britain from 1994 to 2000.

But if we reflect on these beliefs in a foreign context and observe low-income families in underprivileged and developing countries, we find these assumptions lacking: the poor have found remarkably innovative ways of helping themselves, educationally, and in some of the most destitute places on Earth have managed to nurture a large and growing industry of private schools for themselves.

For the past two years I have overseen research on such schools in India, China, and sub-Saharan Africa. The project, funded by the John Templeton Foundation, was inspired by a serendipitous discovery of mine while I was engaged in some consulting work for the International Finance Corporation, the private finance arm of the World Bank. Taking time off from evaluating an elite private school in Hyderabad, India, I stumbled on a crowd of private schools in slums behind the Charminar, the 16th-century tourist attraction in the central city. It was something that I had never imagined, and I immediately began to wonder whether private schools serving the poor could be found in other countries. That question eventually took me to five countries—Ghana, Nigeria, Kenya, India, and China—and to dozens of different rural and urban locales, all incredibly poor. Since the data gathered from Lagos, Nigeria, and Delhi, India, are not yet fully analyzed, this article reports on findings only from Gansu Province, China; Ga, Ghana; Hyderabad, India; and Kibera, Kenya. These are in vastly different settings, but my research teams and I found large numbers of private schools for low-income families, many of which showed measurable achievement advantage over government schools serving equally disadvantaged students.

Myth One: Private Education for the Poor Does Not Exist

Undertaking this research was disheartening at first. In each country I visited, officials from national governments and international agencies that donate funds for the expansion of state-run education denied
that private education for the poor even existed. In China senior officials told me that what I was describing was “logically impossible” because “China has achieved universal public education and universal means for the poor as well as the rich.” At other times, in other places, I met with polite, if embarrassed, apologies that always went something like, “Sorry, in our country, private schools are for the privileged, not the poor.”

In each venue, however, I struck out on my own and visited slums and villages and there found what I was looking for: private schools for the poor, usually in large numbers, if sometimes hidden from view. In the slums of Hyderabad, India, a typical private school would be in a converted house, in a small alleyway behind bustling and noisy streets, or above a shop. Classrooms are dark, by Western standards, with no doors hung in the doorways, and noise from the streets outside easily entering through the barred but unglazed windows. Walls are painted white, but discolored by pollution, heat, and the general wear-and-tear of the children; no pictures or work is hung on them. Children will usually be in a school uniform and sitting at rough wooden desks. Generally, there are about 25 students in a class, a decent teacher-to-student ratio, but the tiny rooms always seem crowded. Often the top floor of the building will have various construction work going on to extend the number of classrooms. The school proprietor will usually live in a couple of rooms at the back of the building.

In rural Ghana, a typical private school might consist of an open-air structure, often no more than a tin roof supported by wooden poles, on a small plot of land. To find these schools you’ll have to wander down meandering narrow paths, away from the main thoroughfares, asking villagers as you go. If you ask simply for the “school,” they’ll send you back to the public school, usually an impressive brick building on the main road. You’ll have to persist and say you want the “small” school to get directions.

In the slums of Nairobi, Kenya, private schools are made from the same materials as every other building: corrugated iron sheets or mud walls, with windows and doors cut out to allow light to enter. Floors are usually mud, roofs sometimes thatched. Children will not be in uniform and will usually be sitting on homemade wooden benches. In the dry season, the wind will blow dust through the cracks in the walls; in the rainy season, the playground will become a pond, and the classroom floors mud baths. Teaching continues, however, through most of these intemperate interruptions.

In order to conduct research in five countries from my base in Newcastle, England, I recruited teams of researchers from reputable local universities and nongovernmental organizations (NGOs). While fielding the research crews, I visited dozens of likely study sites, always in low-income areas, and always found private schools for the poor. I also visited government offices to gain permission to conduct the research. In the end, all of the chosen countries, apart from China, were rated by the Oxfam Education Report as countries where education needs were not being met by government systems. Though China is ranked relatively high on the Oxfam index, we wanted to include it in our study because of the dramatic political and economic changes there in the past several decades. (Because of the threat of SARS, however, our first research team spent a long period in quarantine and thus our research there is not yet complete.) Other countries
were chosen for a mixture of practical and substantive reasons. I was particularly interested in Kenya, where free elementary education had just been introduced to much acclaim. How would this affect private schools for the poor, should they exist? I had conducted research earlier in Hyderabad, India, was familiar with the terrain, and had many contacts in government and the private sector, so it seemed sensible to continue the project there. And because of a chance meeting with the Ghanaian minister of education at a conference in Italy, we were invited to that western African nation.

Many difficulties emerged that I had not taken account of as the project progressed. Heavy rains prevented the research teams from moving around in both Ghana and Nigeria for weeks at a time; intense heat delayed work for days in Hyderabad; early snowfalls hampered movement in the mountains of China. But above all, a major difficulty was getting the extended research teams to take seriously the notion that we really were interested in the low-key, unobtrusive private schools that apparently were easily dismissed. In each of the settings, on unannounced quality control visits, I found unrecognized private schools that had not been reported by the teams.

Hyderabad, India

Visit the ultramodern high-rise development of “High Tech City” and you’ll see why Hyderabad dubs itself “Cyberabad,” proud of its position at the forefront of India’s technological revolution. But cross the river Musi and enter the Old City, with once magnificent buildings dating to the 16th century and earlier, and you’ll see the congested India, with narrow streets weaving their way through crowded markets and densely populated slums. For our survey, we canvassed three zones in the Old City (Bandlaguda, Bhadurpura, and Charminar), with a population of about 800,000 (about 22 percent of all of Hyderabad), covering an area of some 19 square miles. We included only schools that were found in “slums,” as determined by the latest available census and Hyderabad municipal guides, areas that lacked amenities such as indoor plumbing, running water, electricity, and paved roads.

In these areas alone our team found 918 schools: 35 percent were government run; 23 percent were private schools that had official recognition by the government (“recognized”); and, incredibly, 37 percent slipped
under the government radar (“unrecognized”). The last group is, in effect, a black market in education, operating entirely without both state funding and regulation. (The remaining 5 percent were private schools that received a 100 percent state subsidy for teachers’ salaries, making them public schools in all but name.) In terms of total student enrollment in the slum areas of the three zones, with 918 schools, 76 percent of all schoolchildren attended either recognized or unrecognized private schools, with roughly the same percentage of children in the unrecognized private schools as in government schools (see Figure 1).

SOURCE: Author’s calculations based on original research and local government figures. SOURCE: Author’s calculations based on original research and local government figures.

What is clear from our research is that these private schools are not mom-and-pop day-care centers or living-room home schools. The average unrecognized school had about 8 teachers and 170 children, two-thirds in rented buildings of the type described above. The average recognized school was larger and usually situated in a more comfortable building, with 18 teachers and about 490 children. Another key difference between the recognized and unrecognized schools is that the former have stood the test of time in the education market: 40 percent of unrecognized schools were less than 5 years old, while only 5 percent of recognized schools were this new. Finally, tuition in these schools is very low, averaging about $2.12 per month in recognized private schools at 1st grade and $1.51 in unrecognized schools.

While these fees seem extremely low, they must be measured against the average income of each person in the student’s household who is working for pay. For students in unrecognized schools, this was about $23 per month, compared with about $30 per month for students in recognized schools and $17 for government schools. Since the official minimum wage in Hyderabad is $46 per month, it is clear that the families in the private schools we observed are poor. Fees amount to about 7 percent of average monthly earnings in a typical household using a private unrecognized school. For the poorest children, the schools provide scholarships or subsidized places: 7 percent of children paid no tuition and 11 percent paid reduced fees. In effect, the poor are subsidizing the poorest.

Ga, Ghana

The Ga district of southern Ghana, which surrounds the country’s capital city of Accra, is classified by the Ghana Statistical Service as a low-income, urban periphery, and rural area. With a population of about 500,000, Ga includes poor fishing villages along the coast, subsistence farms inland, and large dormitory towns for workers serving the industries and businesses of Accra itself. Most of the district lacks basic social amenities such as potable water, sewage systems, electricity, and paved roads. In Ga’s towns and villages our researchers found a total of 799 schools, 25 percent of which were government, 52 percent recognized private, and 23 percent unrecognized private. In total, 33,134 children were found in unrecognized private schools, or about 15 percent of children enrolled in school (see Figure 2).

The average monthly fee for an unrecognized private school in Ga is about $4 for the early elementary grades, about $7 in recognized schools. With a minimum wage of about $33 per month in the area, monthly fees in the private unrecognized schools are thus about 12 percent of the average monthly earnings of an adult earner. However, many of the poorest schools allow a daily fee to be paid so that, for instance, a poor fisherman could send his daughter to school on the days he had funds and allow her to make up for the days she missed. Such flexibility is not possible in the public schools, where full payment of the “levies” is required before the term starts. (Fees for “public” schools are common in many countries throughout the Third World, especially at high-school level. Thus the cost of private schools, we found, can sometimes be less than that of government ones.)

Unlike India, where there are restrictions on private-school ownership (private schools must be owned by a society or trust), in Ga the vast majority of private schools (82 percent of recognized and 93 percent of unrecognized) are run by individual proprietors; most of the rest are owned and managed by charitable organizations. Sometimes, as is common in other African countries, such schools rent church buildings or use Christian-related names, but only in a few cases are the schools run by churches. Often it is the school that subsidizes the church rather than the other way around!

Gansu, China

With 25.3 million people spread out over an area the size of Texas, Gansu province is a remote and mountainous region situated on the upper and middle reaches of the Yellow River in northwest China. It has an average elevation of over 3,000 feet and 75 percent of its population is rural, with illiteracy rates among people aged 15 or older at nearly 20 percent for men and 40 percent for women. Roughly half of its counties, with 62 percent of the population, are considered “impoverished.”

Figures from the Provincial Education Bureau show only 44 private schools in the whole province, all of which are for privileged city dwellers. Given the paucity of information on private schools, I asked my research teams to survey each major town in each of the counties designated as impoverished (more than 40 of them) and to visit as many of the outlying villages accessible to them as they could. In the early stages I wasn’t worried about getting precise estimates of the numbers of schools or the proportion of children in them, but rather wanted to see if such schools even existed.

In the major towns and the larger villages, all of them crowded and bustling, there is always a public school, usually a fine two-story building that sports a plaque marking it as a recipient of some kind of foreign aid. But researchers had to abandon their cars and either walk or hitch a ride on one of the ubiquitous and noisy three-wheeled farm vehicles to travel up the steeper mountain paths to clusters of houses in smaller villages to find the private schools. And there, nestled on mountain ridges, were stone or brick houses converted to schools, with the proprietor or headmaster living with his family in one or two of its rooms. Occasionally, the school had been built, by the villagers, to be used as a school. Over and over again, researchers followed these trails high into the arid mountains and, in the end, discovered a total of 696 private schools, 593 of them serving some 61,000 children in the most remote villages.

Not surprisingly, the vast majority of Gansu’s private schools were set up by individuals, or the villages themselves, because government schools are simply too far away or hard to get to. Significantly, the majority of the private schools found were in the three poorest regions of Gansu, where average net income per year ranges from $125 to $166. These private schools are serving some of the poorest people on the planet. But surprisingly, the schools, which depend on tuition, are also cheaper than government schools. Average fees for a first-year, elementary-school student are about $7.60 per semester, compared with about $8.00 in the public schools, not an insignificant difference to someone living on $125 per year.

Kibera, Kenya

In Kenya we conducted our censuses in three urban slums of Nairobi (Kibera, Mukuru, and Kawangware), where, according to Kenyan government officials, there were no private schools. The picture in each was similar; here I describe the findings for Kibera only.

The largest slum in all of sub-Saharan Africa, Kibera has, according to various estimates, anywhere from 500,000 to 800,000 people crowded into an area of about 630 acres, smaller than Manhattan’s Central Park. Mud-walled, corrugated iron-roofed settlements huddle along the old Uganda Railway for several miles and crowd along steep narrow mud tracks until Kibera reaches the posh suburbs. In Nairobi’s two rainy seasons, the mud tracks become mud baths. In this setting, we found 76 private elementary and high schools, enrolling more than 12,000 students. The schools are typically run by local entrepreneurs, a third ofwhom are women who have seen the possibility of making a living from running a school. Again, many of the schools offered free places to the poorest, including orphans.

When I first visited Kibera, many private-school proprietors were feeling the effects of so-called Free Primary Education (FPE), introduced by the Kenyan government in January 2003 with great fanfare and a $55 million grant from the World Bank. In fact, when asked by ABC anchorman Peter Jennings which one living person he would most like to meet, former president Bill Clinton told a prime-time television audience that it was President Mwai Kibaki of Kenya, “Because he has abolished school fees,” which “would affect more lives than any president had done or would ever do by the end of this year.” Indeed, official sources estimated that an extra 1.3 million children would be enrolled in public schools after the introduction of FPE: all of them children, it was said, not previously enrolled in school.

The reality may be very different. Private-school owners in Kibera alone reported a total enrollment decline of some 6,500 after Free Primary Education was initiated; some schools closed altogether. We estimated that about 4,500 children had been enrolled in 25 schools that we confirmed had closed as a result of FPE. At the same time five government primary schools on the periphery of Kibera that served the slums reported a total increase of only about 3,300 children during this period. That is, since the introduction of free elementary education, there appeared to have been a net decline in attendance of nearly 8,000 children from one slum alone! Clearly, these figures are based on the reported decline by school owners and may be exaggerated. But they also suggest the possibility that government and international intervention had the effect of crowding out private enterprise.

Myth TWO: Private Education for the Poor Is Low Quality

It is a common assumption among development experts that private schools for the poor are worse than public schools. This is not to say that they have a particularly high view of public education. Indeed, the World Bank’s World Development Report 2004: Making Services Work for Poor People calls public education a “government failure,” with “services so defective that their opportunity costs outweigh their benefits for most poor people.” Yet this just makes the experts’ dismissal of private schools for the poor all the more inexplicable.

The Oxfam Education Report published in 2000 is typical. While the author acknowledges the existence of high-quality private providers, he contends that these are elite, well-resourced schools that are inaccessible to the poor. As far as private schools for the poor are concerned, these are of “inferior quality”; indeed, they “offer a low-quality service” that is so bad it will “restrict children’s future opportunities.” This claim of low-quality private provision for the poor has also been taken up by British prime minister Tony Blair’s Commission for Africa, which recently reported that although “Non-state sectors … have historically provided much education in Africa,” many of these private schools “aiming at those [families] who cannot afford the fees common in state schools … are without adequate state regulation and are of a low quality.”

However, these development experts have little hard evidence for their assertions about private-school quality. They instead point out that private schools employ untrainedteachers who are paid much less than their government counterparts and that buildings and facilities are grossly inadequate. Both of these observations are largely true. But does that mean that private schools are inferior, particularly against the weight of parental preferences to the contrary? One Ghanaian school owner challenged me when I observed that her school building was little more than a corrugated iron roof on rickety poles and that the government school, just a few hundred yards away, was a smart new school building. “Education is not about buildings,” she scolded. “What matters is what is in the teacher’s heart. In our hearts, we love the children and do our best for them.” She left it open, when probed, what the teachers in the government school felt in their hearts toward the poor children.

Facilities and Resources

The issue of the relative quality of private and public schools was at the core of our research, and we relied on both data on school resources and day-to-day operations and on student achievement scores. Our researchers first called unannounced at schools and asked for a tour, noted what teachers were doing, made an inventory of facilities, and administered detailed questionnaires.

Certainly, in some countries the facilities in the private schools were markedly inferior to those in the public schools. In China, where the researchers were asked to locate a public school in the village nearest to where they had found a private school, often many miles away, private-school facilities were generally worse than in those publicly provided. This was predictable, given that the private schools undercut the public ones in fees and served the poorest villages, where there were no public schools. In Gansu province, desks were available in classrooms in 88 percent of private schools, compared with 97 percent of public schools; 66 percent of private schools had chairs or benches in classrooms, compared with 76 percent of public schools. In Kenya, parallel results would be expected, given that the private schools surveyed were located in the slums, while the public schools were on the periphery, accommodating both poor and middle-class children. However, given that there were only 5 government schools on the periphery of Kibera, but 76 private schools within the slum, statistical comparisons would make little sense.

In Hyderabad, however, on every input, including the provision of blackboards, playgrounds, desks, drinking water, toilets, and separate toilets for boys and girls, both types of private schools, recognized and unrecognized, were superior to the government schools. While only 78 percent of the government schools had blackboards in every classroom, the figures were 96 percent and 94 percent for private recognized and unrecognized schools, respectively. In only half the government schools were toilets provided for children, compared with 100 percent and 96 percent of the recognized and unrecognized private schools.

Finally, in Ghana, the picture is mixed. For instance, 95 percent of government schools in Ga had playgrounds, compared with 66 percent and 82 percent of private unrecognized and recognized schools, respectively. Desks were provided in 97 percent of government schools, but only in 61 percent of private unrecognized; recognized private schools provided them in 92 percent of cases. However, only 54 percent of government schools provided drinking water to children compared with 63 percent of private unrecognized and 87 percent of private recognized schools. And 63 percent of government schools provided toilets, compared with 91 percent of recognized but only 59 percent of unrecognized private schools. A library was provided in 8 percent of government, 7 percent of private unrecognized schools, but 27 percent of private recognized schools. At least one computer for the use of children was provided in only 3 percent of government schools, but in 12 percent of private unrecognized and 37 percent of private recognized.

When it came to the key question of whether or not teaching was going on in the classrooms, both types of private schools were superior to the public schools, except in China, where there was no statistically significant difference between the two school types: 92 percent of teachers in private schools were teaching when our researchers arrived, compared with 89 percent in the public schools. When researchers called unannounced on the classrooms in Hyderabad, 98 percent of teachers were teaching in the private recognized schools, compared with 91 percent in the unrecognized and 75 percent in the government schools. Teacher absenteeism was also highest in the government schools. In Ga, 57 percent of teachers were teaching in government schools, compared with 66 percent and 75 percent in unrecognized and recognized private schools, respectively. And in Kibera, even though the number of government schools is too small to make statistical comparisons meaningful, 74 percent of teachers were teaching in private schools when our researchers visited them, and only one teacher was absent.

It was also the case that private and public schools in China had more or less the same pupil-teacher ratio, about 25:1. In Hyderabad, private schools, including the unrecognized ones, had significant advantages over the government schools: the average pupil-teacher ratio was 42:1 in government schools compared with only 22:1 in the unrecognized and 27:1 in the recognized private schools. In Ga the pupil-teacher ratio was superior in private schools, with a ratio of 29:1 in government, compared with 21:1 and 20:1 in unrecognized and recognized private schools, respectively.

Student Achievement

To compare the achievement of students in public and private schools in each location where we conducted research, we first grouped schools by size and management type: government, private unrecognized, and private recognized in Ga and Hyderabad; government and private in Kibera, where the private schools are all of a similar type. (China is not discussed here because research there is continuing.) As noted above, in Ga and Hyderabad we were comparing public and private schools that were located in similar, low-income areas, while in Kibera, private schools served only slum children, and public schools served middle-class children as well as slum children. But this makes the comparisons in Kenya even more dramatic. Although serving the most disadvantaged population in the region, Kibera’s private schools outperformed the public schools in our study, after controlling for background variables.

We tested a total of roughly 3,000 students in each setting in English and mathematics; in state languages in India and Kenya; religious and moral education in Ghana; and social studies in Nigeria. All children were also given IQ tests, as were their teachers. Finally, questionnaires were distributed to children, their parents, teachers, and school managers, seeking information on family backgrounds.

Our analysis of these data is still in progress. However, in all cases analyzed so far—Ga, Hyderabad, and Kibera—students in private schools achieved at or above the levels achieved by their counterparts in government schools in both English and mathematics (see Figure 3).

SOURCE: Author’s calculations based on original research and local government figures

Moreover, the private-school advantage only increases with consideration of the differences in an unusually rich array of characteristics of the students, their families’ economic status, and the resources available at their schools. In Hyderabad, students attending recognized and unrecognized private schools outperformed their peers in government schools by a full standard deviation in both English and math (after accounting for differences in their observable characteristics). In Ghana, the adjusted private-school advantagewas between 0.2 and 0.3 standard deviations in both subjects. Finally, in Kenya, where the raw test scores showed students in private and public schools performing at similar levels, the fact that private schools served a far more disadvantaged population resulted in a gap of 0.1 standard deviations in English and 0.2 standard deviations in math (after accounting for differences in student characteristics). The adjusted differences between the performance of public and private sectors in each setting were highly statistically significant.

In short, it is not the case that private schools serving low-income families are inferior to those provided by the state. In all cases analyzed, even the unrecognized schools, those that are dismissed by the development experts as being obviously of poor quality seem to outperform their public counterparts.

Lessons for America

So the accepted wisdom appears to be wrong. Though elite private schools do exist in impoverished regions of the world, private schools are not only for the privileged classes. From a wide range of settings, from deepest rural China, through the slums of urban India and Kenya, to the urban periphery areas o Ghana, private education is serving huge numbers of children. Indeed, in those areas where we were able to adequately compare public and private provision, a large majority of schoolchildren are in private school, a significant number of them in unrecognized schools and not on the state’s radar at all.

Ironically, perhaps, the accepted wisdom does seem to be right on one point: private is better than public. Of course, no one suspected that private slum schools would be better. Yet our research suggests that children in these schools outperform similar students in government schools in key school subjects. And this is true even of the unrecognized private schools, schools that development experts dismiss, if they acknowledge their existence at all, as being of poor quality.

Clearly the evidence presented here may have implications for the continuing policy discussions over how to achieve universal education worldwide and for American development policy, especially programs of the United States Agency for International Development (USAID) and the World Bank. William Easterly, in his Elusive Quest for Growth (see also “Barren Land,” Fall 2002), notes the ineffectiveness of past investments in public schools by the international agencies and developing country governments, pointing out: “Administrative targets for universal primary education do not in themselves create the incentives for investing in the future that matter for growth,” that is, in quality education. If the World Bank and USAID could find ways to invest in private schools, then genuine education improvement could result. Strategies to be considered include offering loans to help schools improve their infrastructure or worthwhile teacher training, or creating partial vouchers to help even more of the poor gain access to the private schools that are ready to take them on.

But does the evidence have any implications for the school choice debate in America itself? The evidence from developing countries might challenge the claim, made by school choice opponents, that the poor in America cannot make sensible and informed choices if school choice is offered to them. It may also stimulate debate about whether public intervention crowds out private initiative, a question raised by the findings from Kenya.
If a public school is failing in the ghettoes of New York or Los Angeles, we should not assume that the only way in which the disadvantaged can be helped is through some kind of public intervention. In fact, we have already embarked on programs that support private initiative, with government support, with vouchers and charter schools. The findings here suggest this alternative approach may be the preferable one.

Above all, the evidence should inspire those who are working for school choice in America: stories of parents’ overcoming all the odds to ensure the best for the children in Africa and Asia, stories of education entrepreneurs’ creating schools out of nothing, in the middle of nowhere. If India can, why can’t we?

James Tooley is professor of education policy, the University of Newcastle upon Tyne, England. This essay is supported by a grant from the John M. Templeton Foundation.