In the K-12 education market, where countries the world over publicly finance and manage the great majority of their schools, the institutions and policies established by various levels of government must create incentives for school personnel to use their resources in ways that maximize performance. In the private sector, where firms are disciplined by market competition, it is usually assumed that resources are used effectively because firms would otherwise fail to profit. Inefficiency leads to higher costs and higher prices–practically an invitation to competitors to lure away customers. But the relative lack of competition in the K-12 education sector tends to dull the incentives to improve quality and restrain costs. Moreover, in the public system, the ability of parents and students to ensure that they receive a high-quality education is constrained by the enormous obstacles to leaving a bad school. Families must rely almost exclusively on the government, school administrators, and school personnel to monitor one another’s behavior and to create appropriate quality-control measures.
Within a country’s educational system, the relevant institutions and policies include the ways in which a society finances and manages its schools, how a society assesses student performance, and who is empowered to make basic educational decisions, such as which curricula to follow, which teachers to hire, and what textbooks to purchase. If resources are to be used effectively, policies must create incentives that encourage school personnel to behave in ways that do not necessarily further their own interests. For instance, without the right incentives, teachers may avoid using the most promising teaching strategies, preferring to use the techniques they find most convenient. In terms of policy, one might speculate that if a nation assesses the performance of students with some sort of national exam and uses this information to monitor teachers, teachers will put aside their other interests and focus mainly on raising student achievement.
International Evidence
This study asks two basic questions: Do policy and institutional variation help to explain variation in student performance? If so, which policies and institutions are most conducive to student performance? To answer these questions, I turn to the international evidence on student achievement. This is because the institutions within a country do not vary enough to test how different institutions affect student achievement. Only the international evidence, which encompasses many education systems with a wide variety of institutional structures, has the potential to show which institutions heavily affect student performance. My working hypothesis is that differences in educational institutions explain more of the international variation in student performance than differences in the resources nations devote to schooling.
A large body of empirical evidence on the effects of resources on student achievement already exists. It overwhelmingly shows that, at given spending levels, an increase in resources does not generally raise educational performance. Studies summarized by Eric Hanushek of the Hoover Institution have shown the lack of a strong, systematic relationship between resources and performance within the United States, within developing countries, and among countries. Likewise, studies by Erich Gundlach and myself at the Kiel Institute of World Economics have found no systematic relationship between resources and performance across time within most countries in the Organization for Economic Cooperation and Development (OECD) and within some countries in East Asia.
Data from the Third International Mathematics and Science Study (TIMSS) again show that differences from country to country in per-pupil spending do not help in understanding differences in educational performance. The simple correlation between spending per student and average TIMSS test scores is 0.13 in primary school and 0.16 in middle school, on a scale where 1.0 denotes an absolute positive correlation between the two variables and 0 signals no correlation (see figure 2). This means that school productivity, the ratio of educational performance to the level of spending, differs widely across schooling systems.
There is no consensus on the lack of a strong positive relationship between educational resources and performance, however. In the within-country literature, some scholars have questioned the use of meta-analyses, while others have suggested the use of alternative measures of school performance, such as students’ future labor-market performance. Still others point to controlled and quasi-controlled empirical experiments that have shown that more resources can lead to higher achievement. Notwithstanding this debate, the international variation in student performance levels in mathematics and science is a fact, and it is generally accepted that differences in the amount of resources given to the education sector do not fully explain why performance levels vary.
Data
This study uses data from 39 countries to analyze how various institutions affect educational performance at the student level. I constructed a student-level database that combines data from TIMSS with data on education systems from the OECD. TIMSS is the latest, largest, and most extensive international student achievement test ever conducted. In 1994-95, representative samples of students in more than 40 countries were tested (for various reasons, data files were available for only 39 countries for this study). Countries participating in the study were required to administer tests to students in the middle-school years, but could choose whether or not to participate in the primary and final school years. This paper focuses on the middle-school years, where students enrolled in the two adjacent grades containing the largest proportion of 13-year-old students (7th- and 8th-graders in most countries) were tested. This data set includes data on more than 250,000 individual students, who form a representative sample of a population of more than 30 million students in the 39 countries.
TIMSS contains student-level data on achievement and family background and various institutional data: class-level data on teachers, and school- and country-level data on the distribution of decision-making powers within the education system. TIMSS does not include data on spending, so current national public spending per student in secondary education in international dollars was calculated on the basis of UNESCO and World Bank data. Further country-level data on institutional features of the education system–mainly concerning the distribution of decision-making powers and the size of the private-schooling market–come from the OECD educational indicators.
I performed the analysis at the level of the individual student (not the class, school, district, or country) because this directly links student performance to the teaching environment. This allowed me to control for the influence of each individual student’s background, assess the influence of the actual resource level and teacher characteristics each student faces, and look at the institutional features that are relevant to individual students. Previous international studies have used country-level data to analyze what influences student performance. These macro approaches cannot control for individual influences on a student’s performance. Country-level analyses are also limited because they can analyze only institutions that work at the country level, such as centralized exams, and not institutional features that work at lower levels, such as teachers’ influence over the curriculum.
The trouble with performing the analysis at the individual level is that there are no independent, individual observations for many variables. Within the TIMSS data set, the primary sampling unit (or PSU) is the school, not the student or classroom. Individual students who attend the same school may share some characteristics that are not captured by survey data; the individual observations are not wholly independent of one another. Also, several of the resource and institutional variables, such as the school’s decision-making responsibility and the existence of national examinations, are measured at the school or country level, further decreasing the independence of individual observations and reducing the number of independent observations on these variables. For instance, in comparing students in countries with centralized exams with those with no centralized exams, there were only 39 independent observations (the number of countries in the TIMSS sample). Unless the econometric method is adjusted to account for the lack of variation in some of the independent variables, the findings will appear more robust than they are. I use a statistical method known as robust linear regression with countries as strata and schools (or countries where appropriate) as the primary sampling unit to calculate appropriate standard errors for my findings and to adjust for this potential bias.
Results
This study deals with five main institutional features of a nation’s educational system: 1) centralized exams; 2) the distribution of decision-making power between schools and their governing bodies; 3) the level of influence that teachers and teacher unions have on school policy; 4) the distribution of decision-making power among levels of government, from local to national; and 5) the extent of competition from the private-school sector (see Figure 3).
Before we can test hypotheses, we must control for the effects of family background and the level of resources devoted to education. In this study, the educational level achieved by the students’ parents was strongly positively related to students’ educational performance. Students of parents who completed secondary school (or higher) achieved considerably more than students of parents who finished only primary school. The effect of a family’s having more books at home, shorthand for the educational and social background of the family, was even stronger than that of the highest educational level achieved by the parents. The perform-ance of students increases steadily as you go from students having fewer than 10 books at home to those having more than 200 books. Students scored 54 points better in math and 57 in science (on a range with an international average of 500 and an international standard deviation of 100) when they had more than 200 books at home compared with students who had fewer than 10. Just how big are these effects? Quite large. Consider that the average test-score difference between 7th- and 8th- graders is 40 points in math and 47 in science.
The results for school spending are consistent with the literature: no strong positive relationship exists between spending and student performance. When other factors are taken into account, higher spending and smaller class sizes seem to correspond to inferior mathematics and science results, though the overall effect is relatively small. Nevertheless, providing schools with the proper instructional materials and supplies seems to have a positive effect on performance. Students in schools whose principals reported that they do not suffer from inadequate instructional materials scored 7 points higher in math and science relative to students in schools whose principals reported that they were somewhat limited by inadequate materials. Students in schools with a great shortage of materials scored 6 points worse in math and 12 in science. Both of these findings should be interpreted with care, however: inadequate supplies may have led to poor achievement, or principals of low-achieving schools may tend to blame their poor achievement on inadequate supplies.
The quality of a nation’s teaching force also affects student performance and therefore must be controlled for. If teachers’ age is held constant, then more years of experience are positively related to student performance. But if teachers’ experience is held constant, teachers’ age is negatively related to student performance. This may reflect the positive effects of having more-experienced teachers combined with the negative effects of large age differences between teachers and students. Aging teachers may not understand a younger generation as well as younger teachers, and their motivation levels may be in decline as well.
Teachers who finished secondary school plus some teacher training added 16 points to students’ math scores and 24 to science (compared with teachers who did not complete secondary education). Having a bachelor’s degree added 11 points to students’ math scores, 12 points to science. Possessing a master’s or doctorate added 26 points in math, 32 in science. Overall, the effects of teachers’ educational levels were larger in science than in math.
Altogether, the relationship between school resources and student perform-ance is ambiguous. Per-pupil spending and smaller class size do not have positive effects, while having decent instructional materials and experienced, well-educated teachers do show positive effects.
Centralized exams. Of the 39 countries in this study, 15 have some kind of centralized exams, in the sense that an administrative body beyond the schooling level writes and administers the exams to all students. This can profoundly alter the incentive structure within the educational system by measuring student performance against an external standard, making performance comparable across classes and schools. It makes it easier to tell whether a given student’s poor performance is an exception within a class or whether the whole class is doing poorly relative to the country as a whole. In other words, centralized exams make it obvious whether it is the student or the teacher who is to blame. This reduces the teachers’ leeway and creates incentives to use resources more effectively. It makes the whole system transparent: parents can assess the performance of children, teachers, and schools; heads of schools can assess the performance of teachers; and the government and administration can assess the performance of different schools.
Centralized exams also alter the incentive structure for students by making their performance more transparent to employers and advanced educational institutions. Their rewards for learning thus should grow and become more visible. Without external assessments, students in a class looking to maximize their joint welfare will encourage one another not to study very hard. Centralized exams render this strategy futile. All in all, given this analysis, we should expect centralized exams to boost student performance.
And they seem to. All things being equal, students in countries with centralized exams scored 16 points higher in math and 11 points higher in science, although the science finding is not statistically significant due to the small number of countries in the sample (see Figure 3 for results). Furthermore, students in schools where external exams or standardized tests heavily influence the curriculum scored 4 points higher in math, though there appears to be no effect in science. This suggests that science tests may lend themselves less readily to standardization.
Decision-making between schools and their governing bodies. Some school systems are characterized by a high degree of centralization, where decisions on a wide range of issues are taken out of the schools’ hands. Other school systems are highly decentralized; most decisions are made at the local level. For instance, schools have a high degree of autonomy in the Netherlands, where 73 percent of decisions are made at the local level, according to the OECD. By contrast, Greece, Norway, and Portugal allow local school personnel to make fewer than 25 percent of the decisions. Here the question is, What is the division of decision-making powers between schools and the government in a country, and how do these divisions affect student achievement?
The effects of granting more autonomy to schools are hard to predict. On the one hand, schools need a high degree of autonomy in order to respond to the demands of parents–a prerequisite for competition. Also, the educators within a school should have more knowledge of effective teaching strategies for their students than central administrators. Likewise, individual teachers should know what are the best textbooks and supplies for their students. Heads of schools should also have more knowledge than central administrators about which teachers to hire and who deserves promotion or a raise in salary.
On the other hand, enhanced autonomy makes it easier for school personnel to reduce their workload, unless they are subject to external monitoring and evaluation. The more flexibility a school has, the more important it is to have external standards and assessments. Putting decisions on the size of the school budget in the hands of school personnel might also harm performance; it is clearly in their interest to garner additional funds for themselves or resources that lighten their workload.
We should expect, then, that giving schools the power to set their own budgets, performance goals, and standards of what to teach will have an adverse impact on student achievement. Such powers are probably best left to central authorities. By contrast, decisions on how to meet the goals and standards, such as the choice of teaching techniques and the purchase of supplies, are best left to schools, as long as an effective monitoring and assessment program is in place.
The first variable I analyze is whether having a centrally designed curriculum and a centralized list of approved textbooks is conducive to student performance. These are essentially decisions about what schools are expected to cover. Students in countries with centralized curricula scored 11 points better in math, 6 in science. Students in countries with centralized textbook approval scored 10 points better in math, 6 in science. These findings are not statistically significant due to the small number of independent observations, but they are nonetheless suggestive.
Students in schools that had primary responsibility for setting the school budget scored 6 points worse in math and 3 in science (the science effect, however, is statistically insignificant). By contrast, giving schools autonomy in purchasing their supplies goes hand in hand with superior achievement. This is also true for decisions on hiring teachers. Students in schools that hire their own teachers scored 13 points higher in math, 5 in science. Students in schools that determine their own teacher salaries scored 11 points higher in math, 15 in science. Centralized decision-making on curriculum issues seems to prevent schools from seeking to reduce their workload and thus raises student achievement. Conversely, local control of teacher recruitment and compensation may allow schools to retain a more effective staff.
The influence of teachers. Within schools, the incentives that teachers face and their ability to influence the process also affect student achievement. Besides a student’s family, teachers probably have the greatest impact on student achievement. Since they cannot be easily monitored, they also have a great deal of freedom to pursue their teaching in whatever way they wish. Often they face conflicting interests. They clearly have a genuine interest in increasing their income at a given workload or decreasing their workload at a given income. Nonetheless, seeing their students learning also gives teachers pleasure, which encourages them to work harder no matter what their income. Furthermore, teachers who perform poorly may face negative consequences from their heads of school or from parents. The institutional setting will influence them to behave more in one way than another.
It is important to emphasize the difference between teachers acting individually and as part of a union, for these settings may have very different consequences for student achievement. When teachers act collectively, they are a potentially powerful political interest group; their sheer numbers give them voting power that politicians cannot ignore. The aim of teacher unions is to promote the interests of teachers and to defend them against the interests of other groups. The unions, therefore, will focus on the interests that are not advanced by other interest groups–mainly, increasing teachers’ pay and decreasing their workload. They can also exert collective bargaining power. In doing so, they will advance the interest of the median teacher, favoring a leveling of salary scales instead of differentiation by merit. Other things equal, strong teacher unions should promote behavior that is detrimental to student performance. Harvard University economist Caroline Hoxby’s research has shown that teacher unionization helps explain why schools can perform worse when given more resources. Hoxby found that teacher unions act to increase school resources but reduce the productivity with which these resources are used. The cumulative effect is a reduction in school productivity (the ratio of student performance to spending).
By contrast, when teachers act individually, their deep, personal knowledge of their students and the students’ needs may increase their effectiveness and thus outweigh their interest in decreasing their workload. The effect of teacher influence may also differ among decision-making areas. A high degree of teacher leeway in making decisions about which textbooks to buy should be conducive to student learning, since the teachers know best how to teach their students. But a high degree of influence in determining salary levels or the amount of subject matter to be covered should be detrimental to student performance. Altogether, the predicted effect of increasing the power of individual teachers is uncertain.
Here the results are similar to those obtained in the analysis of decision-making divisions between schools and administration. Students in schools whose principals reported that teachers had primary responsibility for determining the school budget scored 13 points worse in math, 5 in science. Likewise, students of teachers who reported that they had a lot of influence on the subject matter to be taught performed worse in science, while the effect in math was statistically insignificant.
By contrast, students scored 14 points better in math and 7 points better in science if teachers had primary responsibility for buying supplies. Teachers’ influence on the curriculum needs to be divided according to the way they exercise it. Students in schools where each teacher individually had a lot of influence on the curriculum performed 12 points better in math, 11 points in science. Teachers’ being able to choose their specific textbooks also has a positive effect in math. But in schools where teachers acting collectively as a union had a lot of influence over the curriculum, students performed 32 points worse in math, 18 points in science.
Overall, these findings on schools’ and teachers’ influence give a clear picture. If schools and teachers can use their intimate knowledge of their students to choose the best teaching method, then they can teach more effectively. But if they can use their influence, whether acting collectively or individually, to reduce their workload, then students’ learning opportunities will suffer.
Decision-making among levels of government. Different levels of government–local, intermediate, and national–have varying degrees of control over school systems worldwide. In the United States most educational decision-making and basically all fund allocation take place at the local level. In Germany the responsibility for planning and purchasing educational resources lies mainly with the intermediate level of government, namely state authorities. In Greece almost all decisions and basically all funding take place at the national level.
Here, again, the predicted incentives are mixed. Local levels of government are more accountable to parents and possess more knowledge about the needs of their particular communities. At the same time, local officials will have closer ties to school personnel, making school-based interest groups more influential. Local officials and school personnel might collude in determining the level and use of funds. Giving the national level more power can make collusion harder to achieve, but national officers do not have enough information to make wise decisions on allocating resources among various needs. A self-interested central administration will also find it easier to develop excessive bureaucracy and to divert resources to the central level. All in all, the intermediate level may be better positioned to govern the schools. It may be far enough away to make lobbying difficult, yet close enough to effectively monitor the schools.
In this study, students in countries where schools have more decision-making powers in managing personnel, planning, choosing their instructional methods, and deciding how to use resources scored significantly higher in science and higher in math (though the effect in math is statistically insignificant). If the percentage of decisions made at the school level increased by 10 percentage points, students scored 8 points higher in science. Students in countries where a larger share of decisions was made at the national level scored lower in both math and science (the effect in math, however, was statistically insignificant).
The level at which schools are funded also affects student performance. More responsibility for purchasing educational resources at the national and local levels appears to correspond to lower student achievement, at least in mathematics. Students performed considerably better when responsibility for purchasing educational resources resided at an intermediate level of government. This suggests that an authority that is close enough to local schools to understand their needs, yet far enough away to avoid collusion between local officials and school employees, is the best place to rest responsibility for funding education.
Competition from private schools. The level of competition that public schools face from private schools is another important institutional feature. The existence of more private schools gives parents who want to raise their children’s achievement the opportunity to choose whether to send them to a particular private school or to a public school. Because the loss of students to private schools may have negative repercussions for the heads of public schools, increased competition from private schools should have a positive effect on the efficiency of resource use in the public schools. The existence of private schools should also increase a country’s overall achievement level. The heads of private schools have clear monetary incentives to use resources in ways that maximize student performance–thereby giving more parents reasons to choose their schools. Therefore, the more privately managed educational institutions there are in a nation, the higher student performance should be.
The degree of competition from private schools varies greatly worldwide. The Netherlands has by far the highest share of students attending privately managed schools (76 percent), followed by the United Kingdom (36 percent) and South Korea (35 percent). However, fewer than 1 percent of Dutch schools are financially independent in the sense that they receive less than half of their core funding from government agencies. Countries with the largest shares of students attending financially independent private schools are Japan (24 percent), South Korea (18 percent), and the United States (16 percent). Australia, Austria, the Czech Republic, Denmark, France, Germany, Hungary, Iceland, Norway, Spain, and Sweden have virtually no independent private schools. Students in countries with larger shares of their enrollment in privately managed schools scored significantly higher in both math and science. If the share of enrollment in privately managed schools was 10 percentage points higher, students scored 6 points better in math, 5 in science. The effect was even larger when only those private institutions that were financially independent were considered.
The Netherlands and Belgium are the countries with by far the largest share of public funds going to private institutions (75 percent and 63 percent, respectively). Meanwhile, less than half a percent of public funding goes to private schools in Austria, Greece, Ireland, New Zealand, the Russian Federation, and the United States. Countries with a higher share of public-education spending going to private institutions performed better in math and science (though the effect in science is statistically insignificant). The effect was even stronger when only those expenditures were counted that went to independent private institutions receiving less than half of their core funding from government. If the share of public funds going to independent private schools rose by 1 percentage point, there was a 12 point increase in math achievement. This suggests that student performance is higher in educational systems where private schools take over resource allocation from public decision-makers.
Institutions Do Matter
Taken together, the effects of all these institutional variables add up to more than 210 points in math and 150 in science. In other words, a student who faced institutions that were all conducive to student performance would have scored more than 200 points higher in math than a student who faced institutions that were all detrimental to student performance. In short, institutional variation across countries explains far more of the variation in student test scores than do differences in the resources devoted to education.
More specifically, having centralized exams and a large private-schooling sector seems conducive to student performance. Generally, school autonomy seems to have a positive impact–but only when schools are given extensive decision-making powers over the purchase of supplies, the hiring and rewarding of teachers, and the choosing of instructional methods. Giving schools power over designing the curriculum syllabus, approving textbook lists, and determining the school budget seems to be detrimental to student performance. The effect of teachers’ influence seems to depend on how it is exercised. Students seem to benefit from their teachers’ having influence over the curriculum, but only when they act as individuals and not as part of a union. It appears to be better for students if teachers significantly influence the choice of supplies, but worse if they have a strong say in the amount of material to be covered.
The only difference between the results for math and science is that the effects of standardization seem to be more positive in math than in science. This shows up in the fact that centralized exams, curricula, and textbook approval have stronger effects in math than in science. One can speculate that mathematics is easier to standardize, whereas science may require more creativity and initiative on the part of teachers.
For education policy, the results of this study suggest that the crucial question is not one of providing more resources but of improving the institutional environment in which schools function. Spending more money within an institutional system that sets poor incentives will not improve student performance. An institutional system in which all the people involved have an incentive to improve student performance is the only alternative that promises positive effects.
Ludger Woessmann is a research associate at the Kiel Institute of World Economics in Kiel, Germany. To view his study in its entirety, log on to www.edmattersmore.org.
