This year, Bengt Holmstrom received the Nobel Prize in Economics for his work on contract theory. Holmstrom does not work on education per se, but he has spent much of his career writing about the use of incentive contracts in settings where worker actions are hard to observe and worker output is difficult to quantify, e.g. educators in schools. We live in an information age, and for decades we have seen organizations employ new measurement technologies and create new performance metrics for workers, teams, or organizations. Many non-economists assume that economists believe that more performance metrics and more performance pay always mean more efficiency. However, Holmstrom’s work teaches us that the world is not so simple. The details of performance pay schemes matter, and new incentive schemes built around new performance metrics do not always improve performance.
These lessons are important for education researchers because proponents of assessment-based incentive systems in education have often argued that policymakers should always favor personnel policies that link educator pay and job security to objective performance measures derived from data on student test score outcomes. Further, these advocates have also asserted that getting the details of these systems correct is a secondary concern. For them, the first order of business is to find some politically acceptable way to hold public educators accountable for student learning outcomes.
Yet, roughly three decades after education policymakers began this push for assessment-based incentive systems, the empirical literature on the effects of these systems suggests that education researchers and policymakers should have been more concerned about the subtle issues that Holmstrom explores. In a review of incentive systems for educators, I argue that serious problems arose during the No Child Left Behind (NCLB) years because policymakers chose to use simple proficiency counts as performance metrics.[1] This design decision runs counter to the informativeness principle in Holmstrom’s work.[2] Holmstrom shows that optimal incentive contracts employ all information that sharpens inferences about the actual performance of workers. NCLB could have and should have required states to create performance metrics for educators that contained careful adjustments for all observable factors that impact student learning but are beyond the control of educators.
Since the passage of NCLB, the empirical literature on the statistical properties of competing metrics of teacher and school performance has grown substantially, but Holmstrom and Milgrom teach us that the statistical properties of performance metrics should not be the only concerns of researchers and policymakers who design assessment-based incentive systems.[3] Holmstrom and Milgrom focus on the alignment between performance metrics and true performance.[4] If a performance metric is well-aligned with true performance, then the actions that generate better measured performance also generate proportionate improvements in true performance. If a performance metric exhibits poor alignment, schemes that attach incentives to it induce agents to waste effort on tasks that improve measured performance without improving actual performance.
As an example, consider a high-stakes testing program that induces all schools to devote three weeks each spring to test-prep sessions that build few lasting skills but do improve scores on the high-stakes assessments. Since all schools devote three weeks to test prep, score differences among students and schools should be highly correlated with true achievement differences. Yet, the ex post correlation between test scores and true achievement should not be a primary concern for policymakers. Their key concern should be that the introduction of this high-stakes testing program caused schools to waste three weeks of class time on activities that build few lasting skills.
Holmstrom’s focus on alignment points us to the possibility that, in some work environments, performance pay schemes may not be helpful. We read a great deal about the big data revolution, but even today, most people do not work under high-powered incentive contracts linked to objective performance metrics. Holmstrom and Milgrom help us understand this reality because they point out that, in some environments, it may be prohibitively costly to gather the data required to create well-aligned performance metrics.[5] In these settings, the best approach may be to adopt hiring procedures that identify workers who will perform well in order to satisfy their own personal norms and the norms espoused by the organization, and then pay these workers a fixed salary.
When applied to education, this observation implies that the value of assessment-based incentive schemes is determined, in part, by what policymakers are able to accomplish without them. In the high-stakes testing scenario above, the social cost of the three weeks of test prep is determined by the quality of teaching that would have existed during those three weeks in the absence of assessment-based incentives. In schools where personal and professional norms among teachers are strong, these costs could be great. In schools where little excellent teaching takes place, these costs are less important.
Holmstrom’s work also helps us think about incentives mechanisms other than explicit performance contracts. Holmstrom explains how the benefits of a strong reputation shape the actions of workers, given different assumptions about how their talents evolve over time and what the public is able to observe about them.[6] In recent years, we have seen debates over whether or not school systems should publicly release performance metrics for individual teachers, and if so, what kind. Holmstrom’s work teaches us that the creation of public report cards for individual teachers could be beneficial or harmful. Policymakers again need to carefully consider what actions educators would take in response to various report card systems.
Holmstrom’s work fleshes out how differences in work environments shape differences in optimal incentive schemes, but most research on incentive systems for educators has not bothered to ask how unique aspects of schools as work environments shape optimal incentive provision for educators. Further, no existing economics research on schools has explored an additional set of questions that Holmstrom and Milgrom examine.[7] How does the structure of information in schools, i.e. the school’s capacity to measure different types of activities and outputs, shape how duties and responsibilities should be allocated among different employees within a school? Using Holmstrom and Milgrom’s language, what is the optimal way to design jobs in schools? Should teachers teach one subject or many? Should administrators develop lesson plans and instruct teachers to follow them, or should they only monitor and evaluate lesson plans that teachers develop on their own? I do not know the answers to these questions, but economists should be asking them, and the tools in Holmstrom and Milgrom provide a useful framework for such explorations.[8]
Economists have conducted numerous ex post evaluations of various incentive schemes for educators, but few economists have explored how policymakers should design incentive systems for educators. This is ironic because economists have much to teach other education researchers and policymakers about the tricky business of creating incentive systems. Holmstrom taught economists many of these lessons. Going forward, economists should do more to make sure that his insights inform the broader community of scholars who study education policy.
Many voices in current education policy debates are advocating an end to all forms of assessment-based incentives. These reactions are understandable given the evidence gathered over the last two decades or more, but we do not yet know how educators would respond to well-designed incentive schemes that incorporate the theoretical insights of Holmstrom and others, as well as the empirical insights produced by decades of research on the use of incentive systems in schools, government agencies, and businesses. We do know that, if policymakers continue to ignore these insights, those who oppose all forms of assessment-based incentives will continue to gather evidence that lends support to their cause.
Derek Neal is professor in the department of economics and member of the Committee on Education at the University of Chicago.
_____
[1] Derek Neal, “The Design of Performance Pay in Education,” Handbook of the Economics of Education 4 (2011): 495-550.
[2] Bengt Holmstrom, “Moral Hazard and Observability,” The Bell Journal of Economics (1979): 74-91.
[3] Bengt Holmstrom and Paul Milgrom, “Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design,” Journal of Law, Economics, & Organization 7 (1991): 24-52.
[4] Ibid.
[5] Ibid.
[6] Bengt Holmstrom, “Managerial Incentive Problems: A Dynamic Perspective,”The Review of Economic Studies 66.1 (1999): 169-182.
[7] Bengt Holmstrom and Paul Milgrom, “Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design,” Journal of Law, Economics, & Organization 7 (1991): 24-52.
[8] Ibid.
