Commentary on “Great Teaching:Measuring its effects on students’ future earnings” By Raj Chetty, John N. Friedman and Jonah E. Rockoff
The new study by Raj Chetty, John Friedman, and Jonah Rockoff asks whether high-value-added teachers (i.e., teachers who raise student test scores) also have positive longer-term impacts on students, as reflected in college attendance, earnings, avoiding teenage pregnancy, and the quality of the neighborhood in which they reside as adults. As a step on the way, the researchers investigate whether such teachers have been properly identified, that is, are the teachers who are producing larger achievement gains from year to year, according to value-added models, actually responsible for those gains? The paper contains valuable evidence indicating that the answer is yes. First, the authors obtain data on family background from federal tax returns not normally available to researchers. This allows them to measure family characteristics (such as parental income) not typically controlled for when teacher value-added is estimated. If introducing such factors reduces the explanatory power of teacher value-added, it is an indication that the value-added estimate was inflated, and that part of what had been attributed to the teacher was in fact due to favorable family circumstances. The study authors find that including such controls does not detract from the explanatory power of estimated value-added.
The authors also investigate whether high-value-added teachers have benefited by being assigned students who would have made greater gains on standardized tests for unobserved reasons (such as family factors that cannot be gleaned even from tax returns). This is normally difficult to do, given the possible influences on the way students are assigned to teachers. The report succeeds by focusing on average test gains in grades within schools where mean value-added within a grade has been affected by the movement of teachers in and out of the grade. What matters for this analysis is not which student was assigned to which teacher within the grade, but how the movement of teachers has altered the quality of teaching in that grade as a whole. It turns out that subsequent gains within these grades are close to those what would be expected from the change in mean teacher value-added. Provided the movement of teachers in and out of a grade has not changed the makeup of students enrolled in that grade, this finding supports the conclusion that measured value-added of teachers is an unbiased predictor of future test-score gains, as there appears to be no other explanation for the resulting improvement in test scores.
When the authors examine the association between teacher value-added and outcomes in young adulthood, however, for the most part they do not undertake the same tests to ensure that these associations are not artifacts of the way students are sorted among teachers. They do not introduce controls from tax returns to see whether the explanatory power of teacher value-added for later earnings, college attendance, and other factors, falls. Nor, with the exception of college attendance, do they test for the influence of unobservable factors in the manner just described.
The omission of such tests undercuts their claim to have demonstrated that high-value-added teachers contribute to better long-term outcomes. Without the same rigorous tests, we cannot be sure that the observed association between teacher value-added and long-term outcomes was not the result of other factors (for example, efforts made by parents with the strongest parenting skills to ensure their children were assigned to the most effective instructors). It is not enough to show that omitted family characteristics have not been confounded with value-added as a predictor of future test-score gains. The factors that shape test performance are not necessarily those that influence future earnings or the avoidance of a teenage pregnancy. Character education and the values parents impart to their offspring are likely to matter for the latter in ways that they do not for cognitive functioning.
In short, the authors provide a persuasive answer to the question: does a high-value-added teacher actually raise subsequent test scores? They have not so far provided equally persuasive evidence answering the question: does a high-value-added teacher improve subsequent life outcomes?
Dale Ballou is associate professor of public policy and education at Vanderbilt University and associate director of the National Center on Performance Incentives.
Return to “Great Teaching” (Summer 2012)
