In the process of writing our recent paper on federal education policy, I spent time re-reading the academic research on school accountability in general and No Child Left Behind in particular. What struck me upon re-reading it was the disconnect between what the research says about the effects of school accountability–the literature tends to find it has small but significant impacts–and the national conversation about it.
As Congress considers legislation to reauthorize NCLB, I reached out to Eric Hanushek, the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University, to talk about his research on education and school accountability. Hanushek is a leader in the development of economic analysis of educational issues and has authored numerous, highly cited studies on the effects of class size reduction, high stakes accountability, the assessment of teacher quality, and other education related topics. His recent work shows that the quality of education is closely related to state and national economic growth.
What follows is lightly edited transcript of our conversation.
Chad Aldeman: You have a new NBER paper attempting to quantify what many state policymakers have long suspected—that differences in human capital matter for their state’s economy. Can you summarize what you found?
Eric Hanushek: The work that I’ve done with two colleagues in Germany—Jens Ruhose and Ludger Woessmann—has focused on finding out whether state human capital policies determine income differences that we see in the United States. When I got into this project I was surprised to see how wide the variation in income is across states—for example, Connecticut has twice the GDP per capita that West Virginia does.
What we did in this work was to go beyond the simplistic notion that human capital is best measured by school attainment and tried to include the quality of learning, or achievement, that people possess in different states. This turns out to be a fairly difficult problem because we know where workers are now but many workers were not educated in the state they’re working in now, or the country they were educated in, in the case of international immigrants. So we had an elaborate project that involved trying to trace workers in every state back to where they were educated and the quality of education in the place they were educated in. And then we looked at the differences in income across states based on where people were educated.
That’s an elaborate lead-up to a relatively straightforward summary, which is that perhaps one-third of the difference in incomes across states that we see today can be attributed to human capital differences in the workers of each state. Of the human capital differences, roughly half come from differences in school attainment, and half of it comes from differences in the quality of learning. (You can think of that as test scores, or achievement differences among the population, representing about half of the gap.)
Aldeman: When you say school attainment, you mean total years of schooling and any degrees they earned?
Hanushek: We didn’t actually look at degrees, we put it all in years of schooling, but that’s what we mean by attainment here.
Aldeman: How do these differences across states compare with any differences across countries?
Hanushek: Internationally, it’s about the same amount of explanation of income differences, maybe a little bit less. But internationally, the prime determinant is actually achievement differences. Internationally, years of schooling has very little value in measuring the income differences across countries, because the amount learned and the quality of schooling is so different across countries.
Aldeman: I’d like to hear more about that. A lot of policy focuses on years of schooling and particularly on completing degrees, but your work suggests that is somewhat insufficient. That is, a simple count of the number of years of school someone has attended leaves out the quality of that education and what the person actually knows and can do. Am I summarizing that correctly?
Hanushek: That’s right. The simple phrase that you might use to explain this is “schooling is not learning.” It’s part of it, but lots of other things go into learning, and there are big quality differences. That’s why so much of policy is aimed at changing the quality of schooling. We know that, if you look at NAEP scores, there’s almost a one standard deviation difference in performance between the top state and the bottom state over time. Just looking at years of schooling is a very imprecise measure both internationally and in the U.S.
Aldeman: What kind of implications does that have for policymakers, in terms of what types of incentives they should be thinking about, or other types of accountability they should be thinking of for schools?
Hanushek: I think part of it is that policymakers tend to focus too much on high school completion or college-going rates. They do that in part because it’s easily measured, and easier to measure than performance, but in reality the issues are all about quality. The quality of learning and cognitive skills of nations determines almost entirely the difference in growth rates across countries. In the long run, the economic well-being of countries depends upon the quality of their workers. And we’re not doing too well in the U.S. as a nation. From my standpoint, we ought to give a lot more attention to quality issues.
Aldeman: Is that true in both K-12 and higher education? Are there any barriers or cutpoints that matter here, or is what you’re saying true across the board?
Hanushek: I think it’s true across the board. Ludger Woessmann and I actually have a new book on this in the last couple months called The Knowledge Capital of Nations. In the book we look in some detail at the impact of achievement differences across nations, and school attainment across nations. In the special case of higher education, it turns out that once you account for what people know—for this we use measures like the international PISA test that compares learning across nations—years of schooling account for none of the differences in economic growth. If you look at higher education, that doesn’t add anything either.
Now, this shouldn’t be misinterpreted. I’m not arguing against people going on to additional schooling or on to college. It would be against my employment at Stanford University to argue against getting more education! But what it does say is that students who come into college with more knowledge subsequently learn more knowledge. In simpler terms, early learning has a cascading effect that affects what you know at later levels of learning and where you’re going to be economically. This is true both for individuals and entire nations.
Aldeman: What about those who would respond to some of your work and say you’re using standardized test scores that aren’t at the individual level, that aren’t predictive of larger economic trends, and that you’re sort of extrapolating here? What would you respond to that line of argument?
Hanushek: Well, I’d say they’re wrong! Lots of people want to take attention away from achievement and say, “Well, it’s just standardized tests.” But it turns out that standardized tests are pretty good measures of overall learning, particularly at the country level but also at the individual level. When we looked at achievement tests in math and reading and problem solving from a recent OECD survey, we find that these achievement differences, measured by tests of workers at all ages, are quite predictive of the earnings of individuals.
Moreover, if you look across countries—we look across 20-plus countries in our work—the United States rewards achievement more than any other country in the world. In other words, achievement pays off more handsomely in the United States than it does in any other country. I think that just reflects the fact that we are in a knowledge economy, and in a knowledge economy what pays off is knowledge.
Aldeman: I want to turn now to your work on school accountability. You’ve done research on accountability policies both here in the United States and abroad. What do you think are the biggest lessons from your work on accountability?
Hanushek: The biggest lesson is that you get a much larger impact of accountability if there are rewards and punishments and incentives that go along with performance. In other words, just putting up a scorecard, or a report card in the case of schools, that says how well they’re doing has much less impact than saying that they’ll be rewarded for doing well.
The second lesson is that you should pay attention to the value-added of schools and teachers and the people in those schools when you’re making these rewards. You shouldn’t just pay attention to the overall achievement level because that reflects a lot of things that are outside the school, the families, the neighborhoods, the peers, and in order to get the right incentives you have to align the incentives with the outcomes that can be attributed to the individuals and institutions like schools.
The third thing I’d point out from other research that’s outside the field of accountability is that No Child Left Behind is in some sense backwards—what No Child Left Behind says is that states decide what should be done, what the standards and the testing should be, and the federal government decides how it’s going to be done, because if the school is failing there are written regulations about how the education system should change to do better. This is just about 180 degrees backwards. The federal government is in a better position to make broad statements about what we expect to achieve and how it should be measured, and the local districts and local schools should be the ones making the decisions about how to adjust and what to adjust if they’re not achieving he results they want.
In other words, I believe in having a lot more autonomy for schools but holding them responsible for outcomes.
Aldeman: Your work on accountability is often framed in effect sizes and standard deviations, which I think people have a hard time understanding, but then you also turn it into larger economic value arguments. Can you explain why you do that and what are the takeaways from that analysis?
Hanushek: The interest in education, I think in large part, is that people believe education is related to economic outcomes. So it’s useful to show, in dollar terms, what it means to change achievement levels. And making those calculations is possible because we know how achievement relates to individual earnings and the growth in national income. It’s possible to translate improvement in schools, which leads to improvement in student achievement, into economic terms. What you find when you did this is that the rewards to better schools are enormous. In my opinion, we should be willing to consider much larger changes in what we’re doing because the rewards are so large.
– Chad Aldeman
This first appeared on Ahead of the Heard.
