In my final piece of this series arguing against the high-regulation approach to school choice, I’m going to discuss whether achievement test results are a reasonable proxy for school quality. Achievement tests are at the center of the high-regulation approach. They are used by regulators — whether authorizers, portfolio managers, or harbor masters — to identify good and bad schools, to determine whether they should be included as choice options, and to shape the goals schools should pursue.
There is no question that growth in student learning provides us with some useful information. The problem is that school quality is much broader than just test score results. I always understood that achievement tests were only a partial and imperfect indicator of school quality, but I used to believe that other aspects of school quality not captured by achievement tests were largely correlated with those test results. That is, I used to think that if a school raised scores it probably meant that students were safer, more students would graduate, more students would learn productive values, and more students would go on to become successful adults.
Unfortunately, the evidence is increasingly clear that test scores are only weakly correlated with all of these other desirable outcomes from schools. All you have to do is look at yesterday’s post. Schools that produce the largest achievement test gains are not necessarily the ones that produce higher graduation, or college-attendance rates. And sometimes schools with unimpressive achievement gains make significant contributions to attainment and annual earnings when students join the workforce. I used to think that this couldn’t be possible. All of these happy outcomes had to be aligned. They just aren’t.
If you are not persuaded by the evidence I reviewed yesterday on the disconnect between achievement results and other outcomes, I suggest you read an excellent book written by Nobel Prize winning economist James Heckman and his students called The Myth of Achievement Tests.
The problem is that the high-regulation approach needs achievement tests to be correlated with all of these other good outcomes. If they are going to pick the school choice winners and losers based on test scores, then test scores need to be strongly predictive of other things we care about. People have been very slow to accept the fact that test scores are only weakly correlated with later life outcomes because it would be so convenient if readily available and relatively inexpensive test scores could capture something as complex as school quality. The fact that they don’t throws a monkey wrench into the entire high-regulation machinery.
The reality is that the average low-income parent has more complete information about their kid’s school quality than does the highly-trained regulator armed only with test scores. When we wonder why parents are choosing schools that regulators and other distant experts deem to be “bad,” it is almost certainly because the parents know more about what is good and bad than do the experts.
The wrong response to recognizing that test scores fail to capture school quality sufficiently is to increase the set of high-stakes measures we collect. We can’t fix the limits of math and reading achievement tests by adding mandatory “grit” surveys or other measures. Even informed by a variety of measures, Chinese officials are no more effective in telling state-controlled banks how to allocate capital than portfolio managers are in determining how to allocate school options. Decentralized decision-making is simply better than central planning.
The school choice movement has to remember that choice is what makes this reform work, not the regulation. I’m perfectly willing to accept that some regulation is necessary and inevitable. And I’m willing to make compromises to get programs adopted. But the cardinal sin of the high-regulation school choice folks is that they believe that heavy regulation is the ideal and should be the starting point for political compromises.
– Jay P. Greene
