In our recent article for Education Next, “Choosing the Right Growth Measure,” we laid out an argument for why we believe a proportional growth measure that levels the playing field between advantaged and disadvantaged schools (represented in the article by a two-step value-added model) is the best choice for use in state and district accountability systems. This argument is based on three key points. First, by leveling the playing field between schools, a proportional growth measure is best suited to elicit optimal effort from all educational personnel, regardless of the school in which they serve. Second, proportional growth measures provide useful system-wide instructional signals and allow underperforming schools to seek out high-performing model schools working in similar educational contexts. Finally, proportional growth measures are less likely to provide performance signals that would further damage the ability of schools serving disadvantaged communities to recruit high quality teachers and school leaders.
However, since the article was published, it has been pointed out (rightly) by Morgan Polikoff of USC that the U.S. Department of Education’s (USDOE) waiver guidelines do not allow for a fully-controlled proportional model to be used by state education agencies as part of their school accountability systems. For example, page 15 of USDOE’s 2009 non-regulatory guidance document states that, “If the State proposes a regression or multivariate/multi-level model, the independent variables may not include race/ethnicity, socioeconomic status, school AYP status, or any other non-academic covariate.” In fact, given this language, many other commonly-used student growth measures would also be excluded by this guideline, including the one-step fixed effects model discussed in our original article.
Given these USDOE restrictions, one might think that the only option is for states and districts to choose a sparse growth model, such as median student growth percentiles (SGPs), that only controls for past student exam scores in estimating student growth. Our response to this is two-fold.
First, from a policy perspective we maintain our argument that a fully-controlled proportional growth model is optimal for school accountability purposes and that the USDOE’s guidelines constitute bad public policy and should be amended. Although well-intentioned given the desire to maintain “the same grade-level academic standards for all students,” the economic forces described above and in our original article suggest that adherence to these guidelines is likely to hurt the disadvantaged students that the USDOE is trying to help. Moreover, it is important to note that none of the growth models under consideration are designed to set student-level academic targets, a goal that is better served using proficiency benchmarks. Furthermore, all growth models create different predictions for different students. Even with a sparse growth model such as SGPs, a student who is eligible for the free lunch program is likely to have a lower growth “target” than a non-eligible student because, on average, students who are eligible for the free lunch program also have lower prior exam scores. Hence, our policy recommendation is that results from a fully-controlled proportional growth model be reported alongside performance against proficiency targets so that no useful information is lost in the accountability system.
Fortunately, in the absence of a policy change allowing for a fully-controlled model, there may be another option. Specifically, in Missouri, a school-level two-step growth model that controls only for prior test scores at the student and school levels comes quite close to achieving proportionality. For example, consider the following figure that compares how the growth estimates from four different models are related to the school share of students who are eligible for free or reduced price lunches.
The first three charts in the table replicate Figure 1 from our Education Next article, while the fourth presents the same chart where the results from the modified two-step model are used as the school value-added measures. As can be seen, the relationship between the modified two-step estimates and the percent of students in the school who are eligible for free or reduced-price lunches is not completely eliminated as it is in the fully-controlled two-step model (the -0.03 correlation in the third chart is not statistically significant). However, it is much closer to proportional than either the one-step value-added or median SGP models. Moreover, at the district-level, this correlation gets even smaller and is no longer statistically significant. Given these results, a modified two-step value-added model may be a workable compromise for states and districts that would like to implement a proportional growth model but also seek to comply with the federal guidelines on growth measures.
—Mark Ehlert, Cory Koedel, Eric Parsons, and Michael Podgursky
Mark Ehlert is research associate professor of economics at the University of Missouri, Columbia, where Cory Koedel is assistant professor of economics, Eric Parsons is research assistant professor of economics, and Michael Podgursky is professor of economics.
