The current administration has vowed to leave education matters up to the states, continuing a movement started with the Every Student Succeeds Act (ESSA), which dramatically limited the federal government’s role in school accountability. While greater local control certainly has some benefits, it risks exacerbating the massive disparities in educational performance across states that already exists.
In 2015, there was almost a 30 percentile point difference in 4th grade math proficiency rates between the top and bottom states, only some of which can be explained by state-level social and economic factors. The massive disparity in progress is perhaps even more disturbing. Between 2003 and 2015, student proficiency rates grew by over 40 percent in some states, while remaining flat or even declining in other states.
The Department of Education (DoED) should take steps to highlight these disparities by identifying the lowest performing states and providing information on the status and progress of all states on a variety of educational metrics. The DoED might also provide modest funding and technical assistance to help demographically similar states work together to improve their public education systems.
On the campaign trail, President Trump often called for giving more discretion over education policy to states and localities, critiquing Common Core and what he viewed as other instances of federal overreach. In her recent confirmation hearing, President Trump’s nominee for Education Secretary—Betsy DeVos—repeatedly argued for leaving education matters up to the states.
And this desire for local control is not limited to the current administration. In 2015, Congress passed the Every Student Succeeds Act (ESSA) with strong bipartisan support. This legislation replaced the No Child Left Behind (NCLB) system of school accountability with a more narrowly tailored and flexible approach to school reform. Instead of requiring all schools to meet annual performance targets, ESSA requires states to focus on a small set of low-performing schools and gives them considerable latitude to design the interventions they deem appropriate.
In discussing ESSA, chair of the Senate Education Committee Lamar Alexander claimed, “The department was in effect acting as a national school board for the 42 states with waivers—100,000 schools. The states were doing fine until the federal government stuck its nose into it…So it was important to get the balls back in the hands of the people who really should have it.”[i]
But the evidence suggests that not all states are doing fine. Indeed, there are massive disparities across states in terms of current student performance, and these differences are not merely a factor of the social and economic conditions in the state. All states have been actively engaged in efforts to turnaround failing schools, but the effectiveness of such efforts has varied dramatically across jurisdictions.
Public education will (and should) always be driven predominantly by local actors—teachers, administrators, school board members, and state legislators. Even under NCLB, states and districts had a mostly unfettered ability to run schools as they saw fit. But with autonomy comes the potential for greater disparity, as more capable, focused, and well-resourced states pull even further ahead of those with less capacity, fewer resources, and greater political dysfunction.
The Department of Education (DoED) should take steps to highlight these disparities by identifying the lowest performing states and providing information on the status and progress of all states on a variety of educational metrics. The DoED might also provide modest funding and technical assistance to help demographically similar states work together to improve their public education systems.
Are states doing fine on their own?
Consider student achievement as measured by the National Assessment of Education Progress. While 4th and 8th graders as a whole have made modest progress in math and reading since the passage of NCLB, these improvements have flattened in recent years. And, more importantly, there are vast differences by state. Figure 1 shows a scatterplot of proficiency rates in 4th grade reading and 8th grade math as an example. At the top, states like Massachusetts boast nearly 50 percent of students meeting the rigorous NAEP proficiency standard. At the bottom, we see cases like New Mexico where less than 23 percent of 4th graders were proficient in reading and Louisiana and Alabama where only 18 percent of 8th graders demonstrated proficiency in mathematics.
Are these differences we saw above merely a reflection of poverty or other factors beyond the immediate control of the state? No. In a 2015 report, Matthew Chingos at the Urban Institute finds that demographically similar students vary dramatically in their NAEP scores depending on which state they live in.[ii] At the state level, one can calculate the proficiency rate one would expect to see in a state given factors such as state size, population density, median household income, and unemployment rates. Figure 2 plots these predicted proficiency rates against a state’s actual proficiency rates.[iii] The 45-degree line reflects the point at which a state’s actual proficiency exactly matches its predicted proficiency. States above the line are doing better than expected based on their social and economic conditions; states below the line are doing worse than predicted. We see, for example, that Massachusetts is not only a high-achieving state, but is scoring substantially higher than what one would expect even after accounting for the relative affluence of the state. At the other end of the achievement spectrum, despite its low overall performance, Mississippi is “beating the odds” given its extremely high levels of poverty. In contrast, Maryland, Michigan, and West Virginia are all scoring significantly lower than predicted based on the social and economic conditions in these states.
How about growth over time? Perhaps states are steadily improving despite their low overall proficiency rates? Figure 3 shows the change in proficiency rates between 2003 and 2015 alongside the 2015 proficiency rate. The figure is split into four groups. The states in the top right quadrant are those who experienced above average growth and above average performance. This set of highly successful states includes Kentucky, Rhode Island, and Massachusetts, among others. States in the bottom left quadrant experienced below average growth and below average proficiency. These states—including Michigan, West Virginia, South Carolina, and Arkansas—are clearly struggling.
Have state- and district-led reform efforts been successful?
In recent years, there have been a series of state- and district-led initiatives to turn around failing schools. How have these efforts fared? Research to date confirms that turnaround efforts result in dramatically different outcomes depending on the context. A newly released study finds that, on average, schools implementing federally funded but locally controlled interventions realized no achievement gains relative to comparison schools.[iv] However, there is good evidence that school turnarounds in Massachusetts have been quite successful, perhaps not surprising given the state’s well-regarded accountability system, its generous support of public education, and its highly professional state department of education.[v] On the other hand, turnaround efforts in Michigan, North Carolina, and Rhode Island have been mostly ineffective according to the latest research.[vi]
Even the most comprehensive turnaround efforts show mixed effects. As part of the state’s Race to the Top grant, Tennessee implemented a number of school turnarounds, all of which involved a change in school leadership and a substantial turnover of teachers. But only a modest subset of turnarounds—those led by large districts and that involved substantial additional resources—experienced any significant achievement gains.[vii]
Many states have used external operators—mostly charter management organizations (CMOs)—to manage failing schools. In New Orleans, this approach has been quite successful according to recent research.[viii] On the other hand, the same approach had no positive impact on student performance when implemented in Philadelphia or Tennessee.[ix]
What explains the notable lack of progress in some states? Are the leaders in those states lazy or actively trying to stand in the way of improving public schools? Probably not.
Many actors play a role in forming and implementing education policy at the state level—the State Board of Education, the state legislators, teacher associations, governors, and officials in the Department of Education. Each of these actors have their own set of interests and priorities. In the absence of a strong external force, there may not be an incentive for these actors to work together and focus on issues directly relevant to academic achievement.
Consider a few of the “education” issues that occupied state legislatures last year. In 2016, nineteen states considered legislation to restrict school bathroom access on the basis of one’s sex assigned at birth. While North Carolina is the only state to have enacted such legislation thus far, the issue of bathroom use by transgender students occupied considerable time in many state houses.[x] For the past several years, states have passed legislation intended to separate themselves from Common Core assessments. In addition to defunding the assessments and issuing requests for newly designed exams, some states have “creatively reinterpreted” disappointing results, papering over the glaring skill deficits the exams were intended to highlight.[xi] Teacher associations, for their part, continue to push back against teacher evaluation systems that focus on objective measures of student achievement and provide any meaningful differentiation between teachers.
While all states pay lip service to the importance of education, budget allocations often do not match the rhetoric. In fact, a recent report by the Center on Budget and Policy Priorities finds that most states are spending less on K-12 education than before the Great Recession.[xii] With the pressure of federal accountability removed by the passage of ESSA, states may feel even less need to adequately fund public education.
What can the federal Department of Education do?
The dramatic variation in student performance across states raises the concern that many children will suffer in coming years as states and districts assume greater responsibility for monitoring and intervening in failing schools.
Given the current political climate and the limited discretionary resources available to the Secretary of Education, it is unrealistic to imagine that the DoED would play an active role in state-level school reform efforts.
However, the DoED can serve an important role simply by highlighting some of these disparities. The department can identify the lowest performing states, and provide biennial information on the progress of these states, both in terms of what reforms they have implemented as well as the change in various outcomes. Indeed, the DoED could provide easily digestible measures of academic performance for all states.
This type of “name and shame” strategy has a long tradition. As Justice Louis Brandeis once remarked, “Publicity is justly commended as a remedy for social and industrial diseases. Sunlight is said to be the best of disinfectants; electric light the most efficient policeman.” And provision of information on state-level performance is well within the department’s broad federal oversight role.
Table 1 lists the bottom 10 states based on several difference measures of academic performance. In all cases, the data comes from the National Assessment of Educational Progress (NAEP).[xiii] I present these not as a definitive assessment of which states should be on a federal watch list. There are many different criteria one might use to judge a state’s academic performance and then many technical nuances about how to create various metrics. The table below provides one initial attempt at creating such a list.
The proficiency rates in columns 1 and 2 are based on 4th and 8th grade math and reading exams in 2015. The list in column 3 comes from the Urban Institute report mentioned above. It ranks states based on 2013 NAEP scale scores, which means that it leverages the performance of extremely low and extremely high scoring students more than the proficiency rate measures. Column 4 ranks states according to the improvement (or lack thereof) made in proficiency rates from 2003 to 2015. Appendix Table 1 provides the underlying scores for each state on each of the measures. (Please see the downloadable PDF for the Appendix.)
While the states vary somewhat across the columns, it is clear that several states stand out as having particularly weak performance as measured by the NAEP. Several poor, and historically low-performing states such as Alabama and West Virginia appear multiple times. But we also see historically higher-achieving states that have made little progress over the past decade, including Connecticut. My home state, Michigan, makes the bottom 10 list on all 4 measures, and ranks dead last in terms of proficiency growth since 2003.
Conclusion
In states that have been successful in improving public education, one or more groups have brought the different factions together and/or provided the political muscle to break through the roadblocks that often stymie reform. In some cases, this has been the business community; in other cases, a strong governor. In states that, for one reason or another, do not seem to be able to focus on education policy designed to actually improve academic achievement, the DoED can serve this role.
By shining a spotlight on states with particularly low student performance, the department can bring attention to the struggles facing public education in these states. The literature on school accountability suggests that this visibility alone can put pressure on educational actors to reform.[xiv] The hope is that a state’s presence on the list galvanizes various stakeholders to come together to work on a common objective.
If DoED is inclined to go beyond merely providing information but still stay well within the umbrella of state and local control, it might look to support interstate compacts by which small groups of states with similar demographics work together with money and TA from ED to improve.[xv]
We shouldn’t kid ourselves. No watch list—no matter how well designed or implemented—will lead to dramatic changes overnight. Massachusetts did not become a leader in public K-12 education with any single action, but rather through a concerted effort by many actors over a number of years.[xvi] But this is one step that the DoED could take to help children who will be left behind as states assume even more control of public education.
— Brian A. Jacob
Brian Jacob is a Nonresident Senior Fellow at the Brookings Institution and the Walter H. Annenberg Professor of Education Policy, Professor of Economics, and Professor of Education at the University of Michigan.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
Notes:
[i] http://www.chalkbeat.org/posts/tn/2016/04/25/sen-lamar-alexander-on-the-nations-new-education-law-and-how-it-could-shape-tennessee-schools/
[ii] http://www.urban.org/research/publication/breaking-curve-promises-and-pitfalls-using-naep-data-assess-state-role-student-achievement
[iii] Specifically, I regress the 2015 proficiency rate in the state on the 2015 state poverty rate, percent rural, percent female, a cubic in the unemployment rate, percent white, log(public school enrollment) and across zip code standard deviation of median household income. The residuals are shown in columns 3 and 4 of Table 1.
[iv] https://ies.ed.gov/ncee/pubs/20174013/pdf/20174013.pdf
[v] Papay, John P. and Hannon, Molly (2016). “The Effect of School Turnaround Strategies in Massachusetts.” Working paper, Brown University.Schueler, B. E., Goodman, J., & Deming, D. J. (2016). Can States Take Over and Turn Around School Districts? Evidence from Lawrence, Massachusetts (Working Paper No. 21895). National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w21895
[vi] Hemelt, Steve and Jacob, Brian (2017). “Differentiated Accountability and Education Production: Evidence from NCLB Waivers.” Working paper. University of Michigan. Heissel, J. A., & Ladd, H. F. (2016). School Turnaround in North Carolina: A Regression Discontinuity Analysis. CALDER Working Paper. Retrieved from http://www.caldercenter.org/sites/default/files/WP%20156.pdf Dougherty, S.M., & Weiner, J. (2016). “The Rhode to Turnaround? The Impact of Waivers to No Child Left Behind on School Performance.” Working paper.
[vii] Zimmer, R., Henry, G. T., & Kho, A. (2016, May). The Role of Governance and Management in School Turnaround Policies: The Case of Tennessee’s Achievement School District and iZones. Presented at the Association of Education Finance and Policy, Denver, Colorado. Retrieved from https://aefpweb.org/sites/default/files/webform/41/ASD_iZone%20Impact%20Paper%20conference%20online%20version.pdf
[viii] Ruble, W. (2015). The Effect of Contracting Out Low Performing Schools on Student Performance (Working Paper No. 1521). Tulane University, Department of Economics. Retrieved from https://ideas.repec.org/p/tul/wpaper/1521.html
[ix] See Zimmer et al. (2016) and Gill, B., Zimmer, R. Christman, J.B., & Blanc, S. (2007). State Takeover, Restructuring, Private Management, and Student Achievement in Philadelphia Santa Monica, CA: RAND Corporation and Research for Action.
[x] http://www.ncsl.org/research/education/-bathroom-bill-legislative-tracking635951130.aspx
[xi] http://www.usnews.com/news/articles/2015/09/21/as-test-results-trickle-in-states-still-ditching-common-core
[xii] http://www.cbpp.org/research/state-budget-and-tax/most-states-have-cut-school-funding-and-some-continue-cutting
[xiii] Here I focus exclusively on academic achievement measures. In theory, one could broaden the set of accountability metrics to include non-cognitive outcomes. However, as pointed out by authors in this space before, there are a host of problems associated with properly conceptualizing and measuring such soft skills. See, for example, https://www.brookings.edu/wp-content/uploads/2016/07/Download-the-paper2.pdf, and https://www.brookings.edu/research/more-on-soft-skills-time-to-flit-the-grit/.
[xiv] Figlio and Loeb or Ladd chapters. Figlio, D., & Loeb, S. (2011). School accountability. In E. Hanushek, S. Machin, & L. Woessmann (Eds.), Handbook of the Economics of Education (Vol. 3, pp. 383–421). Retrieved from https://cepa.stanford.edu/sites/default/files/Accountability_Handbook.pdf
[xv] Thanks to Russ Whitehurst for this suggestion.
[xvi] Reville, Paul. (2015). “The Journey Toward Equity and Excellence: The Massachusetts Experience.” In A. M. Blankstein Editor & P. Noguera Editor (Eds.), Excellence Through Equity (pp. 185-201). Thousand Oaks, CA: Corwin.
