The U. S. government just provided the public with much the same information Education Next (Ednext) shared with readers a year ago: A comparison of state standards in reading and math at the 4th and 8th grade levels.
In a recently released report, the U. S. Department of Education uses data from the National Assessment of Educational Progress (NAEP) to rank the relative rigor of each state’s proficiency standards. Its study is based upon data that is now more than two years old. Its cost? It must be at least a million dollar study, if all the direct and indirect costs are calculated, as the study involved a complicated data collection effort. I invite interested readers to inquire of department officials as to just how much its study added to the nation’s debt.
Since Ednext released pretty much the same information a year ago, the U. S. Department of Education could have saved itself a bucket of dough simply by putting our study up on its website.
Of course, it’s always useful to have the official U. S. government seal of approval on Ednext work, and it’s true that while the two studies virtually replicate one another, they are not a perfect match. Ednext took a census approach to data collection (relying upon the official results released by states) while the Department gathered data from a sample of schools. Each approach has its own pluses and minuses, as I pointed out in a previous blog post.
But the bottom line is that the two studies get to virtually the same place. The correlations between Ednext and U. S. government rankings are as follows:
4th grade reading: 0.86
4th grade math: 0.88
8th grade reading: 0.85
8th grade math: 0.87
When you get that high a set of correlations between two sets of rankings, there is not much difference left to quarrel about. Interested readers can check this out by looking at the side-by-side rank ordering of states in both studies in the attachment below.
So, for example, Ednext ranked Massachusetts, Missouri and New Jersey as the three states with the highest standards in 4th grade reading. So did the Department.
Ednext identified the four states with the lowest standards in 4th grade reading (in order from the bottom) as: Tennessee, Nebraska, Georgia and Alabama. The Department provided no information on Nebraska but agreed that the other three were at the bottom. It added Oregon as the fourth, while Ednext ranked Oregon a bit higher.
Colorado is the one state where we provide substantially different rankings. Ednext ranked it 4th; the Department says it is 45th. I suspect the difference is due to a change in standards in Colorado, but I invite readers to throw light on the discrepancy.
The top 3 conclusions of the Department report are as follows:
1. “There is wide variation among state proficiency standards.”
Yes, Ednext shows that clearly by grading states on an A to F scale.
2. “Most states’ proficiency standards are at or below NAEP’s definition of Basic Performance.”
Yes, Ednext gave most states a grade of C or below, which meant they diverged dramatically from the NAEP proficiency standard (which is set considerably higher than the basic level).
3. “For those states that made substantive changes in their assessments between 2007 and 2009, most moved toward more rigorous standards as measured by NAEP.”
Yes, Ednext reports more rigorous standards in reading. However, it shows a slight slippage in math standards.
In short, we have no complaint about the U. S. Department of Education report other than the authors failed to cite the Ednext one. We do not know whether the Department’s ranking is more precise—or less precise—than the Ednext’s ranking. A good case can be made for the merits of either study’s methodology.
But as to the cost-benefit equation, there is little doubt. The Ednext study was conducted at modest cost—and none at all to the taxpayer. The U. S. Department of Education study needlessly added to the national debt.
– Paul E. Peterson
