Michelle Rhee’s public popularity has shifted upward within the District of Columbia, pollsters tell us, but the elites who chair the committee set up by the National Research Council (NRC) of the National Academy of Science to assess Rhee’s chancellorship are holding firm to their anti-Rhee convictions, no matter what the evidence.
In my recent Education Next essay, I identified the biases and inaccuracies in their report on Rhee’s chancellorship. Specifically, I pointed out that gains on the National Assessment of Educational Progress under Rhee’s tenure were much larger than average gains for the other ten urban school districts participating in the assessment in 8th grade math and in 4th grade reading and math. (I also reported that 8th grade reading results are less favorable for the District of Columbia.)
In their reply to my critique, co-chairs Robert Hauser and Christopher Edley say that D. C. gains were “reliably higher than that of only two districts (Austin and Cleveland) in grade 4 mathematics and one district (Cleveland) in grade 4 reading—but no others. This finding has no particular ideological or political bent; it is the result of careful and straightforward analysis of data.”
Methinks thou dost protest too much, Messrs. Hauser and Edley. You are both good enough statisticians to know that averages across multiple cases (which I reported) are more informative than case-by-case comparisons (which you rely upon), each of which is noisy and therefore more likely to show no statistically significant difference.
In other words, average performance across ten cities provides a more precise and therefore more informative estimate of the underlying truth of the matter than do noisy individual city-by-city comparisons.
To knowingly elevate noisy data over more precise information is to pursue a “particular ideological or political bent.”
What is disturbing about all this is Robert Hauser’s new role as head of NRC’s education division. When the leader of a key division at NRC is engaged in promoting his own ideological agenda, all kinds of misleading reports can emerge from an allegedly scientific agency dependent upon government funding. Members of Congress should take special notice of this fact when asked to fund still another NRC study.
The evidence for all this comes not just from the DC report discussed here. For more on what is going on at the NRC, take a look at the recent report on school accountability, which Eric Hanushek has subjected to withering criticism.
-Paul E. Peterson
