It’s easy to tell when someone is in the grip of a Big Idea That Explains Everything. Tunnel vision sets in; every analysis, whatever the topic, becomes an occasion for the grand theory to appear. Evidence is read and supplied selectively, in such a way that the theory re-mains unscathed. Skepticism is deployed selectively as well. Findings that comport with the Big Idea are held to a relaxed standard, while the work of critics is subjected to withering scrutiny.
Richard Rothstein, author of the New York Times‘s widely read “Lessons” column, a weekly commentary on education issues, frequently exhibits these symptoms. His Big Idea is that economic forces, especially inequality and poverty, largely determine the outcome of American social projects-including attempts at education reform. The effect of this obsession is two-fold. First, his writings display a factual carelessness, suggesting that details hardly matter if one possesses a higher truth. Second, nearly every engagement with issues of schooling, testing, standards, and teaching becomes an occasion to reassert the primacy of the economic factor.
Rothstein believes that most contemporary criticism of the public schools is misplaced. The main problems lie not with the schools, he claims, but with the injustices associated with the American economic system. It isn’t a lack of competition in the public school system, antiquated hiring and compensation systems, or a dearth of solid research on educational methods that depresses student achievement; it’s an economic system that allows for large differences in income and wealth. As a result, to Rothstein’s mind, education reform won’t be effective without far broader social reforms.
The Sky Isn’t Falling
Rothstein is committed to the view that no crisis exists in American education, that all the critics are merely Cassandras trying to scare the public into accepting their pet reforms. His statistical gymnastics are used most often in the service of defending current school practices against most reform proposals, usually by denying the need for reform in the first place. Thus Rothstein’s explanation for falling SAT scores is not a decline in education quality, but an increase in the number and the diversity of test takers. Says Rothstein, in an August 2000 column: “Interpreting the SAT is more complex than it seems. SAT trends would reflect school quality changes only if every 18-year-old took the test. Not all do. Average scores are affected by who takes the SAT. If only the brightest seniors take it, averages are higher. If more lower-ranked seniors aspire to college and take the test, this could indicate better performance by schools, but still depress the average.”
The claim that slipping scores result from a changed demographic (and hence could even be good news) has surfaced repeatedly in the writings of education commentators such as Gerald Bracey, but it is demonstrably false. Washington Post economics columnist Robert Samuelson summarized the matter in a 1994 column by noting: “The change in the student population preceded the drop in test scores. Between 1951 and 1963, the number of test takers went from 81,000 to nearly 1 million; test scores rose slightly.” Moreover, the percentage of test takers remained relatively constant between 1972 and 1984 (see Figure 1). There were still a million test takers in 1985, the first year in which test scores showed a small uptick after 19 years of decline. Scores have been flat or slightly improved since then, with math scores returning to their levels of 30 years ago, but failing to reach their mid-1960s apex.
Changes in the composition of the test-taking pool don’t explain the decline in test scores either. Studies by the Educational Testing Service and others have showed, in the words of Robert Samuelson, that “the main declines occurred among whites and could not be explained by changes in student’s gender, economic class, or parental education.” This analysis was seconded by Harvard sociologist Christopher Jencks, who pointed out that the SAT scores of advantaged white males have also exhibited a steep decline.
Yet Rothstein, exactly one year later, parroted his earlier claims. His reaction to the release of the 2001 scores, which showed no improvement over the previous year and hence were termed “stable” by the College Board, was to write, “Stable in this case does not mean unimproved. Hidden in the data is more hopeful news than most people would expect. These tests are voluntary. If only high achievers take them, average scores mean one thing. But if a broader range of students takes them, the results must be interpreted differently. The number taking the tests has in fact grown a lot. . . . It is remarkable that averages gained at all while the test-taking base was expanding.”
Again, in an October 2001 column, the familiar refrain reappeared in a rebuttal of the 1983 Nation at Risk report. The authors, we are told, “misunderstood the decline in test scores. College Board results had dropped, but that was due to the growth in college-going ambition. The SAT was no longer taken only by top students, and so average scores of test takers naturally fell.”
In short, in three successive bombing runs, Rothstein failed even to acknowledge, much less refute, any evidence that would undermine his assertions. This suggests an opinion resistant to the complexities of the issue.
Having dismissed claims that the achievement of American students has declined over time, Rothstein turns to the other major source of worry: the woeful performance of American students on international comparisons of educational achievement. Take a May 2001 column, in which Rothstein manages to convert lead into silver. He starts by facing the facts. He concedes that on the Third International Mathematics and Science Study, “Our 8th graders scored below their peers in almost every other industrial nation that took part. Students in Japan and Korea ranked near the top. In math, average American 8th graders would have scored below the 25th percentile in Japan or Korea.” But don’t worry, because the kids certainly aren’t: “The study also asked students if they liked math and science. . . . here Japanese and Koreans scored at the very bottom. . . . In the United States, 35 percent felt positively about math and 32 percent about science, more than in almost every other industrial nation.” A Japanese scholar is also invoked to assure us that his countrymen do “not attach great importance to students’ rankings because the exams measure skills valued by the old education system, not the new.” In fact, Rothstein concludes, the dour Japanese want to emulate our schools because of our “zest for living.”
In a July 2001 column, Rothstein tells his readers not to fret over data from the National Assessment of Educational Progress showing that two-thirds of American 4th graders can’t read above a basic level, because “on an international survey of reading ability, American 4th graders scored higher than pupils everywhere except Finland.” In other words, international comparisons are apparently valid when they corroborate Rothstein’s fundamental beliefs, but easily dismissed when they reflect poorly on the American education system.
Poverty’s Pull
For Rothstein, even if there were a crisis in American education, it certainly wouldn’t be the schools’ fault. Any problems that do exist are the result of social inequality. Concerned about low SAT scores? Rothstein insists that “the best predictor of test scores has always been students’ social class.” Thus low scores are to be expected from disadvantaged youngsters.
Rothstein returns to this theme time and again. For instance, Rothstein’s December 2001 column highlighted a study by Eric Dearing of the Harvard Graduate School of Education. Dearing examined a ratio termed the “income-to-needs” of families below the poverty line and showed that when income increased (roughly $4,500 per year over three years), very young children performed better on tasks where they were asked to identify colors, shapes, and letters (skills considered important in school readiness).
This was a only a small-scale study, but that didn’t stop Rothstein from drawing the far-fetched conclusion that “educators have a stake in promoting a federal income policy that focuses on immediate income support for the unemployed, because this in itself could make instruction more effective.” There’s nothing wrong with arguing that the unemployed should receive more generous benefits, but this has hardly anything to do with education; it’s certainly not a conclusion based on solid evidence, which Rothstein is always sure to demand of those who don’t agree with him.
Besides, most of the serious research points in the opposite direction. For example, the University of Chicago’s Susan Mayer undertook a far more comprehensive analysis of the relationship of income to school achievement in a 1997 book, What Money Can’t Buy. She examined nearly 17,000 records in two massive data sets in her search for the true effects of income. The study is important for its methodological sophistication and its conclusions, which take us beyond the traditional left-right political axis regarding welfare programs and the causes of poverty.
Mayer showed that income per se is not a consequential factor in children’s performance. Beyond providing the ability to satisfy basic needs like food and shelter, income is not a necessary, much less a sufficient, explanation of children’s academic achievement. Mayer found that a supportive family structure (a stable, two-parent home), a culture of learning within the family and neighborhood, and natural abilities were much more important than income. Given these factors, income can certainly help people achieve their ends. In their absence, however, income is largely inconsequential.
These findings would seem to present the perfect opportunity for Rothstein to flex his critical muscles by rebutting Mayer’s scholarship-or at least addressing it. Nevertheless, he seems to have ignored her study entirely-despite the fact that her findings run directly counter to his Big Idea.
What’s strange is that Rothstein certainly isn’t shy about attacking his ideological opponents. In his first column of 2001, he tackled a much-publicized Heritage Foundation study of 21 schools that exhibit both high poverty rates and high test scores. This was a direct challenge to Rothstein’s Big Idea-that there is an immutably inverse relationship between income and student achievement. How would Rothstein assail the study? Let’s try invoking ideology first. Heritage is “conservative,” we read, so its report “as a whole is enveloped in such contempt for most public education that its valid messages are lost.” Then let’s supplement the Big Idea. “The report’s biggest flaw is its assumption that poverty alone defines the problems of the low-income school. . . . Rather, schools with consistently low scores typically have children for whom poverty per se is only one problem. They also suffer from crime-ridden neighborhoods, broken families, parental stress, inadequate housing, and poor health.”
Rothstein’s maneuver here is a tactical retreat. Heretofore, income has served for him like an engine’s governor, an upward limit on a school’s (or individual’s) capacity to perform. But Heritage located schools that perform well despite their low-income student body. Rothstein’s way out is to declare the study myopic, saying that it focuses exclusively on income, when other criteria, not examined in the study, are clearly the relevant factors. However, this maneuvering is a bit circular, considering that inadequate housing, bad neighborhoods, and poor health are usually direct proxies for poverty. Still, in an effort to avoid a contradiction, Rothstein makes an important acknowledgment: successful children, he says, come from “stable homes and parents [who are] regularly employed.” Hence, Rothstein suggests, if only the study had asked proper questions, instead of obsessing over income, it would have found those factors (like parents) that really contributed to the schools’ academic success. Here it sounds as if he agrees with Mayer-when it suits his ideological purposes.
The Bell Tolls
So fixated is Rothstein on the determinative role of income that he actually proposes a sliding scale for learning standards. It is inherently unfair, he has argued on several occasions, to judge all schools and students by the same academic standards. Because of their disadvantages, low-income students should not be expected to compete against the affluent. He recommends adjusting the expectations for the schools located within a given neighborhood based on the area’s average income.
Rothstein broached this argument in a November 1999 column entitled, “Does Social Class Matter in School?” We are in no suspense about Rothstein’s answer, of course: the poor, because they are poor, should not be expected to learn as much. Poor children are stuck in shoddy day care, while “typical middle class parents raise their children differently,” providing them their own “head start.” As Rothstein puts it, we must resist the “dangerous myth” that “all children, regardless of academic background, can achieve to the same high standards if only schools demand it.” Accordingly, “schools with privileged children should be termed failing if they test only at the 70th percentile. . . . A school with many poor children, scoring at the 35th percentile, could be highly successful, though it tests below average.” Substituting race for class in such an argument would make it perilously near to the views that were attributed to Charles Murray and Richard Hernnstein in their controversial book The Bell Curve.
Rothstein actually invoked Murray’s controversial theories in a December 1999 column, presumably because he found this framework useful: “We simply cannot set one standard applicable to all,” he writes, since that would be “statistically foolish.” The foolishness derives from the observation that human performance is often normally distributed, which means that some are necessarily one or more standard deviations above or below the average. It follows for Rothstein that differing income levels call for differing academic standards.
Most schools do something like this by “tracking” students, albeit by previous academic performance, not income. Rothstein, however, denounces this form of relativism, apparently because it is not income-based. For him, tracking is based on another “dangerous myth,” since the practice of taking “poor children . . . assigned to tracks and taught less challenging lessons” assumes that “disadvantaged children could not learn.” So you can offer all students, rich or poor, the same curriculum; you just can’t test to see whether they’ve all mastered it. Evidently the only solution is a classless society.
The danger is that, while we await socioeconomic utopia, Rothstein would expect nothing better from the schools themselves. Indeed, his answers to the problem of disparities in student achievement often involve placing limits on high-achieving children rather than improving the education of low-achieving children. Consider homework. In a May 2001 column, Rothstein lamented the fact that teachers are assigning more homework, which is said to be “up 50 percent in the last two decades.” This is a problem, Rothstein believes, because it “may increase the gap between students from middle-class and low-income homes. With growing inequality now a greater danger than middle-class pupils’ inadequate achievement, policies that widen learning differences should be avoided.”
Rothstein cites an academic authority to reinforce the claim, quoting University of North Carolina professor Eugene Brooks, who says, “Because of homework, schools either consciously or unconsciously reproduce social inequality. It can be avoided only if teachers take over homework supervision from parents.” That’s a somewhat breathtaking mission for the school-reducing the impact of social class on learning by expunging parents from the equation, since they are unequal in their degree of helpfulness. It is apparently better for all youngsters to languish in dreary study halls-presumably reducing the amount of time left for instruction-than to take the risk that one mother might help her child learn faster than another. Rothstein concludes, “It is unconscionable for educators to exacerbate inequality by assigning homework without first ensuring [that social programs] are in place.” Perhaps Rothstein ought to teach 5th grade. No homework, no testing, and being graded on their “zest for living” rather than on their achievement in math and science-his students would love him!
Hungry for Facts
Though Rothstein’s column is nominally about education, regular readers find themselves spending as much time rooting about in sundry social programs, organized around the theme that those who would fix schools by attending directly to schooling are misguided. “By focusing only on schools, government may waste money trying to fix academic problems that it could have prevented in the first place at less expense.” For example, if schoolchildren falter on tests, the reason must be malnutrition. In a January 2000 column, he writes, “Forced to spend more, poor families often raid food budgets to pay rent. Children then suffer nutritionally, compounding cognitive problems.”
He amplified this theme in August 2001, asking, “What is the most efficient way to raise low-income pupils’ achievement? . . . Improving nutrition might bring a bigger test-score gain. . . . Undernutrition found in the United States affects performance. . . . Reducing hunger that causes low test scores may be accomplished more easily than the various unproved educational reforms commonly advanced. A higher minimum wage that helps low-wage parents feed their children could be one important step in ensuring that no child is left behind. And those wanting to narrow academic gaps can . . . contribute to the nation’s overstressed food banks. This too would amount to real education reform.”
A claim that malnutrition might affect cognition and learning is not unreasonable. But there is little credible evidence that a large hunger problem exists in America. Activist groups that routinely assert that American children suffer malnutrition are actually measuring something they term “food insecurity,” a condition that usually results more from poor household management than from scarce resources.
Rothstein has shown himself to be as gullible to the claims of activists as the popular media. When a 1990s survey was said to show that there were 12 million hungry children, NBC’s Tom Brokaw quickly transformed that claim into “12 million American children are malnourished.” On CBS, Dan Rather went further, announcing, “A startling number of American children [are] in danger of starvation.” These preposterous facts were generated by the Food Research Action Center (FRAC) and rely on some statistical sleight of hand. The surveys don’t measure nutrition, but instead ask about feelings of “insecurity” concerning food, measuring the percentage of households that reported “difficulty getting enough nutritious, safe food at all times, in a socially acceptable way.” This approach has drawn scorn from such commentators as the New Republic‘s Mickey Kaus and Johns Hopkins professor George Graham, who said of these numbers, “A lot of what the activists are calling hunger is just absolute rubbish . . . irresponsible people making irresponsible claims.” Kaus was more succinct: “The whole project oozes phoniness.”
In fact, the larger food-related problem besetting American children today is not hunger but obesity. Indeed, one reads in one of Rothstein’s November 2000 columns, here citing the U.S. Surgeon General, that “with the percentage of overweight children doubling in the last 30 years, school kids are less healthy.”
In the obesity column, Rothstein is defending mandatory phys ed, which he worries could be sacrificed to increased academic demands. He wonders, “Is all this (obesity) an unintended consequence of raising academic standards? Are schools squeezing in more math by eliminating gym?” This was not the first time he made this claim. In a November 1999 column, Rothstein, fretting that schools are being carried away by standards and testing, wrote, “Test scores are rising. But more teenagers have fatty diets. . . . adolescent obesity has soared. Neither children nor our economy will prosper if graduates have higher scores, but are overweight.” Surely Rothstein can make better arguments against testing than the shaky hypothesis that it leads to obesity. Are testing programs really taking the place of physical education? Do existing phys ed programs make any difference in children’s weight? Here Rothstein is engaging in pure conjecture.
Continuing his seeming obsession with the relationship between food and cognition, Rothstein writes, in a March 2001 column, “Surgeon General David Satcher reported . . . that more than a third of poor children have untreated dental cavities. Pupils taking tests with toothaches are unlikely to score as well as those undistracted by pain.” In other words, we’re unfairly expecting teachers and schools to administer tests before society has ensured adequate dental care for everyone. But are most cavities even associated with toothaches? Does Rothstein have any evidence that toothaches impair performance? And with all this social engineering, when will we ever teach reading?
Double Standards
Rothstein’s own assertions are rarely backed by solid research or even by decent facts. However, when he is confronted with research that counters his worldview, the methodological gloves come off. Consider his approach to voucher research. If poor families were given vouchers redeemable at the schools of their choice, and the achievement of some students rose, it would call into question Rothstein’s notion that income is the master variable. We therefore ought to expect him to resist such research. And that is exactly what he has done.
Confronted with studies that show black students making gains as a result of being given vouchers (full disclosure: these studies were led mainly by Paul Peterson, this journal’s editor-in-chief), Rothstein turns to Stanford education professor Martin Carnoy to build a strict critique. Proper experiments should be double-blind, Carnoy says, as in drug trials. This cannot be done with kids who know what school they’re in. Moreover, some students left the voucher program after one year, and some declined to participate at all. Rothstein writes: “Researchers cannot know if leavers differ significantly from stayers.” The gains reported were “small” and “inconsistent.”
There are many problems with this critique. A study doesn’t need to be double-blind in order to be valid; many studies in the medical field are not double-blind. Moreover, the researchers used well-regarded statistical techniques to control for the fact that some students left the program after one year or didn’t participate at all. The broader problem is that Rothstein fails to use this standard, or anything remotely like it, when tackling research that favors his Big Idea. The information that he routinely asserts with confidence, such as studies supporting the primacy of economic factors, represents inferences based on, if anything, methodological procedures that were far less rigorous than those used in empirical work on vouchers. The impression offered is of one straining at gnats while swallowing camels.
There is more, much more. The pattern persists from week to week. The Big Idea is defended at all costs. Critics are dismissed. Economics and social class explain just about everything-except where they don’t. America’s self-styled newspaper of record has but one regular commentary on education, and its author is gripped by a worldview that often blinds him to evidence and confounds the rules of logic.
-David W. Murray is the former director of the Washington-based Statistical Assessment Service, a nonpartisan organization that examines the use of quantitative research by the media.
