Quick fix
Margaret Raymond and Stephen Fletcher’s findings (“Teach for America,” Research, Spring 2002) from their initial evaluation of Teach for America (TFA) are not too surprising, given the makeup of TFA recruits and the teachers with whom they are being compared.
They find that TFA recruits in Houston are “at least as effective as other teachers in the district.” However, the researchers readily admit to the limits of their research, in that many of the other teachers in the Houston district have gone through quick-fix alternative certification programs.
TFA recruits “are a select group of college graduates, culled from the finest universities and often performing near the top of their class,” write Raymond and Fletcher. Individuals with these qualifications are not the norm in alternative-certification recruitment. In Prince George’s County, Maryland, hairdressers and cabdrivers answered ads for teachers in 2000 and 2001. Usually, alternative routes attract those who have not thought about their career choices and are looking for something to do or are unemployed. In fact, during the recession after September 11, 2001, applications for teaching shot up in various locations across the country. Most alternative-certification applicants usually end up in the classroom because something better has not turned up. This is a far cry from the TFA recruits, with their Ivy League degrees and 3.4 GPAs.
During the past decade, TFA has placed 7,000 recruits in classrooms-but only 2,000 to 3,000 of those are still in the classroom. Retention is lower than among regularly trained teachers, creating constant turnover in those schools that choose TFA recruits. This may give TFA graduates a realistic picture of the problems in education (indeed, they’re contributing to some of them, like teacher turnover in urban schools), and they may advocate for solutions as they move on to their “real” careers.
However, TFA is not an answer to the teacher shortage in America. America needs almost 200,000 new teachers each year, or about 2 million in the next decade. The select group of top college graduates that includes TFA members will not stay for long in jobs that are at the bottom of the pay and perk scale-jobs that in many urban and suburban locations make them eligible for low-income housing. The vast majority did not attend Ivy League schools to earn less in a year than it cost for nine months of their undergraduate education.
If the salaries and working conditions of teachers were raised to a level commensurate with those of other college graduates, the profession would begin to attract more of the best and brightest into teacher preparation programs.
Jane Leibbrand
National Council for Accreditation of Teacher Education
Washington, D.C.
No alternative
David Ruenzel’s article (“Tortuous Routes,” Feature, Spring 2002) accurately describes a major deficiency of most alternative routes to teacher certification: Because they reflect traditional assumptions about teaching, they are no more effective than the unworkable systems they are intended to help fix.
Those systems assume that teacher quality is best attained when the state heavily regulates employment, requiring teachers to take numerous education courses before they can be considered for a teaching job. The problem is that because teaching is not a science, the technical knowledge base is thin. Therefore, many education courses lack academic substance and none is highly predictive of success. Yet they dilute prospective teachers’ undergraduate education by displacing academic courses. Since many top college students thus avoid education degrees, teacher-preparation programs often enroll weaker students to meet school demand. Still, education graduates do not provide a reliable supply of new teachers. Forced to commit in their teens in order to take the required courses, some change their minds by graduation and do not enter teaching, while many leave their jobs within a few years.
Because the traditional approach is unworkable, states have always maintained other pathways. However, these long-standing alternatives merely eschew the enforcement of certification requirements rather than questioning their necessity or effectiveness. If districts claim to be unable to find certified teachers, the state declares an “emergency” and authorizes the employment of uncertified people, who then begin after the fact to meet requirements that are supposed to be legal prerequisites. On the one hand, emergency policies have always barred districts from hiring talented people with degrees in academic subjects when a mediocre yet certified person was available. On the other hand, districts have used shortages to rationalize the employment of people who have not studied and do not know the subjects they will teach.
In 1983, New Jersey challenged the traditional approach at its roots. The state asserted that most undergraduate education courses are not useful and trimmed all but three from its traditional preparation programs, thus strengthening teaching degrees and making them more attractive to top college students. The state then gave all candidates with degrees in academic subjects the option of taking the three courses during their initial year of employment.
New Jersey’s policy reflects a belief that it makes most sense to rely on the circumstantial judgments of principals, who happen to be licensed by the state to assess teaching capability. Teacher quality is achieved by expanding job opportunities, intensifying competition, and challenging school leaders to embrace responsibility for teacher selection and induction.
During the past 16 years, more than 10,000 new teachers have completed New Jersey’s alternative route. Most of the state’s 600 school districts-large and small, urban and suburban, wealthy and poor-have hired them. In any given year, up to 40 percent of all new teachers are alternative routers. A five-year pilot study showed that the initial 2,000 had higher test scores than education graduates, were more likely to hold advanced degrees, and more often had previous experience working with children. They were also three times more likely to be minorities and four times less likely to leave their teaching jobs during the first year.
Although other states responded to such evidence by adopting alternate routes, many deserve the criticisms that Ruenzel expresses. The problem is that most of these alternatives were merely tacked onto traditional certification policies that remain unchallenged.
Some alternate routes inexplicably waive the same education courses that the state continues to require in traditional preparation programs. This causes school administrators to shy away from nontraditional candidates because it portrays them as less well “trained.” Other states require alternate candidates to complete the same courses that are required in undergraduate preparation programs. This produces an alternate route that is unnecessarily cumbersome, a discouragement to candidates and employers alike. Still other states model their alternate routes after the old emergency credential policy, restricting their use to instances of teacher shortage.
The failure to challenge the traditional approach is sometimes the result of legitimate philosophical differences. More often, though, it results from political lobbying by interest groups that want to use government regulation to advance their professional image and that otherwise benefit from a tight and monopolistically controlled job market.
Leo Klagholz
Former New Jersey Commissioner of Education
Pomona, New Jersey
Asking the wrong questions
Florida’s A+ program is one of many education programs being closely watched in education and policy circles. Because it includes vouchers and test-based accountability, the program has received national attention as a potential way to improve public education, and especially “failing schools.” The prominence of the program has increased the stakes in research-and the need for careful analysis.
Jay Greene’s analysis of the A+ program (“The Looming Shadow,” Research, Winter 2001) provides results that are incomplete at best and probably quite misleading.
First, Greene assumes that the effects of the A+ program on school performance are due entirely to the pressure of facing vouchers. Our work, however, indicates that much of the effect came simply from the public embarrassment of a school’s receiving a low grade.
Second, Greene reports only raw gains in test scores, rather than effect sizes, which makes it impossible to determine whether the effects are large or small. This is surprising because estimates of the effect sizes are easy to obtain, simply by dividing the raw score gains by the student-level standard deviation in score levels. According to the published work of Gregory Camilli and Katrina Bulkley of Rutgers University, Greene’s raw score gain in reading translates into an effect size of just 0.04-equivalent to increasing a student’s test score from the 50th to the 51st percentile. This effect is very small even by the standard Greene proposes, and grows smaller still if adjusted for the separate effect of school ratings.
Third, Greene performs a kind of cost-effectiveness analysis: Do vouchers produce gains at lower cost than other education reforms? Economists are trained to answer such questions by focusing on the economic costs. Greene’s analysis has some technical problems, but the greater flaw is that he has asked the wrong question altogether. Those who follow the debate on vouchers closely know that the critique of vouchers has little to do with the effects of vouchers on test scores and monetary costs. It involves the possibility that vouchers will create a hierarchy of schools based on income and race, that increased choices for some may mean decreased freedoms for others, and that the roles of parents, teachers, and citizens in the education process will be changed for the worse.
Weighing these issues is the responsibility of policymakers, not economists and education researchers. However, researchers are responsible for helping to frame the questions properly. In this case, the key questions appear to be: How large do the test-score gains have to be to justify serious consideration in the debate about vouchers, equity, and social values? Is a 0.04 standard deviation gain large enough to be considered important? Also, what do these small gains suggest about the performance of the current education system? These are the questions that researchers should be putting to policymakers.
Martin Carnoy
Stanford University
Doug Harris
Economic Policy Institute
Read the unabridged letter here.
Jay P. Greene responds: The degree to which Martin Carnoy and Doug Harris agree with me, at least implicitly, is remarkable. First, Carnoy and Harris do not take issue with the fact that test scores in Florida increased at schools that faced the prospect of vouchers under the A+ program. Second, they do not dispute my analysis comparing the results of the high-stakes state test and the low-stakes SAT 9, which found that the exceptional test-score gains experienced by schools facing the prospect of vouchers represented real increases in student achievement, not a manipulation of the state testing system or a result of teaching to the state test.
The only issues in dispute here are the extent to which vouchers as a sanction for chronic failure are uniquely capable of producing gains and whether those gains are large or not. Carnoy and Harris suggest that perhaps the threat of reconstitution or the stigma of being labeled a failure can motivate schools to improve. It is certainly plausible that various sanctions could be effective. My position is simply that the evidence from Florida shows that improvements occurred when vouchers were the sanction. We do not have evidence of similar quality from other states demonstrating that reconstitution or embarrassment is equally effective. Given the available evidence, we should operate under the working policy conclusion that vouchers are an effective sanction and that the effectiveness of other sanctions is unknown.
Our disagreement over the magnitude of the improvement experienced by schools facing the prospect of vouchers is more substantial, but even this disagreement is only a difference of degree, not kind. Carnoy and Harris rely upon an analysis by Camilli and Bulkley to claim that the gain experienced by schools facing the threat of vouchers was a modest one. Camilli and Bulkley compute the gains as units of standard deviations for individual students, even though we only know the variation in test scores on the school level and do not know the variation at the individual student level. They get around this by assuming a certain relationship between school-level variation and student-level variation in test score results. In other words, they are guessing, even if it is informed guessing. If their guess is off, gains as measured in terms of standard deviations are larger than they claim.
Rather than guess about what is not known, I prefer to report the gains in terms of a unit that is known and that everyone can understand-dollars. I calculate that to achieve the same test-score gain in math achieved by the prospect of vouchers, Florida would need to increase per-pupil spending by $3,484. Even the smaller gain produced in reading scores would require an increase of $888 per pupil to accomplish the same result. By any normal accounting, these gains are not small.
Even if Carnoy and Harris were correct in claiming that the gain produced in reading scores only yields an average improvement of 1 percentile point, such a gain is not trivial over a one-year period. And the larger gains achieved in math (even assuming their standard deviation estimates are correct) would be more substantial.
Given the difficulty policymakers have historically experienced in trying to devise any programs that produce large-scale, demonstrable gains in student achievement, the improvements realized by the A+ choice and accountability program in Florida suggest a very promising avenue for reform efforts.
