Podcast: Eric Hanushek and Marty West discuss this and another study that look at teacher dismissals.
If principals have the authority to dismiss teachers, will they dismiss the less effective ones, or will they instead make perverse decisions by letting the good teachers go? Evidence from low-stakes surveys suggests that principals are able to identify the most and least effective teachers in their schools, as measured by their impact on student achievement (see “When Principals Rate Teachers,” research, Spring 2006). But would that ability influence their dismissal decisions?
On this topic, debate has been vigorous but research almost nil, in good part because teachers with tenure are not easily dismissed and principals take on that task only if they have a strong backbone or face an extremely urgent situation, or both. In some instances, however, principals have considerable latitude when it comes to dismissing teachers who have not been in service long enough to have earned tenure.
One such situation developed in Chicago in July 2004 when the Chicago Public Schools (CPS) and the Chicago Teachers Union signed a new collective-bargaining agreement that gave principals the flexibility to dismiss probationary (nontenured) teachers beginning in the 2004–05 school year for any reason and without the documentation and hearing process that is typically required for dismissals in other districts. Since CPS provided information that allowed me to link information on CPS teacher dismissals to several measures of teacher performance, I was able to study whether principals exercise their authority wisely. The procedures were fairly straightforward. By comparing the characteristics of dismissed versus nondismissed probationary teachers within the same school and year, I was able to determine just how much weight school administrators place on a variety of teacher characteristics, including their performance in the classroom.
I find that principals in Chicago do exercise their authority in sensible ways. Principals are more likely to dismiss teachers who are frequently absent and who have previously received poor evaluations. They dismiss elementary school teachers who are less effective in raising student achievement. Principals are also less likely to dismiss teachers who attended competitive undergraduate colleges. It is interesting to note that dismissed teachers who were subsequently hired by a different school are much more likely than other first-year teachers in their new school to be dismissed again.
These results suggest that other school districts could possibly improve student achievement if they adopted policies similar to those applied in Chicago. To be clear, however, the analysis presented in this paper does not seek to evaluate the educational impact of this new policy. Instead, it uses the existence of the policy, in conjunction with detailed data on teachers and principals, to provide descriptive evidence on the relationship between the exercise of dismissal authority and teacher effectiveness.
Teacher Dismissals in Chicago
As in many public school districts, teacher layoffs and dismissals in CPS are highly regulated. Prior to 2004, virtually no teachers—not even probationary teachers—were dismissed for cause in CPS. Of course, it is likely that some teachers who switched schools or left CPS entirely were informally “counseled out” by school administrators. But it was impossible to distinguish these “involuntary” separations from truly voluntary attrition.
This situation changed with the signing of a new collective-bargaining agreement in 2004. Each February, principals are able to log into a district computer system that has a list of all of the probationary teachers in their school (i.e., those who have been teaching for fewer than five consecutive years during the period of my analysis). The principal can then check one of two boxes: renew or nonrenew. Although principals are required to provide district officials with at least one reason for the nonrenewal decision, they are not required to justify or explain their decision and they do not need to provide teachers with this reason. If a principal chooses nonrenew, the teacher may reapply to positions in other Chicago public schools. However, nonrenewed teachers are not guaranteed another job in CPS. The ease with which administrators can dismiss a probationary teacher, with a simple “click” of a button, is noteworthy. This policy change made Chicago the only large school district in the country to provide principals with this degree of flexibility over personnel decisions. Already since the conclusion of the analysis period for this study (2005 through 2007), this flexibility has diminished in several ways. For example, the probationary period has been reduced from 4 to 3 years, and principals who choose to nonrenew a teacher now must have conducted at least one formal observation of the teacher prior to nonrenewal.
Data
The data for my study of this policy change come from several sources. Teacher personnel files provide information on teacher background, current assignment, and, for probationary teachers, whether or not they were renewed. I supplement this with information on school demographics, principal characteristics from personnel files, and student test-score information.
I examine dismissal among probationary teachers in CPS in three consecutive school years: 2004–05, 2005–06, and 2006–07. The sample excludes individuals who were employed by the central office, including speech pathologists, nurses, counselors, and teachers working in administrative or professional development capacities. Moreover, I exclude teachers in a handful of “alternative” schools that serve severely disabled students or other special populations, as well as teachers on leave or who were employed less than half time. For a small number of teachers who taught subjects such as art or music in multiple schools, I include only the observation in the school that is listed as their “primary” appointment. The final sample consists of 16,246 elementary school teachers and 7,764 high school teachers spread across 588 schools.
Measures of Teacher Quality
This analysis incorporates three proxies for teacher performance. First, I use teacher absences because they are well measured, are easy to interpret, and impose substantial nonfinancial and financial costs on the school. The second measure is the formal performance rating that the principal gave the teacher in prior years. Traditionally, principals rate teachers every one to three years (depending on the tenure status of the teacher) on a four-point scale that indicates superior, excellent, satisfactory, or unsatisfactory performance. While there are no high stakes associated with these ratings (virtually no teachers receive an unsatisfactory rating), there is considerable variation across teachers in the top rating categories, and they arguably provide a sense of how the principal views the teacher. The third measure is a value-added estimate of teacher effectiveness. This measure is meant to capture the extent to which each teacher contributes to student achievement growth from one year to the next, as measured by the standardized tests taken by students in CPS. While this is an objective and direct measure of one important dimension of teacher effectiveness, only a fraction of teachers work in grades and subjects in which students take standardized tests. It is not possible to calculate value-added measures for many teachers in our sample, including teachers in grade 2 or below, most teachers in grades 10 or above, and any teacher in a noncore subject. Unlike some school districts, Chicago traditionally has not maintained reliable data linking teachers to classrooms, particularly at the elementary level. Working with CPS officials, however, I was able to obtain such links for a limited sample of teachers and years, thus allowing me to create value-added measures for part of my sample.
Methods
The primary goal of my analysis is to determine which teacher, principal, and school characteristics are associated with the likelihood that a teacher will be dismissed. I first compare the probability that a teacher is dismissed across schools and years in order to discern any differences related to school characteristics. Then to examine the influence of teacher characteristics on the likelihood of dismissal, I compare teachers within the same year and school to account for unobserved school-level factors that might be correlated with teacher characteristics and the probability of dismissal.
A concern with this approach is that if the analysis fails to include a teacher characteristic that a) principals consider in the dismissal decision and b) is correlated with one of the included variables, the estimate for the included characteristic may be biased. One potentially important variant of this concern involves the supply of teachers. If it is more difficult to find qualified teachers in certain subjects or grade levels, then the principal may be less likely to dismiss teachers in these areas. To the extent that teachers in harder-to-staff areas are concentrated among particular demographic groups, or tend to graduate from particular institutions, the results for these teacher characteristics could be misleading. Also, schools fund teachers from a variety of revenue streams, and it may be difficult for principals to reallocate positions across funds. For this reason, if a school experiences a decline in a particular revenue fund, the principal may be more inclined to dismiss teachers funded by this source.
To address these concerns, I account in all analyses for the teacher’s program area (for example, regular education grades 1 to 3, regular education grades 4 to 8, secondary math, secondary science, bilingual education, vocational education, etc.) and for the revenue source from which each teacher position is funded.
Of course, it is still possible that my results concerning specific teacher characteristics suffer from a standard omitted variable bias. For example, it may be the case that high rates of absenteeism are associated with a bad attitude or shirking in other dimensions, and it is these factors, rather than the absences per se, that the principal is reacting to in dismissing teachers with more absences. In this case, one may not be able to say anything definitive about principal views regarding teacher absenteeism itself, but rather about behaviors and characteristics associated with absenteeism, all of which presumably speak to performance in some form or another.
Dismissal Policy Impact
Each year under the new policy, roughly 11 percent of probationary teachers were dismissed, despite the fact that more than one-third of schools did not dismiss any teachers. The numbers of teachers who were nonrenewed in any given year likely overstates the impact of the policy because a number of young teachers would likely have left CPS in the absence of the policy, either voluntarily or due to subtle “encouragement” on the part of the principals. If the dismissal policy merely formalized previously informal dismissals, however, then one would not necessarily expect to find a substantial change in separations.
Comparing dismissal rates before and after implementation of the new policy provides insight on this issue. In the three years prior to the introduction of the policy, roughly 10 to 15 percent of first-year probationary teachers left CPS and an additional 4 percent moved to a different CPS school. In the years after the policy was in place, the corresponding rates were roughly 18 and 10 percent, respectively. Comparing the year immediately prior to establishment of the policy (2004) with the first two years of the policy’s implementation (2005 and 2006), it appears that the separation rate increased by roughly 9 percentage points (see Figure 1). In contrast, there was virtually no change among more-experienced teachers (i.e., those with 6 to 15 years of experience), who were not subject to the policy. The dismissal policy therefore appears to have had at least a modest impact on the number of teacher separations, although the impact is not as large as the overall nonrenewal numbers would suggest.
It is worth noting that more than half of the dismissed teachers were rehired the following year by another school in the district. For example, 50.6 percent and 56.4 percent of first-year probationary elementary and high school teachers, respectively, who were dismissed in spring 2005 were rehired by a CPS school in the fall. At least some of the dismissals under the policy were the result of position cuts, in which case the teacher’s former principal may have provided the teacher with a good recommendation; it is therefore not surprising that some fraction of dismissed teachers were rehired. It is also likely that some fraction of teachers dismissed due to poor performance were also rehired by other CPS schools.
Which school and principal characteristics are related to dismissal? In both elementary and secondary schools, principals in the district’s larger schools dismissed a smaller fraction of probationary teachers. In elementary schools, higher student achievement at the school is associated with a smaller fraction of probationary teachers being dismissed. Among high schools, however, schools with higher-achieving students dismissed a larger fraction of their probationary teachers. Principals who attended more competitive colleges and principals who were older dismissed a smaller proportion of teachers in both elementary and high schools. Male high-school principals dismissed a significantly smaller percentage of their teachers, while principal gender did not play as important a role at the elementary level. Finally, principals new to the building dismissed a substantially larger fraction of teachers in elementary schools, but not in high schools.
Teacher Characteristics
Turning to the characteristics of individual teachers, I find that prior-year principal evaluations and current-year teacher absences both influence the likelihood of dismissal (see Figure 2). Teachers who were rated satisfactory in the prior academic year were 22.1 percentage points more likely to be nonrenewed than teachers in the same school who were rated superior. Teachers rated excellent were 4.3 percentage points more likely to be dismissed than those rated superior. Given an average dismissal rate of roughly 11 percent, these results suggest that teacher performance as reflected in prior evaluations is strongly associated with dismissal. Teachers who were absent 11 to 20 times between September and March of the current year were also 11.3 percentage points more likely to be nonrenewed than their colleagues who were never absent. Teachers absent 6 to 10 days were 3.5 percentage points more likely to be dismissed.
The results also indicate that principals value teachers with stronger educational backgrounds as measured by college quality. For example, a teacher who attended a highly competitive college (with a Barron’s ranking of four) is nearly 3 percentage points (roughly 15 percent) less likely to be dismissed than a teacher who attended a least-competitive (unrated) college. On the other hand, on average, principals do not seem to value certification exam performance or advanced degrees, at least after taking into account the other available measures of teacher performance.
Interestingly, probationary teachers who were dismissed from another school in the prior year, and rehired by the current school, are substantially more likely to be dismissed a second time. For example, elementary school teachers who were dismissed from another school in the prior year were 4.9 percentage points (about 45 percent) more likely to be let go relative to first-year teachers in the school. In high school, previously dismissed teachers were 13.4 percentage points (more than 130 percent) more likely to be dismissed than first-year teachers. These results suggest that many of the initial nonrenewal decisions were not idiosyncratic, stemming from a particularly bad match, or based on temporary difficulties experienced by the teacher. Rather, they suggest that, at least in many cases, the initial nonrenewal decision reflected a concern with the teacher’s general productivity.
These results provide evidence that principals consider some measures of teacher performance and qualifications in making their dismissal decisions. To the extent that one views student achievement as the primary outcome of interest, however, one should directly assess how a teacher’s ability to improve student achievement influences the likelihood of dismissal. I provide some evidence on this issue by focusing on the relationship between teacher value-added and dismissal for the subsample of 803 elementary school and 1,134 high school teachers for which value-added measures are available.
For elementary schools, a one-standard-deviation increase in teacher value-added is associated with a 7.1-percentage-point (over 100 percent) decrease in the likelihood of dismissal (see Figure 3). In contrast, I find that teacher value-added has zero association with dismissal among the sample of 9th-grade core-subject teachers in high schools. One possible reason for the difference across grade levels is that the assessment used for the 9th-grade value-added measure is the PLAN test, which is given in the fall of a student’s 10th-grade year. PLAN is developed by ACT and is not tightly linked to any particular curriculum. Hence, because of both the timing of the exam and its content, the 9th-grade value-added measures may not capture teacher effectiveness as well as the elementary value-added measures.
Do Principals Discriminate?
One potential concern about policies like Chicago’s that provide principals with greater discretion in personnel decisions is that principals would dismiss teachers capriciously or on the basis of criteria unrelated to performance. Indeed, I find that several teacher demographics, including age, gender, and race, are associated with the likelihood of dismissal, even after controlling for the measures of teacher performance and qualifications described above. Principals are 3.8 percentage points more likely to dismiss male teachers than female teachers, an effect of more than 25 percent given the baseline dismissal rate of 10 to 12 percent. Principals are considerably more likely to dismiss older teachers. For example, teachers 36 to 50 years of age are 4 percentage points (33 percent) more likely to be dismissed than teachers age 22 to 28. The relatively small number of probationary teachers over age 50 is 10 percentage points (nearly 100 percent) more likely to face dismissal than their youngest counterparts. And black teachers are 2.1 percentage points less likely to be dismissed than their colleagues.
While these results raise some concerns, it would be incorrect to conclude on the basis of this evidence alone that principals in Chicago were acting in a discriminatory manner. The analysis reported here cannot control for many direct measures of teacher qualities that principals could legitimately consider in making a dismissal decision (e.g., energy, enthusiasm, ability to relate to children, familiarity with the best instructional practices). Moreover, the sample selection introduced by nonrandom hiring may lead to biased estimates of the relationship between dismissal and any easily observable, predetermined teacher characteristic such as age or gender. If, for example, male teachers were less productive on average than female teachers (or even if the principal believed this to be the case), then the marginal male teacher who was hired must be more attractive on some other, likely unobservable, dimension relative to the marginal female teacher hired.
In order to shed light on the issue of principal discrimination, I examine whether principals are more likely to dismiss teachers of a different gender, age, or race from their own. Although principals are no more likely to dismiss a teacher of the opposite gender, they are somewhat more likely to dismiss teachers of a different race. While these patterns could indicate discrimination, it is possible that they are explained by other factors. Given the widespread belief that same-race role models are crucial for low-income students, it would not be surprising if principals took into account the composition of their student body when making dismissal decisions. Indeed, insofar as prior research has demonstrated that, all else equal, students learn more when taught by a teacher of the same race, this might be a legitimate determination on the part of the principal. My results provide support for this hypothesis. I find that as the fraction of students in the school that share the race of the teacher rises, the likelihood that the teacher will be dismissed declines. Specifically, an increase of 50 percentage points in the fraction of students who share the teacher’s race decreases the likelihood that the teacher will be dismissed by slightly more than 1 percentage point, or 10 percent. More importantly, the evidence that principals are more likely to dismiss a teacher of a different race becomes statistically insignificant after controlling for this variable.
Finally, I find evidence that younger principals are more likely to dismiss older teachers than they are to dismiss younger teachers. There are no obvious explanations for this pattern, although one might speculate that younger principals may value different characteristics in a teacher than older principals. Regardless, this pattern does seem to warrant further exploration.
Conclusions
By comparing the characteristics of dismissed versus nondismissed probationary teachers within the same school and year, the analysis presented above provides a unique source of evidence on which teacher characteristics principals value most highly. I find that principals do consider teacher performance in determining which teachers to dismiss. Principals are significantly more likely to dismiss teachers who are frequently absent and who have received unsatisfactory evaluations in the past. Perhaps most telling, elementary school teachers who were dismissed had significantly lower impacts on student achievement in prior years than their peers who were not dismissed.
These results suggest that reforms along the lines of the Chicago policy could improve student achievement by providing principals with the tools to manage the quality of personnel in their classrooms. It should be noted, however, that many principals—including those in some of the worst-performing schools in the district—did not dismiss any teachers despite the new policy. The apparent reluctance of some Chicago principals to utilize the additional flexibility granted under the new contract may indicate that issues such as teacher supply and/or social norms governing employment relations are more important factors than policymakers have realized.
Brian Jacob is professor of education policy and economics at the University of Michigan. This article is based on a study that is forthcoming in Educational Evaluation and Policy Analysis.
This article appeared in the Fall 2011 issue of Education Next. Suggested citation format:
Jacob, B.A. (2011). Principled Principals: New evidence from Chicago shows they fire the least effective teachers. Education Next, 11(4), 70-
