Inequality in educational outcomes is substantial and persistent in the United States. Students from high-income families outperform those from low-income families on achievement tests, are more likely to graduate high school, and are more likely to earn a college degree. Black and Hispanic students also earn lower scores on standardized tests, on average, and are less likely to graduate high school and go to college than white and Asian students.
While there are many possible explanations for these differences, one frequent hypothesis is that high-income white and Asian students are taught by more effective teachers. After all, evidence shows that teachers vary a great deal in their impacts on student learning, and that students taught by the best teachers have higher test scores and better outcomes in adulthood, including greater likelihood of college attendance and higher wages.
Studies also have found that teachers working with low-income students, on average, tend to be less experienced and have fewer qualifications that teachers working in high-income communities. In response, federal law currently requires states to ensure low-income students “are not served at disproportionate rates by ineffective, out-of-field, or inexperienced teachers,” and states like Washington offer bonuses to teachers with advanced credentials who work in high-poverty schools. However, more experience and better qualifications do not guarantee better teaching.
We look at student demographics and several measures of teacher quality in 26 public school districts across the United States over a five-year period. We find that, in fact, low- and high-income students have nearly equal access to effective teachers. Effective teachers are found in high-poverty schools, even if their accomplishments are often overlooked because their students typically start out far behind. Conversely, ineffective teachers can be found in high-performing schools, where the impacts of subpar instruction can be camouflaged by students’ other advantages.
Our analysis also suggests that it would take wholesale reassignment of the most effective teachers to the least advantaged students to substantially reduce inequities in learning outcomes, and that differences in the likelihood of low-income and minority students being taught by a novice teacher contribute a negligible amount to gaps in student achievement. The inequitable outcomes experienced by low-income and minority children may have less to do with their teachers and more to do with the supports and resources available to children of greater means.
Which Students Have High-Quality Teachers?
If low-income students were more likely to have less-effective teachers year after year, key questions would include how the effects of those teacher assignments accumulate over time and what contribution that would make to the student achievement gap. To explore these questions, we developed the “effective teaching gap” calculation, which measures average differences between low- and high-income students in access to effective teachers and can be extended to answer questions beyond the average gap in one year.
Data: We focus on the five-year period from 2008-09 to 2012-13, using data on teachers and students in 26 medium and large school districts. The districts are located in 15 states, distributed across all four Census regions, and operate in different geographic areas and under different conditions. The size and geographic diversity of our sample ensures that our results will not be influenced by idiosyncratic conditions in a single district or state and permits us to assess regional variation in access to effective teachers. We look at data on reading and math teachers in grades 4 to 8, students’ scores on statewide tests in grades 3 through 8, and student characteristics such as race and free or reduced-price school lunch status. Our data allow us to track teacher effectiveness from 4th to 8th grade in 12 districts. In the others, we track teacher effectiveness from 6th to 8th grade.
The students in our sample are more likely than average to live in cities and be low-income or Black or Hispanic. Some 69 percent live in large cities and 63 percent qualify for free- and reduced-price school lunch, compared to 46 percent and 53 percent of U.S. students nationwide, respectively. Forty-two percent of students are Hispanic and 29 percent are Black. On state assessments, the average student in our sample scores at the 45th percentile in English and the 46th percentile in math.
Student achievement gaps by family income mirror those at the national level. Among 8th-grade students, the typical low-income student performs at the 36th percentile on reading state achievement tests compared to the 63rd percentile for the typical high-income student, a gap of 0.68 standard deviations of student achievement. In math, the difference is 24 percentile points, or 0.63 standard deviations. In 4th grade, the student achievement gaps are slightly larger. In reading, the gap is 28 percentile points, or 0.72 standard deviations. In math, the gap is 29 percentile points, or 0.74 standard deviations.
Among the teachers in our sample, we find substantial variation in effectiveness and interaction with low-income students. The standard deviation of teacher effects is 0.13 in reading and 0.20 in math, on average. In other words, an average student with a teacher in the 90th percentile for effectiveness in reading could expect to score at the 57th percentile on an end-of-year state test. If that average student were assigned to a teacher in the 10th percentile for effectiveness in reading, the student could expect to score in the 43rd percentile. In math, this student might expect to score at the 60th percentile with a highly effective teacher compared to the 40th percentile with a minimally effective teacher.
Some 23 percent of teachers in our sample work in high-poverty schools where at least 90 percent of students qualify for free or reduced-price school lunch. Another 39 percent teach in schools where 60 percent to 90 percent of students qualify, and 38 percent teach in low-poverty schools where less than 60 percent of students qualify.
Method: Our effective teaching gap calculation starts by estimating individual teachers’ value added to student achievement as measured by statewide tests. We then link each student to the value-added estimate of the student’s teacher and find the average value added of teachers of low- and high-income students in each district. Finally, we subtract the average value added of teachers of low-income students from the average value added of teachers of high-income students.
Our analysis of teachers’ value added accounts for a range of student characteristics, including limited English proficiency, special education status, race, gender, and whether a student transferred across schools during the year. We also account for three types of potential peer effects: the average achievement of students in the classroom at the end of the prior school year, the amount of variation in student achievement within the teacher’s classroom, and the proportion of students in the classroom who are eligible for a free- or reduced-price lunch. We do this to account for the possibility that the characteristics of others in the classroom, such as their prior academic achievement, influences a student’s performance independent of the quality of the teacher.
We then calculate how the cumulative effect of the effective teaching gap translates into changes in the student achievement gap over multiple years. This takes into account the student’s incoming achievement level, contribution of family and other out-of-school factors, and the fact that the impact of an individual teacher’s effectiveness fades over time. We estimate the extent of this fade-out using estimates from the value-added model of how students’ test scores from the prior year are related to their test scores in the current year. We also estimate how student achievement gaps would change if low- and high-income students had equally effective teachers between Grades 4 and 8 (or between Grades 6 and 8, depending on what data are available).
Finally, we investigate the extent to which disproportionality in rates of placement with novice teachers could lead to greater inequity for low-income students, by documenting the proportion of teachers with less than three years of experience working at high-poverty schools, where at least 90 percent of students qualify for free and reduced-price school lunch. We compare that to the proportion of novice teachers at schools where less than 60 percent of students qualify for meal subsidies. We also examine the average difference in value added between novice and veteran teachers.
Results
Low-income students have less-effective teachers than high-income students, on average, but the differences are exceedingly small. The effective teaching gap is 0.005 standard deviations of student achievement in reading and 0.004 standard deviations in math. The average teacher of a low-income student is just below the 50th percentile of teacher effectiveness, while the average teacher of a high-income student is at the 51st percentile.
Black students also have teachers who are less effective than those who teach white students, on average, but only in math. The effective teaching gap in that subject is 0.01 standard deviations. We find no gap in teacher effectiveness in reading. In both subjects, there are no significant differences between teachers of Hispanic and white students, or between teachers of English learners and students who are not English learners.
Despite these broad similarities, pockets of inequity in access to effective teachers could exist within the study districts. To explore this possibility, we examine the likelihood that low- and high- income students are taught by teachers across the distribution of effectiveness. Here, we also find small or no differences (see Figure 1). In both subjects, 10 percent of low- and high-income students have one of the most effective teachers, on average. In looking at the least effective teachers, 10 percent of both low- and high-income students have such teachers in math. In reading, 10 percent of low-income students and 9 percent of high-income students have one of the least effective teachers.
We also investigate the effectiveness of the average teacher across schools with different poverty levels and find relatively small differences. We group schools into 10 categories based on their proportion of low-income students and calculate the average value added of their teachers. These range from 0.02 to −0.01 standard deviations across the school poverty categories for reading and from 0.03 to −0.02 standard deviations for math. In addition, there was no pattern of average value added decreasing as school poverty rates increased, although teachers in the lowest-poverty schools have the highest average value added, at 0.02 to 0.03 standard deviations.
Overall, our results indicate fairly equitable access to effective teachers. While the most effective teachers boost student achievement substantially relative to the least effective teachers, high-income students are not consistently taught by more effective teachers than low-income students. Instead, both low- and high-income students are taught by a mix of more effective and less effective teachers.
Access and the Achievement Gap
The absence of large effective teaching gaps in the districts we study implies that closing those gaps would have little effect on achievement outcomes. To demonstrate this, we first model the impact of all low-income students having teachers who are at least as effective as those of high-income students, from 4th through 8th grade. We find it would have relatively little effect.
The typical low-income 8th grader performs at the 35.4 percentile in reading while the typical high-income 8th grader is at the 60.5 percentile—a difference of 25.1 points. In math, the gap is 24.5 points. We estimate that if low-income students had teachers at least as effective as those of high-income students in grades 4-8, the student achievement gap would shrink to 24.2 points in reading and 22.3 points in math. If low-income students had teachers at least as effective as those of high-income students in grades 6-8, the student achievement gap would shrink by one percentile point or less in both subjects.
What if low-income students had more effective teachers than high-income students? To cut average income-based differences in achievement in half between 4th and 8th grade, districts would need to have an effective teaching gap of -0.102 standard deviations instead of 0.005. (A negative effective teaching gap means that low-income students have more effective teachers than high-income students.) To accomplish that, 30 percent of reading teachers would have to switch places with one another. In math, the effective teaching gap would need to be -0.080 standard deviations instead of 0.004, which would require that 11 percent of math teachers trade classroom assignments. These reductions in the achievement gap would only occur if the best teachers in classrooms with mostly high-income students were to systematically switch places with the worst teachers in classrooms with mostly low-income students.
Even though there is relatively little inequity in students’ access to effective teachers on average, there could be individual districts with greater inequity than others. We explore this possibility and find modest variation at the district level, with effective teaching gaps ranging from -0.024 to 0.023 standard deviations in reading and from -0.050 to 0.040 standard deviations in math. In other words, there are some districts where low-income students have less-effective teachers than high-income students, on average, and other districts where the opposite is true.
This raises the question of whether certain types of district characteristics are associated with greater inequity in access to effective teachers. We look at a variety of characteristics and find two that are significantly related to the effective teaching gap in both math and reading: district size and region. Districts that are larger and located in the southern United States tend to have a less equitable distribution of teachers compared to other districts. These findings are related, as districts in the South tend to be larger than those in other regions. Low-income students’ access to effective teachers is not consistently related to the other district characteristics we consider, such as the student achievement gap, the extent to which high- and low-income students are separated across schools, or the percentage of Black, Hispanic, and white students in the district. In reading, the effective teaching gap is significantly larger in districts with a greater percentage of low-income students and those with a greater percentage of minority students, but these relationships are not evident in math.
Novice Teachers
Across the study districts, 18.3 percent of teachers in high-poverty schools are novices compared with 8.9 percent of teachers in low-poverty schools. Novices are less effective than veteran teachers on average, with 0.022 lower average value added. However, we find that the presence of more novice teachers in high-poverty schools does not create substantial inequity, for two reasons.
First, although there are more low-income students in high-poverty schools than average, these schools still enroll a mix of low- and high-income students. The substantial difference between the prevalence of novice teachers in low- and high-poverty schools does not translate to a substantial difference between high- and low-income students in the probability of having a novice teacher.
When calculated at the student level, the difference between the likelihood of being taught by a novice teacher is modest, at four percentage points. Some 14 percent of low-income students and 10 percent of high-income students are taught by novices. In other words, 86 percent of low-income students and 90 percent of high-income students are taught by veteran teachers.
Second, the average difference in the effectiveness of novices and veteran teachers is also modest. Thus, even if all low-income students were taught by novices and all high-income students were taught by veteran teachers, the effective teaching gap would be 0.022 standard deviations. The actual difference in the proportion of students taught by a novice teacher is only 4 percentage points. Therefore, the component of the effective teaching gap resulting from low-income students being taught more frequently by novice teachers is approximately 4 percent of 0.022 standard deviations, or slightly less than 0.001.
Implications
Our results show that low-income and minority students have equal or nearly equal access to effective teachers in the great majority of the public school districts we analyze. While individual teachers differ substantially in their effectiveness, both high- and low-income students have a mix of the most effective and the least effective teachers. As a result, providing the two groups of students with equally effective teachers—even over a period of five years—would not substantially reduce the student achievement gap in most districts. Similarly, the disproportionate number of novice teachers at high-poverty schools contributes almost nothing to the effective teaching gap, and, by extension, to the student achievement gap.
The findings of our study—based on a cross-section of medium and large public school districts throughout the United States—suggest that a policy emphasis on correcting for an unequal distribution of “ineffective, out-of-field, or inexperienced teachers” (as required by the federal Every Student Succeeds Act of 2015) is misplaced. Value-added estimates identify effective and ineffective teachers in all types of schools. Student test-score data show that high- and low-income students are far apart in their achievement by the end of 3rd grade, and that this achievement gap grows little due to inequitable access to effective teachers.
It may not be reassuring that public schools are just holding the line on a set of unequal outcomes instead of decreasing them. However, public schools are financed and managed within a political system. Our simulation results suggest that it may be difficult to jolt this system and bring about a substantial decrease in achievement gaps through teacher mobility alone. This is not to concede that policymakers need to accept the status quo. But the best policy response likely resides outside the realm of teacher recruitment, school assignment, and retention. Although a well-planned and well-executed set of human capital policies can improve teacher effectiveness overall, that approach alone is not likely to diminish the student achievement gap.
Rather, our results might nudge policymakers to consider a broad spectrum of other cost-effective, evidence-based policies. For example, experimental evidence supports the expansion of tutoring. In addition, well-implemented early-learning programs may disrupt the predictability of student achievement gaps that are already apparent when children enter school. Other experimental evidence demonstrates that coaching teachers can boost students’ literacy levels in the early grades (see “Taking Teacher Coaching to Scale,” research, Fall 2018).
A half-century ago, James S. Coleman’s landmark “Equality of Educational Opportunity” report to Congress declared “differences between schools account for only a small fraction of differences in pupil achievement.” With more sophisticated methods, easier access to data, more computational power, and the ability to take the analysis from the school level to the teacher level, we have concluded much the same thing.
Eric Isenberg is senior study director at Westat. Jeffrey Max is principal researcher at Mathematica, where Philip Gleason is senior fellow and Jonah Deutsch is senior researcher. This article is based on the study “Do Low-Income Students Have Equal Access to Effective Teachers?” published in the June 2022 issue of Educational Evaluation and Policy Analysis.
This article appeared in the Fall 2022 issue of Education Next. Suggested citation format:
Isenberg, E., Max, J., Gleason, P., and Deutsch, J. (2022). Estimating the “Effective Teaching Gap” – Students experience unequal outcomes, but mostly equal access to high-quality instruction. Education Next, 22(4), 60-65.
