Video: Brian Gill talks with Education Next
Charter schools have become a popular alternative to traditional public schools, with some 5,000 schools now serving more than 1.5 million students, and they have received considerable attention among researchers as a result.
Most studies focus on the effects of charter attendance on short-term student achievement (test scores), using either data sets that follow students over time (see “Results from the Tar Heel State,” research, Fall 2005) or random assignment via school admission lotteries (see “New York City Charter Schools,” research, Summer 2008) to control for differences between students in charter and traditional public schools. Beyond measuring achievement effects, however, there has been only limited analysis of the impacts of charters on the students who attend them. Even less research has been conducted on the effects of charter high schools specifically, though a large portion of all charter schools in the U.S. serve some or all of the high school grades.
Developing a high school model suited to the 21st-century student has been the Holy Grail of education reform in recent years, absorbing governors, task forces, and vast sums spent on small schools, university-based schools, and concept schools (see “High School 2.0,” features). With roughly 30 percent of American students dropping out before receiving a diploma—a rate that has been stable for several decades—assessing existing alternatives to the traditional high school is an urgent task.
In this study we use data from Chicago and Florida to estimate the effects of attending a charter high school on the likelihood that a student will complete high school and attend college. Given the impact of educational attainment on a variety of economic and social outcomes, a positive result could have significant implications for the value of school-choice programs that include charter high schools. We find evidence that charter high schools in both locations have substantial positive effects on both high school completion and college attendance. Controlling for key student characteristics (including demographics, prior test scores, and the prior choice to enroll in a charter middle school), students who attend a charter high school are 7 to 15 percentage points more likely to earn a standard diploma than students who attend a traditional public high school. Similarly, those attending a charter high school are 8 to 10 percentage points more likely to attend college (see Figure 1). Results using an alternative method designed to address concerns about unmeasured differences between students attending charter and traditional public high schools suggest even larger positive effects. Our main results are comparable to those of some studies which find that attending a Catholic high school boosts the likelihood of high school graduation and college attendance by 10 to 18 percentage points.
Methods
Determining the influence of charter school attendance on educational attainment is difficult because students who choose to attend charter high schools may be different from students who choose to attend traditional public high schools in ways that are not readily observable. The fact that the charter students and their parents actively sought out an alternative to traditional public schools suggests the students may be more motivated or their parents more involved in their child’s education than is the case for students attending traditional public schools. Since these traits are not easily measured, the estimated impact of charter high schools on educational attainment could be biased.
Our main analysis uses two methods to address students’ self selection into charter schools. First, we control for any observable differences between charter and non-charter high school students prior to high school entry. These include factors such as race/ethnicity, gender, disability status, and family income. The most important characteristic included among our statistical controls is 8th-grade test score, which aims to capture differences in student ability and students’ educational experiences prior to high school.
Second, we limit our analysis to students who attended a charter school in 8th grade, just prior to beginning high school. That is, we compare high school and postsecondary outcomes for 8th-grade charter students who entered charter high schools (the treatment group) with outcomes for 8th-grade charter students who entered conventional public high schools (the comparison group). If there are unmeasured student or family characteristics that lead to the selection of charter schools in general, these unmeasured characteristics should be relatively constant among students and families who choose charter middle schools. Unlike other nonexperimental studies of charter school impacts, our study therefore addresses student self-selection into charter schools directly by ensuring that the comparison students as well as the treatment students were once charter choosers.
Charter school 8th graders who went on to attend a charter high school differed from their peers who subsequently attended a traditional public high school in several respects, particularly in Florida, which suggests the importance of taking such differences into account when assessing the effects of charter attendance (see Figure 2). However, there may still be unmeasured differences that explain why one charter 8th grader attends a charter high school while another charter 8th grader attends a traditional public high school. For this reason, we estimate charter school effects by comparing students who are more likely to attend a charter school because they live closer to one to those less likely to attend a charter school because it is less convenient. For many charter middle-school students, attending a charter high school may be infeasible due to the lack of a charter high school within a reasonable distance. Such students make different choices not because of unmeasured characteristics, but because of a factor out of their control: the distance from home to the nearest charter school.
Data
The data required to analyze the impact of charter high schools on educational attainment are substantial. One must have data on school type (charter or public) and test scores of individual students prior to high school, individual-level high school attendance records and exit information, and college attendance after high school. Finally, the jurisdiction studied must have a sufficient enrollment of students in charter high schools to provide reliable results. The areas we analyze, the state of Florida and the city of Chicago, are two of just a handful of places where all of the necessary data elements are currently in place.
The Florida data, which cover the four cohorts of 8th-grade students from the school years 1997–98 to 2000–01, come from a variety of sources. The primary source for student-level information is the Florida Department of Education’s K-20 Education Data Warehouse (K-20 EDW), an integrated longitudinal database covering all public school students in the state of Florida. The K-20 EDW includes detailed enrollment, demographic, and program participation information for each student, as well as reading and math achievement test scores.
As the name implies, the K-20 EDW includes student records for both K–12 public school students and students enrolled in community colleges or four-year public universities in Florida. The K-20 EDW also contains information that allows us to follow students who attend private institutions of higher education within Florida. Data from the National Student Clearinghouse, a national database that includes enrollment data on 3,300 colleges from throughout the United States, is used to track college attendance outside the state of Florida. Any individual who does not show up as enrolled in a two- or four-year college or university is classified as a non-attendee.
High school graduation is measured using withdrawal information and student award data from the K-20 EDW. Only students who receive a standard high school diploma are considered to be high school graduates. Students earning a GED or special education diploma are counted as not graduating. Similarly, students who withdrew with no intention of returning or left for other reasons, such as nonattendance, court action, joining the military, marriage, pregnancy, and medical problems, but did not later graduate, are counted as not graduating.
The Chicago data, which cover the five cohorts of students who were in 8th grade during the school years 1997–98 to 2001–02, were obtained from the Chicago Public Schools. The data include 8th-grade math and reading test scores and information on student gender, race/ethnicity, bilingual status, free or reduced-price lunch status, and special education status. This data set is also linked to the National Student Clearinghouse. High school graduation is determined by withdrawal information from the Chicago Public Schools data. As in Florida, only students who receive a standard high school diploma are considered to be high school graduates.
Results
The raw data on our study population of students who were in charter schools in 8th grade reveal substantial differences in educational attainment between attendees of charter high schools and those of traditional public high schools. In Florida, 57 percent of students who went from a charter school in 8th grade to a traditional public school in 9th grade received a standard high school diploma within four years, compared to 77 percent of charter 8th graders who attended a charter high school. In Chicago, the corresponding high school graduation rates were 68 and 75 percent. Similar differences are found for college attendance. In Florida, among the study population of charter 8th graders, 57 percent of students attending a charter school in 9th grade went to either a two- or four-year college within five years of starting high school, whereas among students who started high school in a traditional public school the college attendance rate was only 40 percent. In Chicago, the gap in college attendance is smaller but still sizable: among the study population of charter 8th graders, 49 percent of students at charter high schools attended college, compared to 38 percent of students at traditional public high schools.
Controlling for student demographics, 8th-grade test scores, English language skills, special education program participation, free or reduced-price lunch status (a measure of family income), and mobility during middle school does not alter the basic patterns of graduation and college attendance seen in the descriptive comparisons. The estimated impact of attending a charter high school on the probability of obtaining a high school diploma is positive in both Florida and Chicago. In Chicago, students who attended a charter high school were 7 percentage points more likely to earn a regular high school diploma than their counterparts with similar characteristics who attended a traditional public high school. The graduation differential for Florida charter schools was even larger, at 15 percentage points. The findings for college attendance are remarkably similar in Florida and Chicago. Among the study population of charter 8th graders, students who attended a charter high school in 9th grade are 8 to 10 percentage points more likely to attend college than similar students who attended a traditional public high school (see Figure 1).
As discussed above, there remains the possibility that unobserved changes occur between 8th and 9th grade that influence both high school choice and subsequent educational attainment. For example, dissatisfaction with performance in a charter middle school that is not captured by test scores (such as discipline issues or a poor fit between the student’s interests or ability and the curriculum being offered) could lead parents to choose to send their child to a traditional public high school. When we correct for this potential bias by examining students who attended charter or traditional public school based on proximity, we continue to find highly significant positive effects of attending a charter high school on both receipt of a high school diploma and college enrollment. The magnitude of the effects is large, roughly double the size of our main results.
This pattern suggests that, among students enrolled in charter schools as 8th graders, it is those who are less likely to graduate who are choosing to attend charter high schools. We can only speculate as to why this is so. It is possible that parents whose children are at risk of dropping out are more likely to choose charter high schools in a belief that the traditional public school environment would make it more likely that their child leaves school early. Alternatively, although we control for free or reduced-price lunch eligibility, it may be the case that low-income families have a stronger preference for charter schools. If so, families with children in charter high schools would be less likely to be able to afford to send their children to college.
Possible Mechanisms
The analyses reported above cannot explain how or why charter high schools appear to produce positive effects on their students’ educational attainment. Our study lacks data on operations and instruction in the charter schools, so we have little opportunity to explore the mechanisms contributing to their success. Nonetheless, we have a few pieces of information that permit exploratory analyses of factors that might play a role.
First, it is worth considering that charter high schools may raise rates of high school graduation and college enrollment directly, or indirectly through improved academic achievement. We attempt to distinguish between these explanations by controlling in the analysis for math and reading achievement as measured in the 10th grade. Controlling for 10th-grade test scores explains about half the graduation differential for charter high schools in Florida but less than 20 percent of the difference in Chicago. And it has an even smaller effect on the results for college enrollment, reducing the estimated effect of charter school attendance by only about 10 percent in both locations. These patterns suggest that the positive effects of charter school attendance on educational attainment are not due solely to measured differences in the achievement of students in charter and traditional public high schools. This result is similar to those found in some studies of Catholic high schools, which suggest larger benefits for attainment than for test scores.
Second, given that charter high schools tend to be much smaller than traditional public high schools, charter school effects might simply be attributable to their smaller size. In order to assess this possibility, we ran the analyses for high school graduation and college attendance again with an additional control for the total number of students attending the school. The results are comparable to those reported above, indicating that the estimated effects of charter high schools are not due to differences in school size.
Third, we consider the possibility that the charters’ success might be related to grade configurations that often differ from those of traditional public schools. In the traditional public school sector in both Chicago and Florida, high schools are almost always separate from middle schools. This is not the case for charter schools. In 2001–02, about 22 percent of charter schools in Florida offering middle-school grades also offered some or all high-school grades. As a result, about 30 percent of Florida charter 8th-grade students attended schools that also offered at least some high-school grades. In Chicago, 40 percent of charter middle schools offered both middle- and high-school grades, and nearly half of the 8th-grade charter students could attend at least some high-school grades without changing schools. This raises the possibility that the measured effects of attending a charter high school on educational attainment could simply reflect advantages of grouping middle and high school grades together, thereby creating greater continuity for students and eliminating the disruption often associated with changing schools.
In order to examine whether charter-school effects might be attributable to eliminating the transition between middle and high school, we restricted the Florida analysis to those students whose 8th-grade charter school did not offer 9th grade and ran our analyses again. For high school graduation, restricting the sample produces estimates that are nearly identical to the original estimates from our main method. Using the restricted sample and our alternative method, the estimates are about 30 percent smaller than when the full sample is employed, but still large. Meanwhile, estimates of the effect of attending a charter high school on college enrollment are even larger using the restricted sample than with the original sample that includes schools offering both 8th and 9th grade. In Florida, grade configuration is not a primary driver of the estimated positive effects of charter high schools on attainment. In Chicago, however, we could not run similar analyses because grade configuration is too strongly correlated with charter status; we therefore cannot rule out the possibility that positive results in Chicago could be partly attributable to eliminating the transition from middle school to high school.
Finally, we examined an interpretive concern arising from the fact that some charter schools in Florida are former traditional public schools that converted to charter status. If conversion schools were better-than-average traditional public schools to begin with, they may be distorting the estimated impact of charters on educational attainment. We calculated separate effects for Florida conversion and non-conversion (“de novo”) charters in Florida. (In Chicago, virtually all of the charter high schools in our sample were de novo charters). We found that although Florida’s conversion charters have significantly greater effects on high school graduation than do de novo charters, the impact of non-conversion charters is still sizable (nearly equal to the estimate in Chicago). For college attendance, the estimated positive impacts of Florida’s de novo charters are statistically indistinguishable from the estimated positive impacts of Florida’s conversion charters.
Conclusions
Although a number of recent studies analyze the relationship between charter school attendance and student achievement, this is the first analysis of the impacts of charter school attendance on educational attainment. We find that charter schools are associated with an increased likelihood of successful high-school completion and an increased likelihood of enrollment at a two- or four-year college in two disparate jurisdictions, Florida and Chicago. The reasons for these large charter-school effects are not clear. There is certainly room for future work to explore how differences in curricula, expectations, peer characteristics, and other factors may cause charter schools to diminish the high-school dropout rate and ease the transition to postsecondary schooling.
Our findings are consistent with some research on the efficacy of Catholic schools, which finds substantial positive effects of attending a Catholic high school on educational attainment. While just a first step, the results presented here and in the Catholic-school literature suggest that school-choice programs that include alternatives to traditional public high schools may reduce high-school dropout rates and promote college attendance.
Kevin Booker is researcher at Mathematica Policy Research, Inc. Tim R. Sass is professor of economics at Florida State University. Brian Gill is senior social scientist at Mathematica Policy Research, Inc. Ron Zimmer is associate professor at Michigan State University. This article is adapted from research reported in Charter Schools in Eight States (RAND Corporation, 2009).
