Poor Results for High Achievers

Education Next Issue Cover

New evidence on the impact of gifted and talented programs



By , and

13 Comments | Print | PDF |

Winter 2012 / Vol. 12, No. 1

For nearly a decade, the No Child Left Behind Act (NCLB) has focused the attention of policymakers and researchers squarely on the achievement of low-performing students, with some apparent success. The math and reading scores on the National Assessment of Educational Progress of the nation’s lowest-achieving 10 percent of 4th and 8th graders have risen sharply since 2000, continuing a trend that began in the 1990s. Yet some may wonder about the potential cost of this focus on higher-achieving students, for whom improvements over the same time period have been modest. Among the questions related to this debate is whether additional programs and resources should be devoted to students on the higher end of the spectrum, those considered gifted.

Three million students in the United States are classified as gifted, yet little is known about the effectiveness of traditional gifted and talented (G&T) programs. In theory, G&T programs might help high-achieving students because they group them with other high achievers and typically offer specially trained teachers and a more advanced curriculum. While previous research indicates that ability grouping is in fact correlated with higher achievement, these findings could be misleading if students placed in high-ability classrooms were likely to be successful for reasons that researchers are unable to measure, such as stronger motivation. To our knowledge, no existing studies offer convincing evidence on the causal effect of G&T programs on student achievement.

Our research begins to fill this gap with two studies of the G&T programs available to high-achieving middle-school students in a large urban school district in the southwestern United States which, to preserve anonymity we shall refer to as LUSD. Since 2007, all 5th-grade students in LUSD have been evaluated to determine eligibility for gifted and talented programs starting in 6th grade. Those students who are deemed eligible often are grouped in classes with other gifted students. They are also permitted to apply for admission to two middle schools that have oversubscribed magnet G&T programs.

The two studies use different methods to ask distinct but closely related questions. The first exploits the fact that eligibility for G&T programming in LUSD is determined by a well-defined cutoff in students’ evaluation scores. By comparing students who score just above the cutoff to those who score just below, the study provides evidence on the effect of enrollment in a G&T program on achievement for those students on the margin of eligibility. The second study takes advantage of the randomized lotteries that determine admission to the district’s two premier magnet G&T programs. By comparing students who win the lottery and attend the magnet G&T schools to those who lose the lottery and attend other “neighborhood” programs, the research provides evidence on whether the magnet G&T programs provide any additional benefits.

The results of both studies will be discouraging for those hopeful that current G&T programs provide a means to accelerate the progress of our most capable students. The first shows that barely eligible students who participated in LUSD’s G&T curriculum for all of 6th grade and half of 7th grade exhibit no significant improvement in test scores across a range of subjects, despite their being surrounded by higher-achieving peers and taking more advanced courses. The lottery study corroborates these results, as students admitted to the G&T magnet schools show little improvement in test scores by 7th grade, despite having higher-achieving peers and being taught by more effective teachers. The lone exception is in science, where students admitted to G&T magnet schools performed at substantially higher levels.

It is difficult to know what accounts for these puzzling results. Our best guess, which we discuss in detail below, is that being placed with higher-achieving peers is not all that it is cracked up to be. Students admitted to both types of G&T programs suffer a large drop in their relative rank in terms of grades within their classes, which could have adverse consequences that offset any benefits of improvements in their educational environment. But we are getting ahead of ourselves. Let’s first take a closer look at the programs and the evidence on their effects.

Gifted Students in LUSD

LUSD is a large school district, with more than 200,000 students. The district is heavily minority and very low income; the minority population is more heavily Hispanic than African American. All LUSD students are evaluated for placement in middle-school G&T programs during 5th grade, including those who participated in the district’s G&T program in elementary school. In order to be deemed eligible for the middle school G&T program, a student must meet the eligibility criteria set forth in the “gifted and talented identification matrix.” The matrix converts scores on standardized tests—the Stanford Achievement Test for English-speaking students and the Aprenda exam for Spanish-speaking students with limited English proficiency—scores on the Naglieri Nonverbal Ability Test (NNAT), average course grades, teacher recommendations, and indicators for socioeconomic status into an overall index score.

While all students who meet these requirements qualify for the G&T program, not all end up being classified as G&T, because parents are allowed to opt out. Some students also enroll in the program initially but later withdraw. Schools in LUSD have a monetary incentive for attracting gifted students, as LUSD provides a funding boost of 12 percent over the average allotment for a regular student.

Gifted students in LUSD are far less likely to be economically disadvantaged and more likely to be white or Asian than other students in the district. They also perform at far higher levels on the Stanford Achievement Tests, which the district administers annually in five subjects: math, reading, language, social science, and science. Their advantage in math and reading test scores in 5th grade is roughly 0.7 of a standard deviation, which amounts to well over two years of academic progress (see Figure 1). By the time the same students have reached 7th grade, these gaps have widened to 1.5 standard deviations in math and 1.25 standard deviations in reading. While this pattern suggests that the students enrolled in the district’s G&T programs learn at a faster rate between 5th and 7th grade, it does not necessarily mean that the G&T programs are the cause. It is to that question we now turn.

Effects on Barely Eligible Students

Our first study examines the effects of participation in a G&T program on students who were just barely eligible to participate based on their overall index scores. We focus on students who were evaluated for G&T eligibility as 5th graders in the spring of 2008 for whom we are able to observe outcomes as 7th graders in the 2009–10 school year. Our outcome measures include Stanford Achievement Test scores and attendance rates, both of which are drawn from administrative data provided by the district. After restricting the sample to students near the G&T eligibility cutoff, we are able to examine these outcomes for roughly 2,600 students.

The method used in the study, known as regression discontinuity analysis, takes advantage of the fact that the district uses a strict numerical cutoff in the index score assigned to students as 5th graders in order to determine their eligibility to participate in the G&T program the following year. Because the students are unable to precisely manipulate their index scores, those scoring just below the eligibility cutoff should be very similar to those scoring just above the cutoff. We can therefore attribute any differences in student outcomes on either side of the cutoff to the effect of having being deemed eligible.

As noted above, not all eligible students end up participating in G&T programs due to factors such as a parent’s decision to opt out. Similarly, some students who do not initially qualify later become eligible through an appeals process that allows parents to submit an alternative standardized test score or through additional evaluations conducted in 6th grade. As a result, we use standard statistical techniques to account for the fact that the cutoff our regression discontinuity analysis exploits is “fuzzy” rather than sharp. This allows us to provide evidence on the effects of actual participation in the G&T program, not simply eligibility for it.

Before looking at student outcomes, we first used the same method to confirm that participation in the district’s standard G&T programs led to measurable differences in students’ educational experiences. Clearly, it did. The average achievement of the peers in G&T students’ classrooms were between 0.25 and 0.33 of a standard deviation higher in each core academic subject. Participation in the G&T program also increased the number of advanced courses in which students enrolled in 6th and 7th grade. We found no evidence, however, that the teachers to whom students in the G&T program were assigned were any more effective, as measured by their impact on student test scores.

Did these improvements in peer characteristics and curricular rigor translate into improved outcomes? Our results indicate that they did not (see Figure 2). Our estimates of the effects of G&T participation for barely eligible students are close to zero in all five subjects and are sufficiently precise to allow us to rule out with 90 percent confidence effects as small as 0.04 standard deviations (sd) in math, 0.07 sd in reading, 0.12 sd in language, 0.10 sd in social studies, and 0.19 sd in science. We also looked at the impact of G&T participation for specific student subgroups defined by gender, race/ethnicity, socioeconomic status, and whether the students had been classified as gifted in elementary school. We found little evidence of differential impacts for students in any of these groups.

The Effects of G&T Magnet Programs

Why does the G&T program in LUSD not yield benefits for students on the margin of eligibility? One reason could be that the qualification boundary is set so low that such students are not able to take advantage of the programs’ purported benefits. Our second analysis, which uses experimental research methods to study the effects of enrollment in the district’s G&T magnet programs, is intended to shed light on this concern.

LUSD has 41 middle schools, of which 8 have G&T magnet programs, and 2 of these are oversubscribed. As a result, the district uses lotteries to determine which students will be admitted as 6th graders. Our analysis compares the performance of students who win the lottery and attend one of the G&T magnet programs to those who lose the lottery and either attend a neighborhood G&T program in the district, a magnet school based on a different specialty, or a charter school. Because the lottery is random, any differences in outcomes between lottery winners and losers can be attributed to the effect of enrolling in the G&T magnet program rather than one of these alternatives. Moreover, the results of this analysis will apply to the entire population of students who chose to apply.

Our lottery analysis is based on the sample of LUSD 5th-grade students determined to be eligible for G&T programs in 2007–08 who applied for admission to one of the two middle schools with an oversubscribed G&T magnet program. This group includes 542 students, 394 of whom were offered admission and 148 of whom were not. We find no statistically significant differences in the observed characteristics of lottery winners and losers, suggesting that the lotteries were in fact conducted in a random way.

The students in the lottery differ both academically and demographically from the students who were included in the regression discontinuity study. Not only do the lottery students have higher test scores than students at the eligibility cutoff, but their test scores exceed those of the average G&T student in the district. Lottery participants are also less likely to be on subsidized lunch, and less likely to be minority.

Of the 542 lottery participants, only 440 students, including 331 winners (84 percent) and 109 losers (74 percent), remain in LUSD by 7th grade. Fortunately, the observed characteristics of lottery winners and losers who remain in the district continue to be very similar. Even so, when analyzing the data we control for students’ demographic characteristics and prior achievement, and use weights designed to make the final sample comparable in terms of its observed characteristics to the set of students that initially applied for the lottery.

One disadvantage of this second study is that the lottery losers have a range of alternative experiences and most participate in standard G&T programs, so the comparison group’s educational experience is less clear than it was in the regression discontinuity analysis. Nonetheless, our data confirm that students admitted to the G&T magnet schools with lotteries seem to have experienced large improvements in their educational environment. Winning the lottery increased the average achievement of students’ classroom peers by as much as a full standard deviation in some subjects. And in contrast to the G&T program as a whole, students admitted by lottery to G&T magnet program were assigned to more effective teachers.

Turning to student outcomes, however, our results provide little evidence that attending a G&T magnet program leads to improvements in student achievement (see Figure 3). The one exception is science test scores, for which we estimate a positive effect of 0.28 standard deviations. Due to the relatively small sample sizes, all of the effects are imprecisely estimated and do not allow us to definitively rule out reasonably large positive effects. Even so, the estimated effects for math, reading, and social studies are negative, and the estimated effect for language is effectively zero.

Discussion

It is difficult to understand why we find little evidence that G&T programs positively affect achievement. A common concern with studies of high-achieving students is that the available achievement measures may not be well suited to discern improvements for this group. This would be particularly worrisome if we were using a state accountability exam targeted toward low-achieving students, but it is less of an issue with the Stanford Achievement Tests. Indeed, we found little evidence of students performing near the maximum levels on these tests in either the regression discontinuity or lottery samples. Although it is possible that the additional course material taught in G&T classes is poorly aligned with topics covered in the achievement test, research documenting the benefits of being placed with higher-ability peers suggests that we should see improvements, even if that were the case.

The effect of being placed in a higher-ability classroom may not necessarily be positive, however, especially for a marginal G&T student. In particular, the drop in ranking relative to one’s peers may have a negative effect: a marginal G&T student is likely to go from being near the top of the regular class to being near the bottom of the G&T class. Even students in the middle of the G&T distribution are likely to experience a loss of ranking in the magnet G&T schools as compared to their neighborhood schools. It may be that students are demoralized by the drop in their relative rankings or that teachers provide more resources to students at the top of the class.

Substantial evidence from educational psychology indicates that students who are placed in higher-achieving groups can suffer psychological harm. A commonly used measure is a student’s “self-concept,” how a student perceives her abilities relative to an objective measure such as achievement. A 1995 study by Herbert Marsh and colleagues compared G&T students to observably similar students in mixed G&T and non-G&T classes and found that G&T students show declines in their math and reading self-concept. More recent research has documented lower self-concept and greater test anxiety among gifted students in ability-segregated classrooms.

Although we do not have direct evidence on student confidence, we can make use of student course grades and rank within the class to probe for evidence consistent with this kind of effect. We evaluate the impact of G&T program enrollment in the regression discontinuity study and of attending a G&T magnet in the lottery analysis. In both cases, we find clear reductions in student grades. For the regression discontinuity sample, grades fall by a statistically significant 4 points out of 100 (3 points changes a grade from a B+ to a B, for example) in math and by 2 to 3 points in other subjects, although these effects are not statistically significant for 7th grade. For the lottery analysis, the grade reductions are even more dramatic, with drops of 7 points in math, 8 in science, and 4 in social studies.

It is also useful to consider how students’ rankings within their peer groups differ by treatment status, as this provides a direct measure of how a student may perceive his position in the overall distribution of student ability. We assume that students mostly compare themselves to their schoolmates who take the same courses in the same grade. Thus, we rank students within each school, grade, and course by their final course grades and then convert these rankings to percentiles. The rankings based on 7th-grade courses exhibit notable drops when students cross the G&T eligibility threshold. Controlling for race, gender, economic disadvantage, LEP (Limited English Proficiency), and prior gifted status, marginal G&T students have a relative rank in 7th grade that is 13 to 21 percentiles lower than similar students who were not admitted. Attending a premier G&T magnet in 7th grade generates a nearly 30 percentile ranking drop in all four of the courses examined.

In short, the necessary conditions are clearly met for a drop in relative ranking to play a role in offsetting the expected positive impact of more rigorous courses, more effective teachers, and higher-achieving peers. The possibility that G&T students are subject to such a mechanism suggests potential constraints on the benefits of programs that provide more similar peers and an increase in traditional education inputs.

One should not conclude from the lack of achievement results, however, that the G&T programs should be scuttled. Our analysis occurs in a district with a large number of relatively high-quality magnet programs, and thus the alternatives to the G&T programs may be strong. There may also be benefits that we are not able to capture, such as impacts on SAT scores, graduation rates, and college attendance. Further, our study examines a G&T program in one district. Certainly, districts vary in the approaches they take to educating gifted students, so it may be that similar studies of programs in other districts would yield different results. Nonetheless, this study does raise questions about the efficacy of G&T programs and the traditional model of ability-segregated classrooms.

Sa Bui is a doctoral candidate in economics at the University of Houston, where Steven Craig is professor of economics and Scott Imberman is assistant professor of economics.




Comment on this article
  • [...] school program for high achievers? Sa Bui, Scott Imberman, and Steven Craig’s Education Next study will encourage you to revisit the idea.  The University of Houston researchers conducted two [...]

  • John says:

    Great idea to research Gifted Programs and get some data on them, but this is framed very poorly.

    To me, your study suggests:

    1. Magnet schools produce results with the gifted (a no-brainer)
    2. Urban schools are under serving their gifted populations

    The tragedy here is that our low income schools are (also) failing our gifted students, just like any other student group. Instead, your article and headline, except for the last paragraph, suggests that gifted programs are ineffective. You can’t extrapolate like that after looking at one failing, urban district. And the effectiveness of these programs shouldn’t necessarily be evaluated with test scores and class grades. Wrong metrics.

    I found my middle-class G&T experience to be the only worthwhile part of my K-8 education. And it was because we didn’t learn anything that would prepare us for standardized testing. Instead we delved into mock trials, psychology and real estate.

    I’m sure this district’s G&T program was inadequate, but I wonder if your method of analysis would correctly identify even the best of gifted programs.

  • [...] This week, University of Houston researchers Sai Bui, Steven Craig and Scott Imberman offered one more example in their Education Next report on lack of progress among top-performing students attending [...]

  • Jack Donachy says:

    Not sure Bui, Imberman and Craig’s study sheds any light on a complex subject. In many large districts, the bar for admissions to G&T classes is set too low to begin with. Latecomers to these programs–students entering after 3rd grade–typically struggle, often withdraw, and seldom catch up to their new peers.
    The authors missed a big chunk when they failed to take a broader, longitudinal look at G&T programs. What I’d like to know is this: what, if any, correlation is there between these programs and college attainment rates?

  • Micha Elyi says:

    Odd that almost no one objects to ability-grouping in school athletic programs. Why the different attitude toward ability-grouping in academics?

    Also, I’m confident that more is known about coaching, motivating, and teaching athletic high-achievers to maximize their abilities and skills than about the same for academic high-achievers. Yet the core purpose of school is supposed to be academics, not athletic training. Strange, isn’t it?

  • Paul Roberts says:

    Low achievers can not advance with discovery learning, group projects and thinking skills [expletive deleted]. Only clear targets and rabid relearning will advance the lower rank. This is also called drill and practice. I have been reading these same results for forty years. Only direct instruction works but the discovery methods gang is never discouraged despite ever more studies showing no improvement. The “thinking skills” advocates need to think some more.

  • [...] while the recent study co-authored by Sai Bui, Steven Craig and Scott Imberman this past October in Education Next report clearly points to that fact. Considering that most school districts aren’t using Value-Added [...]

  • [...] of Houston researchers Sai Bui, Steven Craig and Scott Imberman revealed last October in their Education Next report in their report, the progress among top-performing students often stagnates once they are in [...]

  • Mick says:

    In response to Jack Donachy,

    There have been longitudinal studies examining the role of educational programs for gifted students, albeit mostly for highly or profoundly gifted students, which have found benefits of gifted programs, acceleration, and other modifications to students’ educational paths. Miraca Gross, David Lubinski, and Linda Brody have published quite a few articles in this area.

    On a more personal note, my experiences with G&T programs, as well as AP courses, roughly mirror this article. Many students in courses I took as a student and in courses have since taught struggle and probably should not take the courseloads they take. A student struggling for B’s and C’s in these courses who would otherwise be getting A’s is probably better off taking the regular coursework for A’s.

    However, there are students who desperately need gifted programs and advanced coursework (as the studies above suggest). Profoundly gifted students, some of whom score >700 on sections of the SAT as preteens, learn little in mixed classrooms and benefit from grouping with students similar to themselves both socially and academically. When I was younger, gifted programs and acceleration, as well as the mentorship of scientists, mathematicians, and writers during my early years of schooling, kept me from dropping out of high school and leaving formal education, which would have severely limited later career opportunities.

  • Josephine Pasquarelli says:

    I only know what I have experienced as a parent of a gifted student (IQ 137). Being “accelerated” has been an abject disaster for her. In the pull-out program she was allowed to explore and create. She is not a wonderful linear thinker but she is an awesome divergent thinker. Most of these accelerated programs are for that linear-logical gifted child. For the past 6 years I have heard her tell me she is “stupid”. I cannot convince her otherwise. She has much to offer the world and I am afraid she will waste her talents because she has given up on herself. This is just plain wrong. Not every gifted child is the same.
    I see the problem as people seeing these kids as little machines who get data in and spit data out.
    Please do not discount this researcher. Children are not numbers. We need to stop worshiping numerical data and begin to see people. Beautiful people with wondeful minds that maybe don’t operate the way others minds do.
    What are we doing to the innovaters and artists by pushing them into a mold. Where will we find our artists?

  • [...] afforded to the best or most capable workers, which could turn the workforce into something akin to what’s happening in schools with the segregation of students into gifted and regular learners. This would only exasperate [...]

  • [...] be high-quality teaching and comprehensive, college-preparatory instruction. The fact that recent data suggest that those programs rarely do well by these students makes their value seem questionable. [...]

  • Mary Murphy says:

    I am troubled that this particular study will be used by school districts as an excuse to lump all students together in a heterogeneous classroom for reading and language arts.

    This research doesn’t show what grades in the classroom are based upon. For example, you could have a regular classroom where many of the students are shown to be high achieving because the classroom has a large percentage of As. If you looked more closely at this classroom, you would realize that 1) there is a large amount of easy extra credit provided to boost grades and 2) some of the coursework is likely designed so everyone can achieve (e.g. group project with at least one highly motivated high achiever, art project in a language arts classroom, etc.) In this type of classroom, semester/trimester grades become mostly meaningless.

    A teacher in a high achieving classroom may only minimally use extra credit to boost achievement or may not use it at all on the theory that many students don’t need it. If students move from a regular classroom as described above to a classroom with higher expectations and more demanding material, they will likely be in academic culture shock.

    I suspect grades are often artificially inflated in regular classrooms to avoid having to send so many kids to credit recovery courses in the summer.

    As a parent, I see the benefit of having fluid ability groups (sometimes in separate classrooms) where high achievement is rewarded, not with extra work, but with work predicated on ability. There is no reason for a child who is capable to have to sit in a classroom with children who can’t or won’t work (although I find it very beneficial to have students with a documented disability and IEP with appropriate accommodations in the same classroom and have no objection to this). Otherwise, you have high achieving students becoming the unpaid volunteer in a classroom where they help other kids or become bored and disruptive.

  • Comment on this Article

    Name ()


    *

         13 Comments
    Sponsored Results
    Sponsors

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform

    Sponsors