There is strong evidence that the choice of instructional materials has large effects on student learning—effects that rival in size those that are associated with differences in teacher effectiveness.  For example, in a large-scale methodologically rigorous evaluation of the differential impact of four leading mathematics curricula, second-grade students taught using Saxon Math scored on average 0.17 standard deviations higher in mathematics than students taught using Scott Foresman-Addison Wesley Mathematics.  By way of comparison, the difference in the impact on student achievement of a teacher at the 75^th percentile of effectiveness compared to an average teacher is only 0.11 to 0.15 standard deviations.  But whereas improving teacher quality through changes in the preparation and professional development of teachers and the human resources policies surrounding their employment is challenging, expensive, and time-consuming; making better choices among available instructional materials should be relatively easy, inexpensive, and quick.

Administrators are prevented from making better choices of instructional materials by the lack of evidence on the effectiveness of the materials currently in use.  The vast majority of materials either have no studies of their effectiveness or have no studies that meet reasonable standards of evidence.  Not only is little information available on the effectiveness of most instructional materials, there is also very little systematic information on which materials are being used in which schools.  In every state except one, it is impossible to find out what materials districts are currently using without contacting the districts one at a time to ask.

This scandalous lack of information will only become more troubling as two major policy initiatives—the Common Core standards and efforts to improve teacher effectiveness—are implemented.  Publishers of instructional materials are lining up to declare the alignment of their materials with the Common Core standards using the most superficial of definitions.  The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials, but there is currently no basis to measure the quality of materials.  Efforts to improve teacher effectiveness will also fall short if they focus solely on the selection and retention of teachers and ignore the instructional tools that teachers are given to practice their craft.

In our Brookings Institution report, we show how this problem can be fixed by states with support from the federal government, non-profit organizations, and private philanthropy.  First, state education agencies should collect data from districts on the instructional materials in use in their schools.  The collection of comprehensive and accurate data will require states to survey districts, and in some cases districts may need to survey their schools.  In the near term, many states can quickly glean useful information by requesting purchasing reports from their districts’ finance offices.  Building on these initial efforts, states should look to initiate future efforts to survey teachers, albeit on a more limited basis.

The federal government’s National Center for Education Statistics should aid states in this effort by developing data collection templates for them to use through its Common Education Data Standards (CEDS), and providing guidance on how states can use and share data on instructional materials.  The most recent version of CEDS contains 679 data elements for K–12 education, none of which relate to instructional materials in use.

Organizations with an interest in education reform should support this effort.  For example, the National Governors Association (NGA) and Council of Chief State School Officers (CCSSO) have put their reputations on the line by sponsoring the Common Core State Standards Initiative.  Research based on current and past state standards indicates that this initiative is unlikely to have much of an effect on student achievement in and of itself.  The NGA and CCSSO should put their considerable weight behind the effort to improve the collection of information on instructional materials in order to create an environment in which states, districts, and schools will be able to choose the materials most likely to help students master the content laid out in the Common Core standards.

States facing severe budgetary pressures may be reluctant to undertake new data collection efforts.  Philanthropic organizations such as the Bill and Melinda Gates Foundation and the Lumina Foundation for Education could have a major impact by providing the start-up funding needed to collect data on instructional materials and support the research that would put those data to use.

In 1955, educational psychologist Lee J. Cronbach wrote that “The sheer absence of trustworthy fact regarding the text-in-use is amazing.”  It is more than a half-century later and we still don’t know.  How can we tolerate ignorance on something that is as critical to student learning as instructional materials?

Matthew M. Chingos and Grover J. (Russ) Whitehurst, who are research director and director of the Brown Center on Education Policy at the Brookings Institution, are the authors of Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core.

Fourteen of the 4th graders at Washington, D.C.’s Hope Community Charter School had chosen the right answer—1/3 and 5/15—on a test written for the school by Boston-based Achievement Network (ANet). But 20 chose the wrong answer, and two didn’t answer at all.

So on a bright November afternoon three weeks after the test, Hope’s math specialist, Christine Madison, and two of the school’s 4th-grade teachers huddled over five pages of test-score data assembled for them by ANet. Hope’s Tolson campus serves 420 youngsters in grades PreK–8, almost all of them African American and two-thirds of them from low-income families. It is one of three D.C. charters that are operated by Virginia-based Imagine Schools and are working with ANet. The city’s charter board calls Hope “mid-performing”—about 40 percent of its elementary-school children and 60 percent of its middle schoolers are considered proficient in math and English.

ANet coach, Amrutha Nagarajan, coaches 14 schools for the organization.

The ANet data showed that the children generally understood fractions. But they also showed that many youngsters—including some with otherwise good scores—were unsteady at fractional models, or word problems, which are among the 15 math standards that Washington schools are expected to teach their 4th graders.

The fraction lesson, drawn from the class textbook, apparently didn’t work when the teachers first taught it. So at this half-day data-analysis exercise scripted by ANet and overseen by an ANet coach, Madison and the teachers debated why it failed and plotted how to reteach it. How about using an art project, fraction charts, flip-books, team competitions, they mused. How about reteaching the lesson to youngsters grouped by ability? How about reteaching boys and girls differently?

Think about how you taught the lesson the first time, and then do something different, urged Madison, who grew more exuberant with each new idea. “I think I may not have used enough visual aids,” one teacher finally conceded as Madison beamed.

Learning Curve

John Maycock is the founder of Achievement Network, a nonprofit organization that provides data-analysis training and coaching for school leaders and teachers.

Data-driven instruction began its spread across the country about a decade ago, in the footsteps of the No Child Left Behind requirement that schools administer yearly achievement tests. Those tests didn’t help teachers spot and backfill learning gaps, though. Scores came back after everyone had moved on to the next grade, and anyway, the tests were designed to hold schools accountable for the performance of groups: Did enough English-learners pass, enough African Americans? They were not intended to show which students didn’t understand decimals.

By most accounts, a few charter schools began testing their youngsters more frequently, with the idea that teachers could use those interim results to inform their teaching. “If you pay attention to what students learn and what they don’t, you learn how to teach more effectively,” says Paul Bambrick-Santoyo, whose book Driven by Data is a primer on data-driven instruction.

But on the ground, data-driven instruction has encountered problems. Schools complain that interim assessments produced by publishers aren’t always aligned with curricula, pacing guides, or year-end state tests. The assessments are often too easy, handing schools an unhappy surprise when state test results are posted.

Some districts have taken over the job of producing interim tests, but their data offices have the reputation of taking so long to return results that the information is too old to be of much use. (Ben Fenton of New Leaders for New Schools says he has encountered schools that sidestep their districts by photocopying their kids’ answer sheets and grading the assessments themselves.)

Schools that have tried to develop their own assessments have found the job overwhelming. Jermall Wright, principal of southeast Washington’s Leckie Elementary, told me that his leadership team tried it when they decided that the district’s assessments were inadequate. But writing, scoring, and analyzing the tests took so much time that they quickly abandoned the effort.

In any event, few teacher-education schools include data-analysis training, so many teachers don’t know how to read the data, or don’t have the time to use the information to rethink their lesson plans.

By the mid-2000s, “data was starting to become a hot topic,” says John Maycock, who at the time was completing a master’s degree in the school-leadership program at Harvard’s Graduate School of Education. But “teachers were saying they wanted help” understanding and using it, he adds.

“We started to see that just having access to better data was not enough to drive improvement,” says Joe Siedlecki, a program officer at the Michael and Susan Dell Foundation, which has given $1.7 million to ANet.

Maycock’s solution was to found a nonprofit organization that combines rigorous, standards-aligned assessments; data-analysis training and coaching for school leaders and teachers; guided peer review; and networking across schools. Schools join ANet, pay a fee for its services, and commit their teachers and principals to a four-times-a-year cycle of testing and data review. The model goes beyond traditional professional-development models by linking ANet’s work to each school’s data feedback loop: student achievement results inform the guidance ANet provides.

Coaching the Team

School leaders also agree to carve out time for teachers to look at the data together, and to take part in the cycle of meetings and reviews themselves.

Two days after Hope’s data-analysis meeting, I returned to the charter school to listen as its leadership team reviewed the session with ANet coach Amrutha Nagarajan, a 28-year-old Wellesley- and Harvard-educated former banker. Nagarajan came to Washington as a D.C. Teaching Fellow, resisting pressure from her Indian-immigrant parents to pursue a business career, she says, and now coaches 14 schools for ANet.

Hope had administered its second cycle of interim assessments in math and English-language arts on November 8 and 9 after downloading the tests from ANet’s web site. The untimed tests are given every six to eight weeks and typically take youngsters about an hour, Nagarajan told me. The 4th-grade math test asked 34 questions; the 3rd-grade language-arts test included three readings—a folk tale, a poem, and a nonfiction passage—and 20 questions.

The school’s leadership team had the option to view the year’s assessments well beforehand to be sure the school’s lesson plans and pacing would prepare kids for the district’s year-end tests. Hope doesn’t factor the ANet interim test scores into youngsters’ overall grades, and in their contract with ANet, network schools agree not to use the scores to rate their teachers, a move designed to dampen teacher resistance. School leaders also agree to carve out time for teachers to look at the data together, and to take part in the cycle of meetings and reviews themselves.

After the early-November tests, Hope shipped its completed answer sheets to ANet’s Boston office. Within 48 hours of receiving them, ANet posted the results online, and Hope printed out a set for every teacher. The data tell teachers how their students answered each question, of course, but also how each youngster, the class, and the grade scored on questions aligned to each standard, like dividing whole numbers or identifying details in a reading passage.

The data showed that among Hope’s 5th graders, for example, 88 percent appeared to understand how to find the area and perimeter of rectangles and triangles, but only 26 percent could do the same with circles. Among 8th graders, 65 percent could analyze details and draw conclusions from two reading passages—they did better at nonfiction than fiction—but just 52 percent could identify the author’s main purpose in writing the piece.

In their contract with ANet, network schools agree not to use the scores to rate their teachers, a move designed to dampen teacher resistance.

ANet’s coaching script next called for Nagarajan and the leadership team to go over the results—in ANet parlance, this is a pre-data meeting—and set priorities for a professional development day, or data meeting, two days later. They agreed that Hope’s 8th-grade language-arts teachers would concentrate on how better to teach “author’s purpose,” a D.C. learning standard. Its 6th-grade teachers would focus on “drawing conclusions,” its 3rd-grade teachers on “analyzing details,” and so on, through each grade and subject.

The idea, Nagarajan told me, is for teachers to “go deep on one or two standards” by dissecting four or five test questions each at the data meeting. The goal, she added, is for that kind of item analysis to become part of each teacher’s routine as she becomes more comfortable with data.

Nagarajan—whose teaching experience includes a year in Chennai, India, after the 2004 Indian Ocean tsunami—remained in the background on data meeting day as Hope’s teachers worked on their reteaching plans. But she and ANet provided a clear structure to keep the school’s improvement plans on track.

During the data meeting, teachers pored over a form called an “item analysis template”—downloaded from the ANet web site—that forced them to think through the test questions that had given their kids the most grief. “What were the misconceptions” that led so many students to choose the wrong answer, the form asked them to consider. What groups of students missed the answer? What did students need to know to get it right?

Next, they worked through a “reteach action plan,” also downloaded from ANet. How was the lesson taught originally, the form asked. How and when would it be retaught, and to whom—the whole class, a small group, individual children?

Nagarajan, meanwhile, pressed Hope’s leadership team to meet deadlines and create what she called “follow-up structures.” When Dr. Chloé Marshall, Hope’s high-energy principal, said her teachers would file their reteaching plans that Friday, Nagarajan asked, “By the end of Friday or the beginning of Friday?” When would they do the reteaching, the next step on the ANet agenda, she asked. Those “reteaches” are supposed to be slipped into a compatible lesson so they don’t derail a teacher’s lesson plans and pacing, and target just those kids who need them.

Nagarajan continued: When would Hope retest—a quick two- or three-question quiz in each class—to make sure the new lesson was effective? When would teachers hold their “reflection meeting,” the last step in the assessment cycle, to look at the new results? “Does that make sense? What do you think?” she pressed the leadership team.

At the postdata-day debrief—more ANet parlance—Nagarajan and the school’s leadership team conceded that the English teachers were still learning how to use the ANet data to break down the broad standards into smaller skills, and to figure out which skills their students were lacking. But they also saw progress: teachers were talking more, sharing strategies, and acknowledging the need to teach differently.

“Some teachers were still challenging the test” by laying the blame on bad questions, Nagarajan said. But many more were “owning the data,” insisted Marshall, making the shift from the-kids-aren’t-learning-it to I’m-not-teaching-it. And with that, the discussion moved on to new teaching strategies, new delivery strategies, resources for new lesson plans, and the team’s goals for Hope’s students.

“The object isn’t to teach kids a process” that leads them to the right answer on a test, “but to visualize a problem and solve it,” Madison said to general agreement. “That’s what will help them in real life.”

Meeting a Need

Many teachers were "owning the data," making the shift from the-kids-aren't-learning-it to I'm-not-teaching-it.

John Maycock, who is now 37 and calls himself ANet’s “chief growth officer,” had managed afterschool centers in San Francisco, where he says he became “hooked forever” on education. But his real interest was “to be part of something entrepreneurial. I wanted to start something that was an expressed need from the schools,” he adds.

In 2004, Maycock and his mentor, Marci Cornell-Feist, assembled leaders from 10 Boston charter schools around the idea for Achievement Network. Cornell-Feist is the founder of the High Bar, which helps charter boards with management and governance issues.

The Boston charters had begun using interim assessments to prepare their kids for the year-end Massachusetts Comprehensive Assessment System, or MCAS. But the interim tests from outside vendors weren’t as rigorous as, or even aligned with, the MCAS. “They weren’t setting up the school leaders and teachers for success,” Maycock says.

The charters told him they needed better assessments, better data, and help understanding how to use the information, he says. They wanted a common assessment so they could compare results among themselves and use the data to identify best practices. And they wanted assessments that would serve as an instructional tool and not another gotcha mechanism to punish teachers.

Maycock raised $200,000 in seed money from a Massachusetts foundation, but also asked the schools each to pitch in $5,000 “to make it count,” he says. Schools now pay on a sliding scale: those like Hope that are in their first year and need intensive coaching pay $30,000. That declines to $14,000 a year once schools have been in the network for a few years and need less coaching.

Seven charter middle schools signed up with ANet in the 2005–06 school year, its first. Massachusetts had released the MCAS questions for the first time, and Maycock separated them by standard and skill, dissected them for rigor, and wrote his own interim assessments that mirrored the state exam.

James Peyser, a partner in NewSchools Venture Fund, which has invested $1.4 million in ANet and holds a seat on its board, says ANet’s assessments are remarkable for their rigor, which he adds are aimed at readying kids for college, not just for the state tests.

Three Boston district schools joined in ANet’s second year after catching wind of it. Maycock formed a second network of charter schools in Washington in 2008, and nine D.C. district schools joined the next year with help from the Dell grant. There are now 74 schools in the D.C. network.

New Orleans, Newark, Chicago, New York City, and Nashville-Memphis have since launched networks. There’s a network of three virtual schools, and a Baltimore network is planned for 2012. ANet says that 250 schools with some 70,000 kids were members of its networks in the 2011–12 school year. The organization has revenues of $9 million this school year, including $6 million in school fees.

Testing has expanded from the initial grades 6 and 7 to cover grades 3 through 8; ANet is piloting interim assessments for 2nd graders and a set of science tests. High school interims are more complicated because of wider course offerings, but they are “on our radar to consider—very much so,” Maycock says.

In 2010, ANet won a competitive $5 million Investing in Innovation (i3) grant from the U.S. Department of Education, which it is using, in part, to fund a large randomized study of its impact.

In its own analysis, ANet says the number of its youngsters who scored proficient or above on state tests last year increased by 7 percentage points in English and 4 percentage points in math in Chicago, and by 5 points in English and 3 points in math in New Orleans. Of the six cities for which it reported scores last year, ANet said four made twice the gains in English as the rest of their respective states, and three made double the state gains in math.

In D.C., about 6,600 youngsters in ANet’s charter and district schools took year-end tests in 2011. ANet says those scoring proficient in English increased by 4.5 percent and in math by 9 percent from the year earlier. That translates into 319 more kids passing the language exam and 662 more passing math, numbers Maycock calls “huge.” In just the D.C. district ANet schools, the increases were smaller—4 percent in English and 6.6 percent in math—but still better than the improvement of less than 2 percent posted by district schools that didn’t partner with ANet.

Network Strength

A learning walk explores peer-group feedback, or how to get teachers to help one another figure out how to reteach a troublesome lesson.

The schools in ANet’s original network were a lot alike: urban with high-need populations. Maycock has recently convinced stronger schools to join each network; in D.C., Janney and Horace Mann Elementary Schools, which are among the district’s highest-performing, white-majority schools, joined a network that is generally minority and struggling. The idea is to get charters and district schools, and stronger and weaker schools—schools that don’t generally cross paths—to share ideas and goad each other to improve.

Network schools have access to each other’s grade-level data, they share ANet coaches, and they’re invited to regular “learning walks,” where one network school models a practice for other network members.

A few days after the data-day review, I visited Powell Elementary, a district school in northeast D.C., for a learning walk on peer-group feedback, or how to get teachers to help one another figure out how to reteach a troublesome lesson. Teachers, data and instructional coaches, and a principal from eight widely different schools attended.

The practice Powell was showing off involved having its teachers present their reteaching plans—developed on data day—to a handful of teachers from other grades and specialties. These “critical friends” ask “clarifying questions” about the plan, and then talk it over among themselves. The presenting teachers can take or leave the suggestions without having to defend their lesson plans.

As I listened, a Powell math teacher modeled the process while the visitors leaned in close and tossed out their own ideas. Consider a math competition, said the dean of an all-boys, entirely African American charter school that seemed to have little in common with Powell: “Kids respond well to that.” Identify the 10 words most commonly used in word problems, said a math specialist from a district school that seemed to mirror Powell’s English-learner enrollment.

“I hadn’t thought about using manipulatives” in the lesson, conceded the Powell teacher as the ideas rolled in—and his kids would benefit from a hands-on lesson that burned up some of their energy, he added. After two hours, with the learning walk long ended, a dozen teachers from around the network were still huddled together, still talking lesson plans.

Powell keeps an ANet data wall in its front lobby and records how many youngsters in each class score proficient or advanced in math and in language arts for each ANet assessment cycle. Powell’s parents attend a data meeting when the results come out each cycle, and “all but three or four” regularly attend, principal Janeece Docal told me.

Powell’s highly public use of the data contrasts with that of Hyde-Addison Elementary, a third-year ANet school in D.C.’s swank Georgetown neighborhood, which uses the ANet data only internally. “We see what you know and what you don’t know. We see what we’ve taught you,” principal Dana Nerenberg told me.

Powell links the data discussion to the kids’ future, Docal explained: good ANet scores translate into good scores on the year-end test, which will land the youngsters in the high school and then the college and then the job of their choice. “Education equals freedom,” she said a dozen times over the afternoon.

How schools use the data “depends on the school’s culture,” says Justin Jones, a former Teach For America corps member and recruiter who heads the D.C. network.

Peyser, at NewSchools Venture Fund, says the goal is to help “change and strengthen school culture toward data” until “it becomes the way they do business.”

June Kronholz is an Education Next contributing editor.

Massachusetts supposedly has the best public schools in the United States, and the best of the best are to be found in the affluent Boston suburbs—Belmont, Lexington and Wellesley, for example.

So when these top-flight schools decide that advanced honors courses in physics and chemistry are to be given the same weight in calculating a student’s official grade point average (GPA) as any other course, including cooking, check-book balancing, and make-up algebra, it becomes ever so clear—once again—that the country’s progressive educators have successfully pushed back the forces of school reform.  And it remains no less apparent that these same progressives continue to bash both talent and hard work.

Belmont and Lexington, with Wellesley in hot pursuit, have said that the official GPA shall no longer be boosted if the grades are earned in honors-level courses.  That antiquated practice of recognizing that some courses are more demanding than others creates social divides and denies students genuine course choice, it is thought.

Previously, students who wanted a top level GPA were forced to take the most challenging courses the school had to offer.  Now a student with a perfect GPA can become valedictorian of the class simply by accumulating a set of A’s in any old class whatsoever.

As usual, it’s a student who tells the truth.  “I feel that if you take the harder classes, that should be calculated in your GPA,” the vice president of the Wellesley student council told a Boston Globe reporter.

It is the Wellesley school board that prevaricates. A report from one of its committees told parents that “students who meet the expectation of a course should have a GPA that reflects the grade that they earned.”  (As if earning an A in computer science is the same as one in cooking.) To those who ask questions, school officials say that colleges pay no attention to GPAs anyhow—they look at the actual courses taken.  If it is not an honors course, the student is penalized by the college admissions office, so the change won’t really make any difference to student chances of getting into a good college.  They will need to take the honors courses anyhow.

Left unsaid is the fact that students are being misled when told every course counts the same.

Of course those from sophisticated families will see through the prevarication the education progressives have concocted in the name of social equality.  Those who suffer are only the bright kids from the less sophisticated families who foolishly believe what their school district tells them.

All this would be less painful to watch, were it not for the fact that what is happening in the best schools is inevitably going to shape what occurs elsewhere.

-Paul Peterson

Proponents of later start times, who have received considerable media attention in recent years, argue that many students who have to wake up early for school do not get enough sleep and that beginning the school day at a later time would boost their achievement. A number of school districts have responded by delaying the start of their school day, and a 2005 congressional resolution introduced by Rep. Zoe Lofgren (D-CA) recommended that secondary schools nationwide start at 9:00 or later. Despite this attention, there is little rigorous evidence directly linking school start times and academic performance.

In this study, I use data from Wake County, North Carolina, to examine how start times affect the performance of middle school students on standardized tests. I find that delaying school start times by one hour, from roughly 7:30 to 8:30, increases standardized test scores by at least 2 percentile points in math and 1 percentile point in reading. The effect is largest for students with below-average test scores, suggesting that later start times would narrow gaps in student achievement.

The primary rationale given for start times affecting academic performance is biological. Numerous studies, including those published by Elizabeth Baroni and her colleagues in 2004 and by Fred Danner and Barbara Phillips in 2008, have found that earlier start times may result in fewer hours of sleep, as students may not fully compensate for earlier rising times with earlier bedtimes. Activities such as sports and work, along with family and social schedules, may make it difficult for students to adjust the time they go to bed. In addition, the onset of puberty brings two factors that can make this adjustment particularly difficult for adolescents: an increase in the amount of sleep needed and a change in the natural timing of the sleep cycle. Hormonal changes, in particular, the secretion of melatonin, shift the natural circadian rhythm of adolescents, making it increasingly difficult for them to fall asleep early in the evening. Lack of sleep, in turn, can interfere with learning. A 1996 survey of research studies found substantial evidence that less sleep is associated with a decrease in cognitive performance, both in laboratory settings and through self-reported sleep habits. Researchers have likewise reported a negative correlation between self-reported hours of sleep and school grades among both middle- and high-school students.

I find evidence consistent with this explanation: among middle school students, the impact of start times is greater for older students (who are more likely to have entered adolescence). However, I also find evidence of other potential mechanisms; later start times are associated with reduced television viewing, increased time spent on homework, and fewer absences. Regardless of the precise mechanism at work, my results from Wake County suggest that later start times have the potential to be a more cost-effective method of increasing student achievement than other common educational interventions such as reducing class size.

Wake County

The Wake County Public School System (WCPSS) is the 16th-largest district in the United States, with 146,687 students in all grades for the 2011–12 school year. It encompasses all public schools in Wake County, a mostly urban and suburban county that includes the cities of Raleigh and Wake Forest. Start times for schools in the district are proposed by the transportation department (which also determines bus schedules) and approved by the school board.

Wake County is uniquely suited for this study because there are considerable differences in start times both across schools and for the same schools at different points in time. Since 1995, WCPSS has operated under a three-tiered system. While there are some minor differences in the exact start times, most Tier I schools begin at 7:30, Tier II schools at 8:15, and Tier III at 9:15. Tiers I and II are composed primarily of middle and high schools, and Tier III is composed entirely of elementary schools. Just over half of middle schools begin at 7:30, with substantial numbers of schools beginning at 8:00 and 8:15 as well. The school day at all schools is the same length. But as the student population has grown, the school district has changed the start times for many individual schools in order to maintain a balanced bus schedule, generating differences in start times for the same school in different years.

The only nationally representative dataset that records school start times indicates that, as of 2001, the median middle-school student in the U.S. began school at 8:00. More than one-quarter of students begin school at 8:30 or later, while more than 20 percent begin at 7:45 or earlier. In other words, middle school start times are somewhat earlier in Wake County than in most districts nationwide. The typical Wake County student begins school earlier than more than 90 percent of American middle-school students.

Data and Methods

The data used in this study come from two sources. First, administrative data for every student in North Carolina between 2000 and 2006 were provided by the North Carolina Education Research Data Center. The data contain detailed demographic variables for each student as well as end-of-grade test scores in reading and math. I standardize the raw test scores by assigning each student a percentile score, which indicates performance relative to all North Carolina students who took the test in the same grade and year. The second source of data is the start times for each Wake County public school, which are recorded annually and were provided by the WCPSS transportation department.

About 39 percent of WCPSS students attended magnet schools between 2000 and 2006. Since buses serving magnet schools must cover a larger geographic area, ride times tend to be longer for magnet school students. As a result, almost all magnet schools during the study period began at the earliest start time. Because magnet schools start earlier and enroll students who tend to have higher test scores, I exclude magnet schools from my main analysis. My results are very similar if magnet school students are included.

The data allow me to use several different methods to analyze the effect of start times on student achievement. First, I compare the reading and math scores of students in schools that start earlier to the scores of similar students at later-starting schools. Specifically, I control for the student’s race, limited English status, free or reduced-price lunch eligibility, years of parents’ education, and whether the student is academically gifted or has a learning disability. I also control for the characteristics of the school, including total enrollment, pupil-to-teacher ratio, racial composition, percentage of students eligible for free lunch, and percentage of returning students. This approach compares students with similar characteristics who attend schools that are similar, except for the fact that some schools start earlier and others start later.

The results produced by this first approach could be misleading, however, if middle schools with later start times differ from other schools in unmeasured ways. For example, it could be the case that more-motivated principals lobby the district to receive a later start time and also employ other strategies that boost student achievement. If that were the case, then I might find that schools with later start times have higher test scores, even if start times themselves had no causal effect.

To deal with this potential problem, my second approach focuses on schools that changed their start times during the study period. Fourteen of the district’s middle schools changed their start times, including seven schools that changed their start times by 30 minutes or more. This enables me to compare the test scores of students who attended a particular school to the test scores of students who attended the same school in a different year, when it had an earlier or later start time. For example, this method would compare the test scores of students at a middle school that had a 7:30 start time from 1999 to 2003 to the scores of students at the same school when it had an 8:00 start time from 2004 to 2006. I still control for all of the student and school characteristics mentioned earlier.

As a final check on the accuracy of my results, I perform analyses that compare the achievement of individual students to their own achievement in a different year in which the middle school they attended started at a different time. For example, this method would compare the scores of 7th graders at a school with a 7:30 start time in 2003 to the scores of the same students as 8th graders in 2004, when the school had a start time of 8:00. As this suggests, this method can only be used for the roughly 28 percent of students in my sample whose middle school changed its start time while they were enrolled.

Results

My first method compares students with similar characteristics who attend schools that are similar except for having different start times. The results indicate that a one-hour delay in start time increases standardized test scores on both math and reading tests by roughly 3 percentile points. As noted above, however, these results could be biased by unmeasured differences between early- and late-starting schools (or the students who attend them).

Using my second method, which mitigates this bias by following the same schools over time as they change their start times, I find a 2.2-percentile-point improvement in math scores and a 1.5-point improvement in reading scores associated with a one-hour change in start time.

My second method controls for all school-level characteristics that do not change over time. However, a remaining concern is that the student composition of schools may change. For example, high-achieving students in a school that changed to an earlier start time might transfer to private schools. To address this issue, I estimate the impact of later start times using only data from students who experience a change in start time while remaining in the same school. Among these students, the effect of a one-hour later start time is 1.8 percentile points in math and 1.0 point in reading (see Figure 1).

These estimated effects of changes in start times are large enough to be substantively important. For example, the effect of a one-hour later start time on math scores is roughly 14 percent of the black-white test-score gap, 40 percent of the gap between those eligible and those not eligible for free or reduced-price lunch, and 85 percent of the gain associated with an additional year of parents’ education.

The benefits of a later start time in middle school appear to persist through at least the 10th grade. All students in North Carolina are required to take the High School Comprehensive Test at the end of 10th grade. The comprehensive exam measures growth in reading and math since the end of grade 8 and is similar in format to the end-of-grade tests taken in grades 3–8. Controlling for the start time of their high school, I find that students whose middle school started one hour later when they were in 8th grade continue to score 2 percentile points higher in both math and reading when tested in grade 10.

I also looked separately at the effect of later start times for lower-scoring and higher-scoring students. The results indicate that the effect of a later start time in both math and reading is more than twice as large for students in the bottom third of the test-score distribution than for students in the top third. The larger effect of start times on low-scoring students suggests that delaying school start times may be an especially relevant policy change for school districts trying to meet minimum competency requirements (such as those mandated in the No Child Left Behind Act).

Why Do Start Times Matter?

The typical explanation for why later start times might increase academic achievement is that by starting school later, students will get more sleep. As students enter adolescence, hormonal changes make it difficult for them to compensate for early school start times by going to bed earlier. Because students enter adolescence during their middle-school years, examining the effect of start times as students age allows me to test this theory. If the adolescent hormone explanation is true, the effect of school start times should be larger for older students, who are more likely to have begun puberty.

I therefore separate the students in my sample by years of age and estimate the effect of start time on test scores separately for each group. In both math and reading, the start-time effect is roughly the same for students age 11 and 12, but increases for those age 13 and is largest for students age 14 (see Figure 2). This pattern is consistent with the adolescent hormone theory.

To further investigate how the effect of later start times varies with age, I estimate the effect of start times on upper elementary students (grades 3–5). If adolescent hormones are the mechanism through which start times affect academic performance, preadolescent elementary students should not be affected by early start times. I find that start times in fact had no effect on elementary students. However, elementary schools start much later than middle schools (more than half of elementary schools begin at 9:15, and almost all of the rest begin at 8:15). As a result, it is not clear if there is no effect because start times are not a factor in the academic performance of prepubescent students, or because the schools start much later and only very early start times affect performance.

Of course, increased sleep is not the only possible reason later-starting middle-school students have higher test scores. Students in early-starting schools could be more likely to skip breakfast. Because they also get out of school earlier, they could spend more (or less) time playing sports, watching television, or doing homework. They could be more likely to be absent, tardy, or have behavioral problems in school. Other explanations are possible as well. While my data do not allow me to explore all possible mechanisms, I am able to test several of them.

I find that students who start school one hour later watch 12 fewer minutes of television per day and spend 9 minutes more on homework per week, perhaps because students who start school later spend less time at home alone. Students who start school earlier come home from school earlier and may, as a result, spend more time at home alone and less time at home with their parents. If students watch television when they are home alone and do their homework when their parents are home, this behavior could explain why students who start school later have higher test scores. In other words, it may be that it is not so much early start times that matter but rather early end times.

Previous research tends to find that students in early-starting schools are more likely to be tardy to school and to be absent. In Wake County, students who start school one hour later have 1.3 fewer absences than the typical student—a reduction of about 25 percent. Fewer absences therefore may also explain why later-starting students have higher test scores: students who have an early start time miss more school and could perform worse on standardized tests as a result.

Conclusion

Later school start times have been touted as a way to increase student performance. There has not, however, been much empirical evidence supporting this claim or calculating how large an effect later start times might have. My results indicate that delaying the start times of middle schools that currently open at 7:30 by one hour would increase math and reading scores by 2 to 3 percentile points, an impact that persists into at least the 10th grade.

These results suggest that delaying start times may be a cost-effective method of increasing student performance. Since the effect of later start times is stronger for the lower end of the distribution of test scores, later start times may be particularly effective in meeting accountability standards that require a minimum level of competency.

If elementary students are not affected by later start times, as my data suggest (albeit not definitively), it may be possible to increase test scores for middle school students at no cost by having elementary schools start first. Alternatively, the entire schedule could be shifted later into the day. However, these changes may pose other difficulties due to child-care constraints for younger students and jobs and afterschool activities for older students.

Another option would be to eliminate tiered busing schedules and have all schools begin at the same time. A reasonable estimate of the cost of moving start times later is the additional cost of running a single-tier bus system. The WCPSS Transportation Department estimates that over the 10-year period from 1993 to 2003, using a three-tiered bus system saved roughly $100 million in transportation costs. With approximately 100,000 students per year divided into three tiers, it would cost roughly $150 per student each year to move each student in the two earliest start-time tiers to the latest start time. In comparison, an experimental study of class sizes in Tennessee finds that reducing class size by one-third increases test scores by 4 percentile points in the first year at a cost of $2,151 per student per year (in 1996 dollars). These calculations, while very rough, suggest that delaying the beginning of the school day may produce a comparable improvement in test scores at a fraction of the cost.

Finley Edwards is visiting assistant professor of economics at Colby College.

Birdette Hughey is the 2011 Mississippi Teacher of the Year.

In February 2012, the New York Times took the unusual step of publishing performance ratings for nearly 18,000 New York City teachers based on their students’ test-score gains, commonly called value-added (VA) measures. This action, which followed a similar release of ratings in Los Angeles last year, drew new attention to the growing use of VA analysis as a tool for teacher evaluation. After decades of relying on often-perfunctory classroom observations to assess teacher performance, districts from Washington, D.C., to Los Angeles now evaluate many of their teachers based in part on VA measures and, in some cases, use these measures as a basis for differences in compensation.

Newspapers that publish value added measures no doubt relish the attention they generate, but the bigger question in our view is whether VA should play any role in the evaluation of teachers. Advocates argue that the use of VA measures in decisions regarding teacher selection, retraining, and dismissal will boost student achievement, while critics contend that the measures are a poor indicator of teacher quality and should play little if any role in high-stakes decisions. The Obama administration has thrown its weight squarely behind the advocates, launching a series of programs that encourage states to develop evaluation systems based substantially on VA measures.

The debate over the merits of using value added to evaluate teachers stems primarily from two questions. First, do VA measures work? In other words, do they accurately capture the effects teachers have on their students’ test scores? One concern is that VA measures will incorrectly reward or penalize teachers for the mix of students they get if students are assigned to teachers based on characteristics that VA analysis typically ignores.

Second, do VA measures matter in the long run? For example, do teachers who raise test scores also improve their students’ outcomes in adulthood or are they simply better at teaching to the test? Recent research has shown that high-quality early-childhood education has large impacts on outcomes such as college completion and adult earnings, but no study has identified the long-term impacts of teacher quality as measured by value added.

We address these two questions by analyzing school-district data from grades 3–8 for 2.5 million children, linked to information on their outcomes as young adults and the characteristics of their parents. We find that teacher VA measures both work and matter. First, we find that VA measures accurately predict teachers’ impacts on test scores once we control for the student characteristics that are typically accounted for when creating VA measures. Second, we find that students assigned to high-VA teachers are more likely to attend college, attend higher-quality colleges, earn more, live in higher socioeconomic status (SES) neighborhoods, and save more for retirement. They are also less likely to have children during their teenage years.

Teachers in all grades from 4 to 8 have large impacts on their students’ adult lives. On average, a 1-standard-deviation improvement in teacher value added (equivalent to having a teacher in the 84th percentile rather than one at the median) in a single grade raises a student’s earnings at age 28 by about 1 percent. Replacing a teacher whose value added is in the bottom 5 percent with an average teacher would increase students’ total lifetime incomes by more than $1.4 million for a typical classroom (equivalent to $250,000 in present value). In short, good teachers create substantial economic value, and VA measures are useful in identifying them.

Our findings address the three main critiques of VA measures raised in a recent Phi Delta Kappan article by Stanford education professor Linda Darling-Hammond and her colleagues. We show directly using quasi-experimental tests that standard VA measures are not biased by the students assigned to each teacher. Hence, value-added metrics successfully disentangle teachers’ impacts from the many other influences on student progress. We also show that although VA measures fluctuate across years, they are sufficiently stable that selecting teachers even based on a few years of data would have substantial impacts on student outcomes such as earnings.

Students assigned to high-VA teachers are more likely to attend college, earn more, live in higher socioeconomic status neighborhoods, and save more for retirement.

Data

We draw information from two sources: school-district records on students and teachers, and information on the same students and their parents from administrative data sources such as tax records. The school-district data contain student enrollment history, test scores, and teacher assignments from the administrative records of a large urban school district. These data span the school years 1988–89 through 2008–09 and cover roughly 2.5 million children in grades 3 through 8.

The school-district data include approximately 18 million test scores. Test scores are available for English language arts and math for students in grades 3–8 from the spring of 1989 to 2009. In the early part of the sample period, these tests were specific to the district, but by 2005–06 all tests were statewide, as required under the No Child Left Behind law. In order to calculate results that combine scores from different tests, we standardize test scores by subject, year, and grade. The district data also contain other information on students, such as race or ethnicity, gender, and eligibility for free or reduced-price lunch (a standard measure of poverty).

Our data on students’ adult outcomes include earnings, college attendance, college quality (measured by the earnings of previous graduates of the same college), neighborhood quality (measured by the percentage of college graduates in their zip code), teenage birth rates for females (measured by claiming a dependent born when the woman was still a teenager), and retirement savings (measured by contributions to 401[k] plans). Parent characteristics include household income, marital status, home ownership, 401(k) savings, and mother’s age at child’s birth.

Having spent a single year in the classroom of a teacher with value added that is 1 standard deviation higher increases earnings at age 28 by about 1 percent. If that 1 percent advantage were to remain stable throughout an individual's career, it would add up to about $25,000 in total earnings.

Do Value-Added Measures Work?

Value-added analysis aims to isolate the causal effects teachers have on student achievement by comparing how well their students perform on end-of-year tests relative to similar students taught by other teachers. These comparisons take into account students’ test scores in the prior year as well as their race or ethnicity, gender, age, suspensions and absences in the previous year, whether they repeated a grade, special education status, and limited English status. We also control for teacher experience as well as for class and school characteristics, including class size and the academic performance and demographic characteristics of all students in the relevant classroom and school.

Many other researchers use methods for measuring teacher value added that are similar to ours, so it is not surprising that we obtain similar results. For example, we find that a 1-standard-deviation increase in teacher value added corresponds to increases in student math and English scores of 12 and 8 percent of a standard deviation, respectively. In both subjects, this difference is equivalent to approximately three months of additional instruction.

Can we take this as evidence of teachers’ causal impact on student test scores? Recent studies by economists Thomas Kane, Doug Staiger, and Jesse Rothstein, among others, have reached divergent conclusions about whether VA measures should be interpreted in this way. In particular, critics contend that VA measures are likely to be biased as a result of the way that students are assigned to teachers. For example, some teachers might be consistently assigned students with higher-income parents (which typically cannot be accounted for by school districts when generating VA measures because they do not collect precise data on family income). We implement two new tests to determine whether VA estimates are biased.

Our first test examines whether in fact high-VA teachers tend to be assigned students from more-advantaged families. We calculate an overall measure of parents’ socioeconomic status, combining the parental characteristics listed above. Not surprisingly, parent socioeconomic status is strongly predictive of student test scores, and, looking at simple correlations, we find that less-advantaged students do tend to be assigned to teachers with lower VA measures. However, controlling for the limited set of student characteristics available in school-district databases, such as test scores in the previous grade, is sufficient to account for the assignment of students to teachers based on parent characteristics. That is, if we take two students who have the same 4th-grade test scores, demographics, classroom characteristics, and so forth, the student assigned to a teacher with higher VA in grade 5 does not systematically have different parental income or other characteristics.

This first test shows that any bias in VA estimates due to the omission of parent characteristics that we are able to observe is minimal. The possibility remains, however, that students are assigned to teachers based on unmeasured characteristics unrelated to parent socioeconomic status. For example, principals may consistently assign their most-disruptive students to teachers whom they believe are up to the challenge. Alternatively, principals might assign these same students to their least-effective teachers, whom they are not worried about losing. Our second test seeks to determine the amount of bias introduced by this kind of sorting.

To do so, we exploit the fact that adjacent grades of students within the same school are frequently assigned to teachers with very different levels of value added because of idiosyncrasies in teacher assignments and turnover. During our analysis period, roughly 15 percent of teachers in our data switched to a different grade within the same school from one year to the next, 6 percent of teachers moved to a different school within the same district, and another 6 percent left the district entirely. These year-to-year changes in the teaching staff at a given school generate differences in value added that are unlikely to be related to student characteristics.

To illustrate, suppose a high-VA 4th-grade teacher enters a school at the beginning of a school year. If VA estimates capture teachers’ true impact on their students, students entering grade 4 in that school should have higher year-end test scores than those of the previous cohort. And the size of the change in test scores across these consecutive cohorts should correspond to the change in the average value added across all teachers in the grade. For example, in a school with three equal-sized 4th-grade classrooms, the replacement of a teacher with a VA estimate of 0.05 standard deviations with one with a VA estimate of 0.35 standard deviations should increase average test scores among 4th-grade students by 0.1 standard deviations.

In fact, that is exactly what we find, as shown in Figure 1. To construct this figure, we first define the top 5 percent of teachers as “high VA” and the bottom 5 percent as “low VA.” Figure 1 displays average test scores for cohorts of students in the years before and after a high-VA teacher arrives. We see that end-of-year test scores in the subject and grade taught by that teacher rise immediately by about 4 percent of a standard deviation. This impact on average test scores is commensurate in magnitude with what we would have predicted given the increase in average teacher value added for the students in that grade.

We obtain parallel findings when we examine the departure of high-VA teachers and the entry and exit of low-VA teachers. When a high-VA teacher leaves a given subject-grade-school combination, test scores of subsequent students in that subject, grade, and school fall. Likewise, students benefit from the departure of a low-VA teacher and are harmed by the arrival of a low-VA teacher.

Together, these results provide direct evidence that removing low-VA teachers (bottom 5 percent) and retaining high-VA teachers (top 5 percent) improves the academic achievement of students. But what about the remaining 90 percent of teachers? When we perform a similar analysis for all teachers, we again find that changes in the quality of the teaching staff strongly predict changes in test scores across consecutive cohorts of students in the same school, grade, and subject. Moreover, in middle schools, where students usually learn math and English from different teachers, we confirm that the arrival or departure of math teachers affects math scores but not English scores (and vice versa).

Using these techniques, we can calculate the amount of bias in our VA estimates. We find that the degree of bias is, on average, less than 2 percent. We therefore conclude that standard VA estimates accurately capture the impact that teachers have on their students’ test scores. Although the results could differ in other settings, our method of using natural teacher turnover to evaluate bias in VA estimates can be easily implemented by school districts to evaluate the accuracy of their VA models.

Do Value-Added Measures Matter?

Even though value-added measures accurately gauge teachers’ impacts on test scores, it could still be the case that high-VA teachers simply “teach to the test,” either by narrowing the subject matter in the curriculum or by having students learn test-taking strategies that consistently increase test scores but do not benefit students later in their lives. To address this issue, we measure the relationship between teachers’ value added and their students’ outcomes in adulthood. We compare students who were assigned high-VA vs. low-VA teachers in grades 4–8 and study their outcomes in adulthood.

We find that high-VA teachers raise students’ chances of attending college at age 20 (see Figure 2a). A student assigned to a teacher with a VA 1 standard deviation higher is 0.5 percentage points more likely to attend college at age 20 (an increase of 1.3 percent). Students of higher-VA teachers also attend higher-quality colleges, as measured by the average earnings of previous graduates of those colleges.

A person’s income doesn’t begin to stabilize until their late twenties, so our analysis of earnings focuses on the year when students were 28, the oldest age at which we observe a sufficiently large number of students. We find that having spent a single year in the classroom of a teacher with value added that is 1 standard deviation higher increases earnings at age 28 by $182, or about 1 percent (see Figure 2b). If that 1 percent advantage were to remain stable throughout an individual’s career, it would add up to about $25,000 in total earnings.

In addition to improved earnings, we also find that improvements in teacher value added significantly reduce the likelihood that female students will have a child during their teenage years, increase the socioeconomic status of the neighborhoods in which students live in adulthood, and raise 401(k) retirement savings rates. Moreover, it is likely that improved education would yield benefits that we are not able to measure but have been shown by other studies, such as reduced crime and improved citizenship.

To sum up, our evidence confirms that the students of high-VA teachers benefit not just by scoring higher on math and reading tests at the end of the school year, but also through improved outcomes later in life. The size of these effects may seem small, but recall that they reflect the impact of a higher-VA teacher for a single year and could compound over time to the extent that students are exposed to multiple high-VA teachers. As important, a single high-VA teacher has this effect not only on a single student but rather on an entire classroom—and often on many classrooms of students over the course of a career.

Wanda Booth, Florida's 2011 Charter School Teacher of the Year, works with students. Teachers in all grades have large impacts on their students' adult lives.

Policy Implications

In a recent article (see “Valuing Teachers,” features, Summer 2011), Eric Hanushek argues in favor of dismissing the bottom 5 percent of teachers based on their VA scores. While such a policy would have many costs and benefits that are beyond the scope of our study, we can illustrate the magnitudes implied by our analysis by calculating its impacts on students’ earnings. Our estimates imply that replacing a teacher whose value added is in the bottom 5 percent with an average teacher would increase students’ cumulative lifetime income by a total of $1.4 million per classroom taught. This gain is equivalent to $267,000 in present value at age 12, discounting at a 5 percent interest rate. However, it is important to realize there is uncertainty in VA measures, which are estimates that may be based on only a few classrooms of students, so the gains from removing teachers identified as ineffective based on a limited number of years of data are smaller. We estimate the gains from “deselecting” the bottom 5 percent of teachers to be approximately $135,000 in present value based on one year of data and $190,000 based on three years of data. These benefits, while still large, would have to be weighed against any costs associated with the policy, such as teachers demanding higher pay to compensate them for the risk of dismissal.

We also measure the expected gains from policies that pay higher salaries or bonuses to high-VA teachers in order to increase retention rates. The gains from such policies appear to be only somewhat larger than their costs. Although the benefit from retaining a teacher whose value added is at the 95th percentile after three years is nearly $200,000 per year, most bonus payments end up going to high-VA teachers who would have stayed even without the additional payment. Replacing low-VA teachers is therefore likely to be a more cost-effective strategy to increase teacher quality in the short run than paying to retain high-VA teachers. In the long run, higher salaries could attract more high-VA teachers to the teaching profession, a potentially important benefit that we do not measure here.

While these calculations illustrate the magnitudes of teachers’ impacts on students, they do not by themselves offer a blueprint for the design of optimal teacher evaluations, salaries, or merit-pay policies. Teachers were not evaluated based on test scores in the school district and time period we study. VA measures may not be as useful for identifying teachers with positive long-term impacts on their students if teachers respond to their use in evaluation systems by engaging in practices such as teaching to the test or even outright cheating. In addition, our analysis does not compare value added with other measures of teacher quality, like evaluations based on classroom observation, which might be even better predictors of teachers’ long-term impacts than VA scores.

In summary, our research demonstrates that good teachers are of great value to their students, and that VA measures are a potentially valuable tool for measuring teacher performance. The most important lesson we draw is that finding policies to raise the quality of teaching is likely to yield substantial economic and social benefits.

Raj Chetty is professor of economics at Harvard University. John N. Friedman is assistant professor of public policy at Harvard Kennedy School. Jonah E. Rockoff is associate professor of business at Columbia University’s Graduate School of Business. For further information on the study, see http://obs.rc.fas.harvard.edu/chetty/value_added.html.

Commentary

In light of the widespread attention given to the Chetty, Friedman, and Rockoff research, Education Next asked four experts to comment on the study’s implications for teacher policy.

Implications for Policy Are Not So Clear - By Douglas Harris
Profound Implications for State Policy - By Chris Cerf and Peter Shulman
More Evidence Would Be Welcome - By Dale Ballou
Low-Performing Teachers Have High Costs - By Eric A. Hanushek

Raj Chetty, John Friedman, and Jonah Rockoff have carried out a remarkable study, but I suspect it will be misinterpreted.

The main contribution of their research is quantifying the importance of teaching. Specifically, the authors conclude that students taught by a more effective teacher will collectively earn hundreds of thousands of dollars more over their lifetimes, and that good teachers similarly influence college going and teenage pregnancy. Because each teacher influences thousands of students over a career, this suggests that one excellent teacher could generate enormous social and economic benefits.

I find these results plausible, though there are some real limitations. The researchers present convincing evidence that their estimates of teacher contributions to student achievement are valid and do not simply reflect differences in student background. But this type of “selection bias” could influence effects on earnings and other long-term outcomes. So, the most intriguing findings here are also still somewhat tenuous. Given the small size of the effects for each individual student, even a slight bit of selection bias could dramatically alter the estimated benefits of an individual teacher.

Perhaps the more important question is, what do the results mean for policy? Policymakers had already concluded that we need to do more to improve teaching. As a result, schools and districts around the country are now experimenting with a wide range of policies to improve teacher performance measures and use these to make high-stakes decisions such as dismissing low-performing teachers.

And here is the rub. The authors, recognizing the interest in dismissing low performers, conduct a simulation of such a policy and emphasize these results in their summary. But it would be a mistake to interpret even these careful simulation results as evidence about actual policies. The effects of actual policies never play out the way simulations suggest, because policies are rarely implemented as intended and the inevitable secondary effects are hard to predict.

There are substantial legal, political, and organizational problems associated with dismissing low performers. For example, in a simple system, many teachers would be fired unjustifiably as a result of imprecision in the performance measures—a lawsuit waiting to happen. High stakes associated with the tests will inevitably distort student scores and the assignment of students to teachers, worsening the measurement problem. A more elaborate evaluation system can address this measurement problem, but such systems are costly, and those costs are not considered here. Such an approach could also change the makeup of the profession, in both positive and negative ways.

There is good reason to think that dismissing more low-performing teachers would improve student outcomes, but the Chetty study is not designed to tell us much about that, or about any of the various policy alternatives. What it does provide is the best evidence yet that teachers matter a great deal and that we should continue looking hard for ways to improve teaching and learning in schools.

Douglas Harris is associate professor of educational policy studies at the University of Wisconsin-Madison.

Return to “Great Teaching” (Summer 2012)

Over the last decade, research in public education has led us to three conclusions about the teaching profession: teachers are the most important in-school factor in determining student achievement; there is wide variation in teacher effectiveness; and those differences really matter for kids.

These findings should have profound implications for policymakers and practitioners. Now that we have evidence attesting to the enormous contributions of the most effective educators, if we are truly serious about improving student learning and closing the achievement gap, we must think anew about teacher recruitment, placement, evaluation, professional development, retention, and separation.

Raj Chetty, John Friedman, and Jonah Rockoff have helped advance the conversation through their longitudinal study of 2.5 million students over a 20-year span. The correlation between teacher effectiveness (as demonstrated by value-added student growth measures) and student life outcomes (higher salaries, advanced degrees, neighborhoods of residence, and retirement savings) is staggering; it’s not an exaggeration to say that great teachers substantially improve students’ future quality of life and those students’ contributions to the common good. Conversely, traditional education output measures like student course completion, grades, and diplomas have a substantial degree of subjectivity across schools and districts and can potentially provide a misleading account of a student’s college and career readiness.

In New Jersey, we are assessing where our finite resources are best invested. The Chetty study contrasts the opportunity cost of providing retention incentives to effective teachers with that of investments to attract new teachers. Similar cost/benefit questions arise in relation to shaping teacher-placement strategies, developing career ladders, and providing meaningful professional development. To make informed decisions in these areas, we first need to be able to differentiate among our teachers and, ideally, identify strengths to build on and weaknesses to address. That’s why the foundation of our human-capital efforts is a new educator-evaluation framework that’s substantially based on student learning outcomes. If we are able to assess an educator’s effectiveness accurately, we can improve the array of policies and practices that influence our teachers and school leaders. The hallmark of these efforts in our state will not be based on separating ineffective teachers but rather on using evaluation results to target resources toward improving teaching practice.

New Jersey is still in the early innings of this work. Eleven districts, through a pilot initiative, have joined with the state to create the new teacher-evaluation system. This collaboration has helped jump-start this work across the state and shed light on the many significant challenges associated with overhauling the hoary systems in place, such as measuring student achievement in “untested” grades and subjects, ensuring inter-rater agreement and accuracy of teacher practice observations, and ending the long-standing culture of “The Widget Effect.”

The primary takeaway from this critically important research, as the study authors note, is that “finding policies to raise the quality of teaching… is likely to have substantial economic and social benefits in the long run.” We agree with this conclusion, and New Jersey, like other states, must develop such policies over time through a confluence of national and local research, lessons learned from our classrooms, and an unwavering resolve to provide our students with high-quality teachers.

Chris Cerf is acting commissioner of education for the State of New Jersey. Peter Shulman is chief talent officer for the New Jersey Department of Education.

Return to “Great Teaching” (Summer 2012)

The new study by Raj Chetty, John Friedman, and Jonah Rockoff  asks whether high-value-added teachers (i.e., teachers who raise student test scores) also have positive longer-term impacts on students, as reflected in college attendance, earnings, avoiding teenage pregnancy, and the quality of the neighborhood in which they reside as adults. As a step on the way, the researchers investigate whether such teachers have been properly identified, that is, are the teachers who are producing larger achievement gains from year to year, according to value-added models, actually responsible for those gains? The paper contains valuable evidence indicating that the answer is yes. First, the authors obtain data on family background from federal tax returns not normally available to researchers. This allows them to measure family characteristics (such as parental income) not typically controlled for when teacher value-added is estimated. If introducing such factors reduces the explanatory power of teacher value-added, it is an indication that the value-added estimate was inflated, and that part of what had been attributed to the teacher was in fact due to favorable family circumstances. The study authors find that including such controls does not detract from the explanatory power of estimated value-added.

The authors also investigate whether high-value-added teachers have benefited by being assigned students who would have made greater gains on standardized tests for unobserved reasons (such as family factors that cannot be gleaned even from tax returns). This is normally difficult to do, given the possible influences on the way students are assigned to teachers. The report succeeds by focusing on average test gains in grades within schools where mean value-added within a grade has been affected by the movement of teachers in and out of the grade. What matters for this analysis is not which student was assigned to which teacher within the grade, but how the movement of teachers has altered the quality of teaching in that grade as a whole. It turns out that subsequent gains within these grades are close to those what would be expected from the change in mean teacher value-added. Provided the movement of teachers in and out of a grade has not changed the makeup of students enrolled in that grade, this finding supports the conclusion that measured value-added of teachers is an unbiased predictor of future test-score gains, as there appears to be no other explanation for the resulting improvement in test scores.

When the authors examine the association between teacher value-added and outcomes in young adulthood, however, for the most part they do not undertake the same tests to ensure that these associations are not artifacts of the way students are sorted among teachers. They do not introduce controls from tax returns to see whether the explanatory power of teacher value-added for later earnings, college attendance, and other factors, falls. Nor, with the exception of college attendance, do they test for the influence of unobservable factors in the manner just described.

The omission of such tests undercuts their claim to have demonstrated that high-value-added teachers contribute to better long-term outcomes. Without the same rigorous tests, we cannot be sure that the observed association between teacher value-added and long-term outcomes was not the result of other factors (for example, efforts made by parents with the strongest parenting skills to ensure their children were assigned to the most effective instructors). It is not enough to show that omitted family characteristics have not been confounded with value-added as a predictor of future test-score gains. The factors that shape test performance are not necessarily those that influence future earnings or the avoidance of a teenage pregnancy. Character education and the values parents impart to their offspring are likely to matter for the latter in ways that they do not for cognitive functioning.

In short, the authors provide a persuasive answer to the question: does a high-value-added teacher actually raise subsequent test scores? They have not so far provided equally persuasive evidence answering the question: does a high-value-added teacher improve subsequent life outcomes?

Dale Ballou is associate professor of public policy and education at Vanderbilt University and associate director of the National Center on Performance Incentives.

Return to “Great Teaching” (Summer 2012)

The movie Waiting for Superman chronicles the role of chance in determining the fate of a relatively small number of families trying to enroll their children in oversubscribed charter schools. Raj Chetty, John Friedman, and Jonah Rockoff document the much larger problem of ineffective teachers scattered about a multitude of schools. From the viewpoint of the student, this latter issue may appear to be chance when class assignments are made, and when some get good teachers and others get ineffective ones. From the standpoint of the system, however, it is not chance but mismanagement that allows ineffective teachers to continue harming students.

Chetty et al. have produced new and elegant estimates of how teacher effectiveness relates to long-run student outcomes. As economists are prone to do, they have produced a paper that deals with a long list of technical questions that have absorbed the scientific literature on teacher effectiveness. Their work is thorough, convincing, and scientifically innovative.

The overarching idea of the paper is linking gains from having a high-value-added teacher in grades 4–8 to subsequent long-run outcomes, including college attendance, earnings, and family creation. But, from the outset, they must deal with the two primary challenges leveled at teacher value-added measures based on student test scores. First, are these  estimates biased measures of effectiveness? The answer is no. The wealth of information that Chetty et al. have about families from tax records and some clever analyses effectively rule out the possibility that conventional estimates of value-added based only on school administrative data are misleading. Second, do the effects of good teachers (or bad teachers) quickly fade away? Again, the answer is no. Even as these students leave school and enter into adult careers in their late 20s, the significant trace of their early schooling is quite discernible.

But the warranted attention to this work derives not from its technical aspects but from the policy implications of the results. The fundamental finding is that good teachers have an extraordinarily powerful impact on the future lives of their students. Symmetrically, the researchers show the lasting damage that poor teachers have on the lives of their students. This work sweeps away a variety of attempts to deflect questions about the importance of teacher quality and our ability to identify it. It also brings us back to the question of informed policy.

As the evidence on the importance of teacher quality has grown, policy discussions have actually moved. In the beginning, there were doubts about the impact of teacher quality relative to families, curriculum, or a host of other influences. Those doubts have largely receded and been replaced by questions of how policy should proceed. And here is where the additional evidence presented in the Chetty study comes into play.

Much of the discussion has centered on the political difficulties of reforming the schools by dealing with the problem of the most ineffective teachers. The unions have dug in their heels, resisting any change that does not ensure perfect identification of the worst teachers. Their resistance has resulted in many policymakers simply asserting that it is too politically costly to make active decisions about teacher effectiveness and instead looking to alternatives such as more professional development, better mentoring, or heightened requirements of certification.

Chetty et al.’s evidence shows that bad teachers cost hundreds of thousands of dollars in lost income and productivity each year that they remain in the classroom. These costs are large enough that failing to address them is simply inexcusable. It is time that we develop policies that truly are designed to help our children and not just the adults in schools today.

We have recently seen a number of brave states step out and legislate better evaluations of teachers including, when possible, the use of value-added measures. Coupled with both pay and tenure reforms, these movements show real promise and should be encouraged on a wider scale.

Eric A. Hanushek is senior fellow at the Hoover Institution of Stanford University.

Return to “Great Teaching” (Summer 2012)

As a policy wonk, I push for high academic expectations for all students. I know that American competitiveness requires excellence in subjects such as math and science that our schools do not teach very well. As a father, however, I find that what matters most to me is that my daughters are happy in school.

In Montgomery County, Maryland, where I live, academic expectations are extremely high. Our school district aims to teach math, for example, in a rigorous way. I appreciate this goal, but to date “increased rigor” has primarily meant that some students skip grade-level math classes and enroll in classes meant for older kids. Basic skills that are taught and reinforced in the grades being skipped are often given short shrift. In 2nd grade, my daughter brought home worksheets on probability before she had any real understanding of the concept, or even a strong foundation in simple division. Her frustration with probability, and consequently math, grew as we substituted times-table drills for play dates. Last year, to my horror, she said that she hated math. This year, which has included an increased focus on math facts and an inspiring teacher, math has become her favorite subject.

With my policy hat on, I know that a teacher’s academic background is critical. As a father, however, I want a teacher who manages a calm, safe, and fun classroom, and who loves children. One of the best teachers my children have had is our regular babysitter, who speaks English as a second language and never graduated from high school.

Of course, there are some gems at our school (thank you, Ms. Bederman, now retired) who are knowledgeable, skilled, passionate about learning, and passionate about children. To a father, Ms. Bederman was a gift from heaven; to a policy wonk she is the Holy Grail. Why can’t we identify and train more of these treasures? Why wasn’t every teacher in our school crowded into Ms. Bederman’s classroom to witness her magic? Why didn’t the principal require every teacher to crowd into her classroom?

As a policy wonk, I believe that student learning flourishes in classrooms that include students with a wide range of abilities and backgrounds. As a father, I want my daughters to appreciate diversity of all types. But I also want them to be surrounded by children who come to school ready and eager to learn. These goals come into conflict when some students are constantly disruptive; the policy wonk must preach patience to the father who wants the class disrupter out.

My daughter’s kindergarten class included a troubled boy who was going through the foster-care placement process. He is exactly the type of child that can benefit most from an excellent education, but he regularly disrupted class. One day, when I was in the classroom, the teacher—talented, but inexperienced—spent more than half of her time trying to keep this boy on task.

I feel for children like him; my company works with schools and districts to improve outcomes for these kids. But I was angry. The other children were clearly uncomfortable. His disruptions reduced learning time for my daughter, and seemed to steal some of her innocence and excitement about school.

The tension between my understanding of good education policy—driven by a deep commitment to equity and the belief that an outstanding education can transform lives, and this country—and what is right for my daughters makes me both a better policy wonk and a better father. The tension also illustrates why school reform is so difficult.

Scott Joftus is the president of the education-consulting firm Cross & Joftus.

In 2005, Education Next sent Mark Zanger, a restaurant critic, to Boston schools to report on the state of school lunches. Read “What’s For Lunch” to get the inside scoop. Ron Haskins wrote about the federal school lunch program in “The School Lunch Lobby: A charmed federal program that no longer just feeds the hungry,” in the same issue.

The questions addressed by the panel included:

Is digital learning education’s latest fad or its future?
What fundamental changes to the ways we fund, staff, and govern American schools are necessary to fulfill the technology’s potential?
Will policy tweaks suffice or do we need a total system overhaul—and a big change in the reform priorities that can bring this about?
Who will resist—and do their objections have merit?

If you missed the event, you can watch it above or read more about it here.

For more on this topic from Ed Next, please see

“Bright Spots Shine in Blended, Online Learning,” by Michael Horn

“Can Khan Move the Bell Curve to the Right?” by June Kronholz

“The Flipped Classroom,” by Bill Tucker

Is digital learning education’s latest fad or its future? What fundamental changes to the ways we fund, staff, and govern American schools are necessary to fulfill the technology’s potential? Will policy tweaks suffice or do we need a total system overhaul—and a big change in the reform priorities that can bring this about? Who will resist—and do their objections have merit? Fordham is bringing together experts on all aspects of education policy—from governance to finance to human capital—to examine how policymakers can make digital learning a transformative tool to improve American education…and weigh the dangers that lie ahead.

More information on the events and the panelists can be found here.

In the video, leaders from Rocketship and School of One discuss the advantages of digital learning while sharing their concerns about California laws and union regulations that have limited the role of online learning.

More about the book is available here.

HT: Joanne Jacobs

Anyway, wanted to share one title that’s uber-relevant today. It’s Great Promise Thwarted: The Humbling History of the Common Core, 2008-2018. It’ll be written by my good friend, eminent NYU edu-historian Jonathan Zimmerman, and e-published by Harvard University Press, in 2022.

It’s worth quoting a long excerpt from the book’s conclusion:

For a brief time, during 2010-2012, the success of the Common Core seemed assured. Proponents had compelling arguments. Existing state standards were generally awful. The No Child Left Behind accountability system designed to accommodate variation in state standards and assessments was problematic. Conservative supporters argued that the Core would make it possible to do away with intrusive federal regulations governing accountability and easier to provide transparency and accountability with a light touch. Moreover, the Core would make it possible to credibly compare student and school performance across the nation, while allowing mobile students or those learning online to move across schools or programs with minimal disruption.
Proponents argued that the Core would reduce the barriers that hindered virtual schools, online instruction, and the emergence of “21st century” assessments and instructional tools. Observers generally characterized the standards as a substantial improvement on those in place in most states. And Core proponents enjoyed enormous political muscle. A push that would have been laughable in 2006 seemed a fait accompli by 2010, with forty-plus states on board. The effort enjoyed the enthusiastic backing of the Gates Foundation (what we today would call Gates-ECB; this was before the Foundation absorbed the European Central Bank following the third Greek default), the Obama administration, nearly the whole of the education “reform” community, and Republican leaders including both members of the 2016 GOP presidential ticket. Major publishers and test-developers were quiescent or supportive, while education technology entrepreneurs were enthusiastic.

So, what went wrong? Why is it that today just eleven states use a Common Core assessment, less than a third of the states are judged to have made any effort to adhere to the Core, and the phrase “Common Core” remains polarizing and generally unpopular with Republicans, parents, and teachers? How did such a promising effort run aground?

In hindsight, four factors were responsible. Notably, none turned on technical debates over the merits and rigor of the standards. All were the product, to varying degrees, of the “we’re-in-a-hurry” hubris that has so often humbled would-be social reformers. Indeed, as one of the Core’s great champions, Thomas B. Fordham Foundation president Chester E. Finn, Jr.,prophetically wrote in early 2012, “It will, of course, be ironic as well as unfortunate if the Common Core ends up in the dustbin of history as a result of actions and comments by its supporters.”

First, an effort that began as a bipartisan, state-driven enterprise, spearheaded by the National Governors Association and Council of Chief State School Officers, started to look to skeptics like a federally-inspired, politicized project. The Department of Education’s decision to link federal funding to the Core in its Race to the Top program, its NCLB waiver effort, and its “ESEA blueprint,” and the provision of $350 million in federal funds for Core-related tests, all alienated anti-Washington conservatives who would have remained neutral if the question had merely concerned states collaborating to set standards in math and English language arts. By the time nationally influential conservative pundit George Will questioned in 2012 whether the federal government had exceeded its legal authority, the challenge for proponents was clear. Indeed, “Tea Party” conservatives came to regard the Common Core as part and parcel of Obama administration efforts to extend the federal role in domestic policy, an extension of contemporaneous fights over health care, spending, clean energy, the auto industry, housing, and financial regulation. Secretary of Education Arne Duncan demonstrated an unfortunate knack for making it appear that the Core was a pet Obama project–initially, when he excoriated South Carolina in 2012 for expressing second thoughts, but most famously when he futilely blasted the dozen states that announced their “implementation hiatus” in 2014. All of this served to make the Core a partisan question viewed with suspicion by conservatives, undermining the bipartisan support needed to sustain implementation in many “red” and “purple” states.

Second, the Common Core advocates were tripped up by their own impatience. After nearly all states adopted the Common Core in an early rush, proponents exhibited little interest in making the case for its merits, responding to critics, or explaining what was in store. Outside of the occasional op-ed, little sustained attention was devoted to explaining the changes or building broad-based support. For instance, hardly anyone other than Core enthusiasts realized that the comfortable, familiar high school math curriculum of math, algebra and geometry was to be eliminated and replaced with the antiseptically titled Integrated Math I, II, and III. When the magnitude of the shift became clear in 2014, confused parents and irate math teachers bombarded legislators and state board members with calls to delay implementation or alter course. Enthusiasts concentrated on designing instructional materials, consulting with states and districts, and training leaders and teachers, seemingly presuming that the public knew what they were up to and supported their effort. In the event, this turned out to be a fatal miscalculation. The early success of the Common Core was remarkable, but proponents failed to recognize that this quick success meant few voters or legislators really understood what was involved or that real success would depend crucially on the breadth and depth of support.

Third, Core advocates never did a good job of explaining how their efforts intersected with other reform priorities. Observers asked about whether the math assessment would strangle the abilities of charter schools or specialty district schools to use nonstandard math curricula. Core proponents never really answered such questions in public, tending instead to favor quiet, technical fixes (in this case abandoning mandatory “through-course” assessment) that didn’t address broader concerns. Skeptics wondered whether the testing “windows” needed to assess all children with the new computer-assisted tests would be so wide as to undermine the viability of sophisticated value-added evaluation systems that states were eagerly building. TheWashington Post’s Jay Mathews pointed out, in 2012, that the new assessments would “delay, if not stop altogether, the national move toward rating teachers by student score improvements” and that radical change would force systems “to wait years to work out the kinks in the tests” before they could resume those efforts. In hindsight, the backlash produced by the chaos over teacher evaluation and school accountability systems during 2014 and 2015 was predictable and preventable.

Finally, insufficient public attention to practical questions of cost, technology, and practice ultimately proved crippling. Despite frenzied efforts to support new assessments, instructional materials, and implementation during 2011-2014, interviews from that era with state legislators, district officials, educators, and parents showed remarkably little awareness of the costs and practical difficulties that lay ahead. When the 2012 technology scan showed that most districts had the requisite technology platform, few realized that the minimum specs had been dumbed-down or that this meant the new tests would sacrifice most of the hoped-for features–turning them into little more than traditional paper-and-pencil tests taken on a computer. At the same time, lousy records and a desire to avoid embarrassment meant that many districts had overstated their capacity in the tech census; they were suddenly faced with millions or even hundreds of millions in unanticipated new expenses, even as they dealt with the practical headaches of inadequate technology. And when the price tag for the full cost of new technology, training, leadership, teacher preparation, and all the rest became clear in 2014 and 2015, just as states emerging from the Great Recession were restoring cuts to state agencies and hoping to trim taxes, it was no surprise that a slew of states decided they’d keep the Core standards but also their old assessments, instructional materials, training, and teacher preparation.

The Core is still with us, of course, but it remains a shadow of what its more optimistic proponents envisioned a decade ago.

As I perused Zimmerman’s account, I could only feel for my many friends working so hard to make the Common Core a success. But then I thought, “Wait a minute. The future hasn’t happened yet. It’s like Marty McFly using his knowledge of the future to change the future. They can still alter course.” Will they? I suppose that’s up to them.

(Oh, and by the way, my favorite paper from the 2022 AERA conference? “When All Your Hurtful Yesterdays Become All My Gendered Tomorrows”: Transgressive Ontologies Disrupting the Heteronormative Praxis Posed by a Post-Foulcauldian, Neo-Ravitchian Autoethnography of the Lived Lives of Three Indigenous Culture-walkers in a Neo-liberal Dystopia.)

-Frederick Hess

This post originally appeared on Rick Hess Straight Up.

In this video, members of the VIVA Project Chicago Teachers Writing Collaborative talk about the empowering experience of working together and having their voices heard by the massive Chicago Public Schools system. CPS CEO Jean-Claude Brizard says The VIVA Project, as a neutral third party, made it possible for him to hear from teachers, the real experts on how to use time in school to better serve students.

The VIVA (Voices, Ideas, Vision, Action) Project brought Chicago teachers into the education policy discussion by providing not only a platform to share their ideas, but also guidance to build on each other’s ideas and create a report of recommendations that was ultimately shared with the Chicago Public Schools CEO Jean-Claude Brizard.

You can find out more about The Viva Project here.

He has more to say about a larger strategy for boosting teacher quality in “An Effective Teacher in Every Classroom,” which appeared in the Summer 2010 issue of Ed Next.

He also authored “Valuing Teachers: How Much is a Good Teacher Worth?” which appeared in the Summer 2011 issue of Ed Next.

KIPP Empower Academy is a Los Angeles-based elementary school that opened in 2010. It currently serves kindergarteners and 1st graders, and it plans to grow by one grade each year up to 4th grade. A blended-learning school, students rotate between individualized online-learning, and small-group stations within each classroom. In the school’s first year, its now 1st-grade students experienced some notable results. As reported on its website, “Though many students at KIPP Empower Academy entered kindergarten without basic letter and number recognition skills, by the end of the year, 98 percent were reading and performing math at or above the national average.” Not only that, but many students were also reading at a “2.5” grade level and performing math almost at the 3rd-grade level. And reported teacher satisfaction at the school was sky high.

Carpe Diem is a blended school based in Yuma, Ariz., which will be expanding beyond the state into Indiana in the next school year. The school, which serves grades 6 through 12, uses an individual-rotation model. In 35-minute increments students rotate from online learning for concept introduction and instruction to face-to-face for reinforcement and application. In 2010, Carpe Diem ranked first in its county in student performance in math and reading and ranked among the top 10 percent of Arizona charter schools.

The Los Altos School District began using the Khan Academy last year in a handful of 5th-grade and two 7th-grade classrooms to blend its math learning. This year the district has incorporated Khan Academy into its math curriculum for all 5th- through 8th-grade students—about 1,000 in all. With Khan Academy, teachers are able to individualize learning for each child based on real-time data. The blended-learning environment in Los Altos schools allows for seamless targeted intervention and flexible groupings, as well as real collaboration among students—all of which allows them to exercise their own student voice and choice.

Quakertown Community School District (QCS) is a traditional school district in Pennsylvania that has embraced the power of online learning to create a “self-blend” learning environment for students. All students in grades 6 through 12 have the option to take one or more online courses, and district teachers teach all the courses with the exception of those, like Mandarin, where there is no certified teacher available within the district. Two district teachers are responsible for only online courses, and roughly 75 percent of all QCS teachers are responsible for at least one online course. Courses are asynchronous; students can work on their assignments at any time during the day. Many students take advantage of this option in order to work around vocational programs, work schedules, and extracurricular interests. Some take these classes at home, and others work on them during free periods during the school day. There are designated areas in the high schools and middle schools, called cyber lounges, where students can work comfortably in a cafe setting between their face-to-face classes. The online courses allow students to move at their own pace and complete courses based on competency rather than being tethered to the traditional semester timeline.

Most powerfully, students in the district have produced a number of videos that speak to the power of the district’s approach, from the advantages of online learning from students’ point of view to the perspective of a face-to-face and online teacher, as well as a video that summarizes the district’s positive and improving student outcomes.

*  *  *

For more video viewing of blended-learning schools, I also recommend checking out the Alliance College-Ready Public Schools BLAST school, which is turning heads in Los Angeles.

Critics of voucher programs often argue that private schools do exclude most disabled students, and the matter occasionally has been the subject of litigation. Yet accurate information on students with disabilities served by private schools is notable for its absence.

The main reason for the lack of accurate information is that private schools do not operate under the provisions of the federal law that furnishes aid to the states for students identified as needing special education. Public schools expend considerable resources identifying children eligible for special services, both because they are under an obligation to provide those services and because they receive additional funds from federal and state governments if a child is identified as having a disability that affects their learning. Those obligations, rights, and funding support do not apply if parents choose to place their children in private schools with the help of a voucher. By and large, private schools have not developed the capacity to identify children with disabilities, and many of them are reluctant to do so, as they believe it leads to stigmatization of the children.

In other words, a child who may be classified as in need of special education in a public school may not be classified as such if his or her family chooses a private school, using a voucher to defray the cost. As a result, any official statistics on the prevalence of students with disabilities in public and private schools can be highly misleading.

We have not been able to surmount all of the obstacles to identifying the percentage of students in private schools who would have been identified as in need of special education in public schools, but we believe we have fairly accurate information on this question for the country’s largest and longest-running school-voucher program. The Milwaukee Parental Choice Program (MPCP), first established in 1990 and steadily expanded to include more private schools and more students in subsequent years, now serves more than 23,000 students who attend 107 different private schools. The annual voucher a school receives for each MPCP student is approximately $6,000. MPCP thus provides an excellent context for detecting the admission policies of private schools when a modest-value voucher program for low-income students is operating at scale.

In 2006, the State of Wisconsin authorized our research team to conduct a five-year evaluation of MPCP. Through the course of that study, we collected a wealth of data about the students in the voucher program and in the Milwaukee Public Schools (MPS) that permit us to estimate what proportion of the voucher student population would qualify for special education if the students were enrolled in public schools instead.

Drawing on different sources of data and various analytic methods, we estimate that anywhere between 7.5 and 14.6 percent of voucher students have disabilities that would land reported by the Wisconsin State Department of Public Instruction (DPI), a figure that gave rise to a lawsuit alleging discrimination by the MPCP program.

Following is a discussion of the procedures we followed to obtain our estimates and an explanation for the disparity between our estimates and the ones DPI has provided.

Structure of Special Education

As mentioned previously, receiving a special education designation brings with it certain legal rights for services or accommodations in the public educational sphere, as provided by the federal law known as the Individuals with Disabilities Education Act (IDEA). Once so designated, public school students are entitled to receive a free and appropriate public education (FAPE), to include special education services in the least restrictive environment possible and according to an individualized education program (IEP). A student’s IEP is drawn up by a committee that includes the student’s parents or guardians, local public-school officials, and relevant medical or psychological diagnosticians and care providers. The resulting special services and accommodations are funded through a combination of federal, state, and local monies based on formulas established in law. In Wisconsin, the federal government pays about 11 percent of the extra cost of educating each special-education student, with the state paying 26 percent and the local public-school district covering the remaining 63 percent.

The legal and funding structure surrounding students with disabilities in the private sector differs greatly from the situation in the public sector. Unless a public school district itself places a special education student in a private school, the IEP and additional funding associated with a student with a disability in the public sector does not transfer with the student if the child enrolls in a private school. The point is made in an August 2011 DPI memo on the subject:

Students with disabilities attending voucher schools as part of the MPCP are considered parentally placed private school students and as such, DPI treats them in the same fashion as students attending private non-voucher schools. Under [state law] parentally placed private school students are…not entitled to a Free and Appropriate Public Education.

If a parent enrolls a student with special needs in a private school, that student must surrender her legal rights to special educational services. Private schools are not required by federal law to enroll students with special needs, and they are not entitled to any additional resources from the state if they do so. Private schools can either accommodate the student themselves, using whatever resources they have, or negotiate with public school officials regarding the provision of special services to the student by the public school system with additional public funds (a process called “equitable services”).

Maintaining a count of those thought to be in need of special services also varies by sector. In the public sector, careful record keeping is stressed because disability status has major implications for the kinds of instructional and other services students will receive. In the private sector, special education tends to be handled much less formally, inasmuch as schools are ordinarily not required to follow formal procedures in diagnosing or serving students with special educational needs.

Given the contrasts between how special education is governed and managed in the public and private education sectors, we hypothesize the following:

1. The same student will have a higher likelihood of being identified as in need of special education if in a public school than if in a private school.

2. Given the funding available for extra services for disabled children attending public schools, a higher proportion of students with disabilities than those without disabilities will choose to remain in the public sector rather than use a voucher.

3. Any data that rely on official reports of disability will under-count the percentage of students in private schools who would have been identified as in need of special education had they attended public schools.

To test these hypotheses, we used two alternative methods to estimate the actual percentage of students in private schools who would have been identified as in need of special education in public school had they selected that sector.

Method I: Same Student, Different Sector

The better of our two methods relies on information from those students who attended schools in both the public and the private sectors during the course of our study. During the five years of our evaluation, 20.1 percent or 1,475 of the 7,338 students in our MPCP and MPS study panels switched from one school sector to the other, in some cases multiple times.

We received enrollment files from MPS each year that included information on the special education status of each MPS student. We also collected enrollment lists from every private school in MPCP and asked school officials to indicate if students had disabilities that qualified them for special education. For students who switched school sectors during the study period, we can determine whether those who were identified as needing special education in the public sector were similarly identified when they attended private schools, and vice versa. In other words, we can use each student in our study as his or her own control group to learn whether disability designations vary by sector.

Our analysis indicates that Milwaukee students who switched between the public and private school sectors were much more likely to be identified as in need of special education when they were in the public sector. On average, controlling for factors such as year and student grade, those who attended schools in both sectors were classified as in need of special education at the rate of 9.1 percent when attending private schools but at a rate of 14.6 percent when attending Milwaukee’s public schools. If we assume that a student’s need for special education did not change at the time the student switched sectors, this suggests that 5.5 percent of students attending private schools were not identified as in need of special education but would have been had they been attending public school. In other words, the identification rate in the public schools appears to be 60 percent higher (the 5.5 percent increment divided by 9.1 percent) than in the private schools. The identification rate was higher when students were in MPS both because many students who switched from MPCP to MPS received special education designations in MPS and because many students with special education designations in MPS shed them when they enrolled in MPCP schools.

The 14.6 percent MPCP disability rate is based only on students who switched sectors (35 percent of MPCP students). Those students appear to have higher rates of disability than those who did not switch. Based on principal surveys, for the 65 percent of MPCP students who did not switch, the disability rate was 3.75 percent. To get an overall rate for MPCP students, we compute a weighted average for the two groups of 7.5 percent. We suspect that this rate is conservative, since several voucher school principals told us they resist labeling students in such a way. Combining this conservative estimate with the estimate from our analysis of only students who switched sectors yields a range of 7.5 to 14.6 percent, which we think captures the likely student disability rate in MPCP.

Click to enlarge

Method II: Parental Estimates of Disability Rates

Our second estimate of the student disability rate in MPCP comes from interviews with parents. In 2007 we interviewed a random sample of parents of MPCP students in grades 3–8, all the parents of MPCP 9th graders, and a sample of parents of MPS students who were matched to the sample of MPCP students based on their grade in school, neighborhood of residence, ethnicity, test-score performance, and other characteristics. We expanded this sample with additional parents of 3rd-grade students similarly chosen in 2007 and 2008. Altogether, we interviewed a majority of the parents of 3,669 students in MPCP and 3,669 students in MPS.

The survey included the following questions:

• Does [child’s name] have any physical disabilities?

• Does [child’s name] have any learning disabilities?

If a parent answered yes to the learning disabilities question, we further asked,

• How well do the facilities at [child’s name] school attend to his/her particular needs?

According to parental responses to the first two of these questions, 2.5 percent of students in MPCP have a physical disability and 9.8 percent have a learning disability (see Figure 1). The corresponding rates reported by parents of MPS students were 4.1 percent and 18.5 percent for physical and learning disabilities, respectively. Combining the categories and eliminating overlapping cases, it is estimated that the disability rate in the MPCP sector is 11.4 percent, as compared to 20.4 percent for the MPS sector.

There is every reason to believe that these parental responses are consistent and fairly accurate indicators of what the parents are told by school officials and what they themselves know about their children. The official MPS rate for this time period is between 18 and 19 percent, just slightly less than the 20.4 percent reported by our MPS parents. The 11.4 percent disability rate for MPCP students based on our survey is midway between the 7.5 percent rate for all students in MPCP based on school staff designations and the 14.6 percent rate based on observing some of the students in both school sectors.

It is interesting that within a scaled-up, long-standing voucher program, parental satisfaction with services for students with disabilities achieves a balance across sectors. Similar levels of satisfaction with special education services are reported, regardless of whether the student was in MPCP or MPS (see Figure 2). Presumably, the choice of sectors and schools allowed parents to obtain an educational setting they view as appropriate for their child.

Discussion

Our estimates of the prevalence of MPCP students who have a disability range from 7.5 to 14.6 percent. The 14.6 percent estimate is based on the identification by public schools of the need for special services for those students who attended school in both sectors, while parental reports peg the rate at 11 percent, and the combination of MPCP and MPS school personnel suggest it is 7.5 percent.

All of these estimates are higher than the one provided, on March 29, 2011, by DPI, which said that “the private schools [participating in MPCP] reported about 1.6 percent of choice students have a disability.” That statement provoked a lawsuit by disability rights groups against DPI, which administers MPCP, based on the charge that the program discriminates in admissions against students with disabilities.

The estimate provided by DPI was based on the percentage of MPCP students who were given test accommodations on the 2010 state accountability exams. Only a fraction of students with disabilities receive accommodations on exams, and accommodations are only permitted if an IEP committee of school personnel requests them. Since few students with disabilities in private schools have IEP committees, the student-testing accommodation rate for MPCP may bear little relationship to the actual student-disability rate in the program. In fact, using administrative data we collected from the MPCP schools, we were able to determine that only one-quarter of the MPCP students judged by their school to have a disability were actually given any accommodation for last year’s test.

Using multiple measures of student disability, each of which is more valid and reliable than testing accommodation statistics, the estimates we produced indicate a 7.5 to 14.6 percent participation rate for students with disabilities in the voucher schools in comparison to the 17 to 19 percent participation rate reported for students with disabilities by the public schools. The difference could be due to discrimination against disabled students, as has been alleged, but the evidence is not sufficient to draw any such conclusions. Where disabilities are severe, private schools may not have the necessary facilities, and even in less severe instances, parents may prefer the legal entitlements and the greater range of funded services in the public sector.

What we do know, with considerable certainty, is that while the percentage of students in the voucher schools with disabilities is substantially lower than the disability rate in the public schools, it is at least four times higher than public officials have claimed. These statistical findings reinforce our views that the sectors cannot be easily compared to one another on this particular metric, because they operate under different legal obligations, financial incentives, and cultural norms. Special education is special in very different ways in public schools and in voucher programs.

Patrick J. Wolf is professor of education reform at the University of Arkansas. John F. Witte is professor of political science and public affairs at the University of Wisconsin-Madison. David J. Fleming is assistant professor of political science at Furman University.

More than 40 states have now signed onto the Common Core standards in English language arts and math, which have been both celebrated as a tremendous advance and criticized as misguided and for bearing the heavy thumbprint of the federal government. Assessing the merits of the Common Core math standards are Ze’ev Wurman and W. Stephen Wilson. Wurman, who was a U.S. Department of Education official under George W. Bush, is coauthor with Sandra Stotsky of “Common Core’s Standards Still Don’t Make the Grade” (Pioneer Institute, 2010). Wilson is a professor of mathematics at Johns Hopkins University, served on the National Governors Association-Council of Chief State School Officers “feedback group” for the Common Core standards, and was mathematics author of Stars by which to Navigate? Scanning National and International Education Standards in 2009: An Interim Report on Common Core, NAEP, TIMSS, and PISA.

Education Next: Are the Common Core math standards “fewer, higher, and clearer” than most state standards today? Can you provide some specific examples where you think the Common Core marks a step forward or backward?

Ze’ev Wurman: Common Core standards may in fact be clearer and more demanding than many, though not all, of the state standards they replaced. The Fordham Institute reviewed them last year and found them so. While I have no reason to doubt the technical quality of that review, there is good cause to note what it does not say.

Ze’ev Wurman

It does not say that Common Core standards are fewer. Indeed, if one compares them to the better state mathematics standards like those of Minnesota or California, they are more numerous. Minnesota’s standards fill 42 pages and California’s 59 pages, while the Common Core takes 73 pages even without the advanced statistics or calculus sections that are included in California’s standards. Counting the standards rather than pages, in grades 1 to 4 California has, on average, a few more standards than Common Core, but in grades 5‒8 the Common Core standards are more numerous than California’s.

Fordham’s review does not unequivocally say the standards are higher, either. They may be higher than some state standards but they are certainly lower than the best of them. For example, the 2008 report of the National Mathematics Advisory Panel, Foundations for Success, called for fluency in addition and subtraction of whole numbers by the end of grade 3, and fluency in multiplication and division by the end of grade 5. This is also what California calls for, along with high achievers like Singapore and Korea. (Japan and Hong Kong finish with multiplication and division of whole numbers even earlier, by grade 4.) Yet the Common Core defers fluency in division to grade 6. Fractions are touted as the Common Core’s greatest strength, yet the Common Core pushes teaching division of fractions to grade 6 without ever expecting students to master working with a mix of fractions and decimals. Students in Singapore, Japan, Korea, and Hong Kong achieve fluency in fractions and decimals in grade 5.

Nor are the Common Core standards necessarily clearer. They may be clearer than many state mathematics standards, but they still tend to be wordy and hard to read. Table 1 compares a few grade 4 California standards with their Common Core counterparts.

Andrew Porter, dean of the University of Pennsylvania’s Graduate School of Education, recently evaluated the Common Core standards with his colleagues, and their conclusion was stark:

Those who hope that the Common Core standards represent greater focus for U.S. education will be disappointed by our answers. Only one of our criteria for measuring focus found that the Common Core standards are more focused than current state standards…Some state standards are much more focused and some much less focused than is the Common Core, and this is true for both subjects.

We also used international benchmarking to judge the quality of the Common Core standards, and the results are surprising both for mathematics and for [ELA].… High-performing countries’ emphasis on “perform procedures” runs counter to the widespread call in the United States for a greater emphasis on higher-order cognitive demand.

Another recent analysis, by University of Southern California professor Morgan Polikoff, found the Common Core mathematics standards similarly repetitive, and hence as unfocused across elementary grades as the state content standards they attempt to replace, with only somewhat less redundancy in the middle grades.

In summary, analyses of the Common Core standards find them to be mediocre and not obviously better than many sets of state standards.

W. Stephen Wilson

W. Stephen Wilson: It turns out that nearly everyone was in favor of Common Core standards in mathematics if, and this is a big if, they got to write them. As it turns out, no one got to write the standards. A committee wrote them. Worse, the committee was hired by the very states whose standards would be replaced, so states got first crack at suggesting “corrections” to the standards. The pressures on the writing committee must have been enormous. The only reasonable expectation was that the result would resemble some sort of middle way between the states’ various standards. What is surprising is that the standards don’t rank in terms of quality in the middle 20 percent of state standards, but, instead, fall in the top 20 percent.

There is much to criticize about them, and there are several sets of standards, including those in California, the District of Columbia, Florida, Indiana, and Washington, that are clearly better. Yet Common Core is vastly superior—not just a little bit better, but vastly superior—to the standards in more than 30 states.

Where this gap is most obvious, and most important, is in laying the foundation for college readiness in mathematics early, by grade 6 or 7. Judging by state standards, few people see a connection between elementary school mathematics and college math, let alone really understand how the foundation is built.

Arithmetic is the foundation. Arithmetic has to be a priority, and it has to be done right. A number of things can and do go wrong with state standards for arithmetic in elementary school.

With the introduction of calculators, many states have downplayed the importance of arithmetic, apparently not realizing its true educational value. Instead, they spend time on statistics and probability, both of which Common Core has tossed out of early elementary school. Another thing that states love is geometric slides, turns, and flips, sometimes presented every year in grades K‒11, perhaps under the mistaken belief that they are really doing mathematics.

Fewer than 15 states are explicit about the need for students to know the single-digit number facts (think multiplication tables) to the point of instant recall. States love to have kids figure out many ways to add, subtract, multiply, and divide, but often leave off the capstone standard of fluency with the standard algorithms (traditional step-by-step procedures for the addition, subtraction, multiplication, and division of whole numbers). For example, only seven states expect students to know explicitly the standard algorithm for whole number multiplication. Fractions are even harder to find done well. Standards for fractions are generally so vague that nearly everything is left to the reader. Often states expect students to develop their own strategies or a variety of strategies for dealing with fractions. For example, only 15 states mention common denominators. Common Core does a pretty good job with arithmetic, even a very good job with fractions.

EN: Will the Common Core put an end to what has sometimes been termed the “math wars”? In your view, do the math standards resemble those recommended by the National Council of Teachers of Mathematics (NCTM), and what do you make of that similarity (or lack thereof)?

WSW: The end of the math wars! You must be joking.

There will always be people who think that calculators work just fine and there is no need to teach much arithmetic, thus making career decisions for 4th graders that the students should make for themselves in college. Downplaying the development of pencil and paper number sense might work for future shoppers, but doesn’t work for students headed for Science, Technology, Engineering, and Mathematics (STEM) fields.

There will always be the anti-memorization crowd who think that learning the multiplication facts to the point of instant recall is bad for a student, perhaps believing that it means students can no longer understand them. Of course this permanently slows students down, plus it requires students to think about 3rd-grade mathematics when they are trying to solve a college-level problem.

There will always be the standard algorithm deniers, the first line of defense for those who are anti-standard algorithms being just deny they exist. Some seem to believe it is easier to teach “high-level critical thinking” than it is to teach the standard algorithms with understanding. The standard algorithms for adding, subtracting, multiplying, and dividing whole numbers are the only rich, powerful, beautiful theorems you can teach elementary school kids, and to deny kids these theorems is to leave kids unprepared. Avoiding hard mathematics with young students does not prepare them for hard mathematics when they are older.

There will always be people who believe that you do not understand mathematics if you cannot write a coherent essay about how you solved a problem, thus driving future STEM students away from mathematics at an early age. A fairness doctrine would require English language arts (ELA) students to write essays about the standard algorithms, thus also driving students away from ELA at an early age. The ability to communicate is NOT essential to understanding mathematics.

There will always be people who think that you must be able to solve problems in multiple ways. This is probably similar to thinking that it is important to teach creativity in mathematics in elementary school, as if such a thing were possible. Forget creativity; the truly rare student is the one who can solve straightforward problems in a straightforward way.

There will always be people who think that statistics and probability are more important than arithmetic and algebra, despite the fact that you can’t do statistics and probability without arithmetic and algebra and that you will never see a question about statistics or probability on a college placement exam, thus making statistics and probability irrelevant for college preparation.

There will always be people who think that teaching kids to “think like a mathematician,” whether they have met a mathematician or not, can be done independently of content. At present, it seems that the majority of people in power think the three pages of Mathematical Practices in Common Core, which they sometimes think is the “real” mathematics, are more important than the 75 pages of content standards, which they sometimes refer to as the “rote” mathematics. They are wrong. You learn Mathematical Practices just like the name implies; you practice mathematics with content.

There will always be people who think that teaching kids about geometric slides, flips, and turns is just as important as teaching them arithmetic. It isn’t. Ask any college math teacher.

The end of the math wars! You must be joking.

ZW: Math wars erupted as a result of the unfocused and mostly math-less 1989 NCTM standards. NCTM rewrote those terrible standards in 2000, yet much of what mathematicians found objectionable remained in place. Only in 2005, with the publication in Notices of the AMS [American Mathematical Society] of “Reaching for Common Ground in K–12 Mathematics Education,” did the two sides make a serious attempt to bridge the chasm. NCTM followed shortly with its 2006 Curriculum Focal Points, a document that finally focused on what mathematics is all about: mathematics. Since then, NCTM seems to have regressed, as evidenced by its 2009 publication Focus in High School Mathematics, a document that is full of high-minded prose yet contains little rigor or specificity.

The Common Core mathematics standards are grade-by-grade‒specific and hence are more detailed than the NCTM 2000 standards, but they do resemble them in setting their sights lower than our international competitors, by, for example, locking algebra into the high school curriculum.

And they contain inexplicable holes even when compared to the much shorter NCTM Curriculum Focal Points, the major one being the absence of fraction conversion among their multiple representations (simple, decimal, percent). Other puzzling omissions include geometry basics such as derivation of area of general triangles or the concept of pi. One can argue those can be inferred, but the same can be said regarding all those state standards we acknowledge as “bad”—that all those missing pieces “can be inferred.”

What to make of such obvious deficiencies and omissions? Unfortunately, the main authors of the Common Core mathematics standards had minimal prior experience with writing standards, and it shows. While they may have had a long and distinguished list of advisers, they did not seem to have sufficient experience to select the wheat from the chaff. How, otherwise, can one explain their selecting an experimental approach to geometry, teaching it on the basis of rigid motions, that has not been successfully tried anywhere in the world? Simple prudence and an ounce of experience would tell them either to stick to what is known to work or to recommend a trial phase before foisting it sight-unseen on a nation of 300 million.

EN: How do the Common Core math standards compare to those in use in the world’s highest-performing nations? Crucially, on what do you base that assessment?

ZW: It is not difficult to show that the Common Core standards are not on par with those of the highest-performing nations.

Here is what Professor R. James Milgram of Stanford, the only professional mathematician on the Common Core Validation Committee, wrote when he declined to sign off on the Common Core standards:

This is where the problem with these standards is most marked. While the difference between these standards and those of the top states at the end of eighth grade is perhaps somewhat more than one year, the difference is more like two years when compared to the expectations of the high achieving countries—particularly most of the nations of East Asia.

And here is what a non-American member of the Validation Committee wrote to the Council of Chief State School Officers when declining to validate the standards:

I cannot in all conscience, endorse statements 2 and 3 [(2) Appropriate in terms of their level of clarity and specificity; (3) Comparable to the expectations of other leading nations] The standards are, in my view, much more detailed, and, as Jim Milgram has pointed out, are in important respects less demanding, than the standards of the leading nations.

We also have it straight from the horse’s mouth, so to speak. Professor William McCallum, one of the three main writers of the Common Core mathematics standards, speaking at the annual conference of mathematics societies in 2010, said,

While acknowledging the concerns about front-loading demands in early grades, [McCallum] said that the overall standards would not be too high, certainly not in comparison [with] other nations, including East Asia, where math education excels.

Jonathan Goodman, a professor of mathematics at the Courant Institute at New York University, found exactly that: “The proposed Common Core standard is similar in earlier grades but has significantly lower expectations with respect to algebra and geometry than the published standards of other countries.”

It is also worth mentioning that the standards, in addition to being “[c]omparable to the expectations of other leading nations,” were also supposed to be “[r]eflective of the core knowledge and skills in ELA and mathematics that students need to be college- and career-ready.” That is, at least, what the other Common Core Validation Committee members certified when they signed off on the standards in 2010.

College readiness is defined by what colleges require as prerequisites from their incoming freshmen. The enrollment requirements of four-year state colleges overwhelmingly consist of at least three years of high school mathematics including algebra 1, algebra 2, and geometry, or beyond. Yet Common Core’s “college readiness” definition omits content typically considered part of algebra 2 (and geometry), such as complex numbers, vectors, trigonometry, polynomial identities, the Binomial Theorem, logarithms, logarithmic and exponential functions, composite and inverse functions, matrices, ellipses and hyperbolae, and a few more.

What should we make, then, of a recent study purporting to “validate” that Common Core standards indeed reflect college readiness? The study, led by David Conley, was published more than a year after Common Core standards were already certified as college-ready by…David Conley as a member of the Common Core Validation Committee. Paraphrasing Shakespeare, he doth attest too much.

In summary, the Common Core mathematics standards fail on clarity and rigor compared to better state standards and to those of high-achieving countries. They do not expect algebra to be taught in grade 8 and instead defer it to high school, reversing the most significant change in mathematics education in America in the last decade, supported by the 2008 recommendations of the National Mathematics Advisory Panel, and contrary to the practice of our international competitors. Moreover, their promise of college readiness rings hollow. Its college-readiness standards are below the admission requirement of most four-year state colleges.

WSW: When you are so far behind, comparing the United States with better-performing countries through the incredibly narrow lens of standards doesn’t make a lot of sense. I think Common Core is in the same ball park, certainly not up there with the best of countries, but Common Core isn’t up there with the best state standards either, and what does that mean? Look at California’s standards for example. They are great standards and have been unchanged for over a decade, but many in math education hate them. They think they are all about rote mathematics, but I think such people have little understanding of mathematics.

So, let’s just pretend for a moment that Common Core is just as good as the very best. Who, in education circles, will agree with that enough to put it all in practice? The standard algorithm deniers will teach multiple ways to multiply numbers and mention the standard algorithm one day in passing. Korea will say “no calculators” in K–12, a little extreme perhaps, but some in the U.S. will say “appropriate tools” means calculators in 4th grade. We, in this country, are still not on the same page about what content is most important, even if everyone says they’ll take Common Core. Without a unified, concerted effort to teach real mathematics, there isn’t much chance of catching up.

In other countries, if you say “learn to multiply whole numbers,” no one questions how this should be done; students should learn and understand the standard algorithm. In the U.S., even if you say “learn to multiply whole numbers with the standard algorithm,” some people will declare wiggle room and try to avoid the standard algorithm.

There is one big hope for our international competitiveness. Other countries see that their best STEM students come to the U.S. for graduate school—more than half of our STEM graduate students are foreign—and to start high-tech companies. Instead of thinking that this is possible because of their strong K–12 mathematics education, they erroneously conclude that they should adopt our version of K–12 mathematics education. We just might catch up with these countries without any effort on our part.

EN: What, then, are your main areas of disagreement?

WSW: Ze’ev refers to Andrew Porter’s work to support his argument that Common Core lacks focus. In the corrected version of Porter’s paper, he says that 39.55 percent of grades 3‒6 coarse-grained topics for the states are on Number Sense and Operations, but Common Core gets 55.47 percent. To me, that says that Common Core focuses on arithmetic in grades where arithmetic should be the focus, and that the states did not focus on arithmetic.

My only serious disagreement with Ze’ev is his summary that “analyses of Common Core standards find them to be mediocre and not obviously better than many sets of state standards.” If Common Core is mediocre, then mediocre is being set at a high standard. There are many states that set a very different, and much lower, standard for mediocre.

ZW: Steve sees the benefit of having Common Core standards that are better than those of “more than 30 states,” while I see the disadvantage of confining the whole nation to mediocre standards that are worse than those of highly rated states and high-achieving countries.

Taking this a step further, I believe the Common Core marks the cessation of educational standards improvement in the United States. No state has any reason left to aspire for first-rate standards, as all states will be judged by the same mediocre national benchmark enforced by the federal government. Moreover, there are organizations that have reasons to work for lower and less-demanding standards, specifically teachers unions and professional teacher organizations. While they may not admit it, they have a vested interest in lowering the accountability bar for their members. With Common Core, they have a single target to aim for, rather than 50 distributed ones. So give it some time and, as sunset follows sunrise, we will see even those mediocre standards being made less demanding. This will be done in the name of “critical thinking” and “21st-century” skills, and in faraway Washington D.C., well beyond the reach of parents and most states and employers.

Getting Smart: How Digital Learning Is Changing the World

As reviewed by Mark Bauerlein

“The revolution is on,” Tom Vander Ark declares in this review of digital learning circa 2011. With long experience in education, including time as a superintendent in Washington State, officer with the Gates Foundation, and CEO of Open Education Solutions, not to mention an endorsement from the former governor of West Virginia, Bob Wise, Vander Ark outlines the current moment as a welcome and overdue threshold in primary and secondary education. On one page alone he repeats, “The learning revolution underway is the shift from print to digital…,” “The revolution will yield powerful learning platforms…,” “The revolution will yield a new generation of schools…,” and “The learning revolution is underway but progress will be lumpy….”

As the subtitle indicates, we stand at a critical moment, and there is good reason for optimism, given the ways in which digital technology can customize learning and dismantle the old calendars and spaces of schooling. Extraordinary innovations have arrived—online curricula, learning games, customized play-lists—and they are ready for implementation across the land if only educators and public officials break with standard procedure and embrace them. It’s time to “get smart,” and hence this 10-chapter exhortation on the efficacious future. Every few pages Vander Ark adds a bold prediction sidebar: “In five years…Information from keystroke data will unlock the new field of motivation research…,” “In five years…Most learning platforms will feature a smart recommendation engine, similar to iTunes Genius…,” and “In five years…Science will confirm the obvious about how most boys learn and active learning models will be developed in response using expeditions, playlists, and projects.”

He accumulates rousing examples of individuals and institutions in breakthrough practice:

• students tapping into iTunes U, compiling e-portfolios, and editing web sites

• peer-to-peer learning sites and learning games such as Mangahigh

• online organizations such as K12 and School of One that replace wasteful “seat-in-class” time with customized learning time

• social networking that “will augment and then replace the classroom as the dominant organizing unit”

Experts, too, assert the radical advances of digital tools, such as Tom Chatfield, author of Fun Inc.: Why Gaming Will Dominate the Twenty-First Century, who says of games, “I’m in awe at their power to motivate, to compel us, to transfix us, like really nothing else we’ve ever invented has quite done before.” Vander Ark encapsulates the advent in a simple formula: “It changes everything when anyone can learn anything almost anywhere.”

As the effusions pile up, however, one wonders about how much the enthusiasm obscures some circumstances that complicate Vander Ark’s bold and sanguine vision. After all, broad, well-funded digital initiatives such as Maine’s statewide laptop program for middle schoolers have been around for a decade, and yet their academic impact has proven disappointing again and again. And Vander Ark affirms that social media “can help build a common culture and help make sense of a confusing world—and increasingly so for school communities,” but all he says about the dark side of social networking among teens—including excess peer pressure and gossip, sexting and bullying, cheating—is, “Some of their reasons for connecting will not be as noble as we’d like, so we’ll need to stay on top of this.”

These conditions don’t change the overall potential of digital learning, but more acknowledgment of them sustains a more sober, less partisan advocacy. Without it, Vander Ark slips too often into dramatic predictions and platitudes. He announces, “If we can help enough people get smart, I believe we can confront the challenge of climate change, public health, peace, and security,” as if smart people never pollute the earth or start wars. After glimpses of three bright kids learning online in creative ways, Vander Ark writes, “These portraits represent how millions of students could be learning with tools that are currently available to schools,” as if the cases of three prove millions more. And the trick of motivating kids, he says, has been found: “Any thirteen-year-old could tell you the answer. It’s game designers”—a too pat and blunt answer.

All this hype and prophecy is unnecessary. The digital future is here, and its main educational advantage, the individualization of learning, is recognized by everyone. At this point, the pressing questions are practical: how much it costs, how to overcome bureaucracy, for example. Vander Ark does include an appendix of concrete advice, such as urging state leaders to allow students to personalize their learning and base matriculation on demonstrated competency, not on seat time, but these are precisely the points to expound in the main text, not stick in an appendix. We don’t need any more puffy announcements of youth liberation, such as “This greater self-awareness and freedom brings with it new responsibilities and opportunities for students to better advocate for themselves.” And overdone assertions, such as “games have the motivational power to help us change the world,” don’t mean anything to public officials. What we need is sound evidence, presented without hyperbole, of scalable and cost-effective digital programs that yield higher reading, writing, and math achievement.

Mark Bauerlein is professor of English at Emory University.

Click to enlarge

Not all school systems are moving in that direction. In Cambridge, Massachusetts, a district with surprisingly low student performance given the substantial per-pupil resources at its command, the school committee has decided to try to boost student achievement by abandoning its K–8 model in favor of having separate middle schools that serve grades 6 through 8 (though, in an unusual twist, each of the latter will be housed in the same facility as an elementary school).

In short, policymakers nationwide continue to wrestle with a basic question: At what grade level should students move to a new school? In the most common grade configuration in American school districts, public school students make two school transitions, entering a middle school in grade 6 or 7 and a high school in grade 9. This pattern reflects the influence of enrollment pressures and pedagogical theories that, over the past half century, all but eliminated the K–8 school from the American education landscape. A small fraction of students do attend public schools encompassing grades K–8, 6–12, or even K–12, however. We exploit this variation by comparing the achievement trajectories of Florida students entering a middle school or a high school to those of their peers who do not make those transitions.

Our study extends research conducted in New York City (see “Stuck in the Middle,” research, Fall 2010), in which Jonah Rockoff and Benjamin Lockwood found that entering a middle school causes a sharp drop in student achievement relative to the performance of those remaining in K–8 schools. It is hard to know whether one can generalize from results from the nation’s largest city (and school district), however, especially when it employs a complex procedure for assigning students to middle schools. Also, the New York City study was unable to follow students after 8th grade, making it impossible to know whether the negative impacts that were observed were temporary or extended into high school. This is a critical question inasmuch as a key rationale for middle school is its potential for easing the transition to high school. What is lost at the first transition may be more than gained at the second, which is presumably less abrupt for the middle-school child than for the one entering high school directly from an elementary-school environment.

To explore these issues, we use statewide data covering all students in Florida public schools who were in grades 3 to 10 between 2000 and 2009. Although a large majority of Florida students enter a middle school in grade 6, some do so in grade 7. Still others attend K–8 schools and avoid the middle-school transition altogether. To determine whether entering a middle school in grade 6 or grade 7 has any effect on achievement, we examine whether students experience a drop in test scores relative to students in K–8 schools that coincides with their transition to the new school. In the same way, we compare the learning trajectories of students entering high school in grade 9 to those of students who attend K–12, 6–12, or 7–12 schools in order to determine whether high-school transitions affect achievement.

Our results cast serious doubt on the wisdom of the middle-school experiment that has become such a prominent feature of American education. We find that moving to a middle school causes a substantial drop in student test scores (relative to that of students who remain in K–8 schools) the first year in which the transition takes place, not just in New York City but also in the big cities, suburbs, and small-town and rural areas of Florida. Further, we find that the relative achievement of middle-school students continues to decline in the subsequent years they spend in such schools. Nor do we find any sign that the middle-school students catch up with those who remained in the K–8 environment once all of them have entered high school. On the contrary, students entering a middle school in grade 6 are more likely not to be enrolled in any Florida public school as 10th graders (despite having been enrolled in grade 9), a strong indication that they have dropped out of school by that time.

We also find that the transition to high school causes a small drop in student achievement for all students who make this transition (as distinct from those in schools with 6–12 grade configurations). However, this drop holds far less policy significance both because of its size and because the decline does not appear to persist beyond grade 9.

The achievement drops we observe as students move to both middle and high schools suggest that moving from one school to another (or simply being in the youngest grade in a school) adversely affects student performance. The size and persistence of the effect of entering a middle school, however, suggests that such transitions are particularly damaging for adolescent students or that middle schools provide lower-quality education than K–8 schools provide for students at the same point in their education.

Data and Method

We draw the data for our analysis from the Florida Department of Education’s PK-20 Education Data Warehouse. The data contain state math and reading test scores for all Florida students attending public schools in grades 3 to 10 from the 2000–01 through 2008–09 school years. They also include information on the school each student attends and its location as well as student characteristics such as ethnicity, gender, special education classification, and eligibility for a free or reduced-price lunch.

We use different samples of students for different parts of our analysis. First, to estimate the effect of entering a middle school in grade 6 or 7, we examine only students enrolled in grade 3 between 2001 and 2004 who completed the state test in both math and reading in each of the subsequent five years. Second, to investigate whether the effects of middle-school entry persist through grades 9 and 10, we examine only students enrolled in grade 3 in 2001 or 2002 who were tested in both subjects each of the following seven years. Finally, to estimate the effect of entering high school in grade 9, we examine students enrolled in grade 6 between 2001 and 2005 who were tested in both math and reading in the following four years.

Our strategy for identifying the effects of alternative grade configurations on student achievement parallels and extends that of Rockoff and Lockwood’s study of New York City middle schools mentioned above. Specifically, we examine changes in individual students’ achievement over time, focusing on differences in the timing of students’ entry into middle school that result from the grade configuration of the school the student attended in 3rd grade. For example, we are interested in whether students who attended a K–6 school in 3rd grade experience a drop in their achievement in 7th grade relative to students who attended a K–8 school in 3rd grade and thus did not switch schools between grades 6 and 7.

The key assumption of our methodology is that there are no unobserved differences between students who in 3rd grade attended schools that had these different grade configurations that affect achievement precisely in the year when students enter middle school. In other words, we are assuming that the negative effect of a transition is not anticipated by parents and reflected in the choice of a school with a particular grade configuration in grade 3. We conduct an analogous analysis of high-school entry, taking advantage of the different grade configurations of the schools students attended in 6th grade.

Because we compare the achievement of individual students to themselves over time, our analysis takes into account all student characteristics (both observed and unobserved) that do not change over time. In addition, we also control for whether the individual student had been retained in a grade, whether the student had ever been retained, and whether the student attends a charter school (which in Florida are more likely than traditional public schools to have K–8 configurations).

The Middle-School Cliff

We find that students who will enter a middle school in 6th or 7th grade have positive achievement trajectories in math and reading from 3rd grade to 5th, relative to their counterparts who will never enter a middle school because they attend a school that continues through 8th grade. Achievement in both subjects falls dramatically in 6th grade for students who enter middle school in that grade. Students who will enter middle school in grade 7 continue to improve relative to their K–8 peers through grade 6, but experience a sharp drop in achievement upon entering middle school in grade 7.

Specifically, we find math achievement falls by 0.12 standard deviations and reading achievement falls by 0.09 standard deviations for transitions at grade 6 (see Figure 1). Students who make the transition at grade 7 experience even larger drops in their achievement of 0.22 and 0.15 standard deviations in math and reading, respectively. National data indicate that student achievement increases by roughly 0.30 standard deviations in math and 0.25 standard deviations in reading each year for typical 6th- and 7th-grade students. The drops in achievement we observe for students entering middle schools therefore amount to between 3.5 and 7 months of expected learning over the course of a 10-month school year.

Just as troubling is the fact that these students’ relative performance in both subjects continues to decline in subsequent middle-school grades. After three years in a middle school, students who entered in 6th grade score 0.23 standard deviations in math and 0.14 standard deviations in reading worse than we would have expected had they attended a K–8 school. After two years in a middle school, students who entered in 7th grade underperform by 0.31 standard deviations in math and 0.15 standard deviations in reading.

We also find little evidence that students who attend middle school make larger achievement gains than their peers in grades 9 and 10, by which time most Florida students have entered high school. In addressing this issue we must limit our attention to the two cohorts of students entering 3rd grade prior to 2001 or 2002, whose progress we are able to follow through the 10th grade. Although the math achievement of students who entered middle school in 7th grade improves by 0.05 standard deviations in 9th grade relative to students who attended K–8 schools, the same pattern is not evident in reading or in either subject for the much larger group of students who entered middle school in 6th grade (see Figure 2). In other words, we can safely reject the hypothesis that students who attend middle schools benefit at the transition to high school from their previous experience with school transition or from the specific educational programs available in middle schools.

Investigating the transition to high school, we find that students moving to a new high school between grades 8 and 9 suffer a small drop in achievement of 0.03 standard deviations in math and 0.04 standard deviations in reading (relative to those in grade 6–12 schools or schools with another configuration that requires no transition at this point). However, their relative achievement trajectories become positive again after this drop at the transition point.

We supplement our analysis on math and reading achievement with similar analyses of the effects of entering a middle school on the probability of students’ not being enrolled in a Florida public school in 10th grade (a proxy for dropping out of high school by this time) and on being retained in 9th grade (often a strong predictor that a student will leave school prior to graduation). Our results suggest that entering a middle school in 6th grade increases the probability of early dropout by 1.4 percentage points (or 18 percent). Although entering a middle school in 7th grade does not appear to increase early dropout, it increases the probability that a student will be retained in 9th grade by 1 percentage point. Both results provide additional cause for concern with the middle-school model.

Is it possible that our results reflect differences across school districts that employ alternative grade configurations? We explore this question by conducting our test-score analysis separately for schools in Miami-Dade County. With more than 345,000 students, Miami-Dade is the largest district in Florida and offers a wide range of grade configurations for students up through grade 8. We find that the negative effects of entering a middle school for grade 6 or grade 7 are, if anything, even more pronounced in Miami-Dade County than they are statewide.

Not Just an Urban Problem

This result for Miami-Dade County raises the possibility that the negative effects of middle-school entry are only notable in urban settings. We address this issue by looking separately at the effects of entering a middle or high school across communities of varying sizes. Using Census Bureau classifications, we group students into three categories according to the location of the school they attended in 3rd grade: 1) a large or midsize city, 2) suburbia (specifically, the urban fringe of a large or midsize city), and 3) towns and rural areas. The results suggest that the negative effects of entering a middle school are most pronounced in cities, but they remain sizable even in rural areas, confirming that the negative effects of configurations that separate the middle-school grades are by no means limited to urban school districts.

We also examine whether the middle-school effect varies across subgroups of students defined in terms of prior test performance, ethnicity, and gender. Students whose 3rd-grade scores were below the statewide median saw substantially larger declines in math scores at both the middle- and high-school transition points than higher-achieving students. These patterns are consistent with the theory that lower-achieving students have access to fewer educational resources outside of school and may therefore be at higher risk of being adversely affected by school transitions. We find no clear indication that the negative effect differs in size for higher- and lower-achieving students in reading, however.

Results for students of different ethnicities follow a similar pattern. Grade configuration has a larger effect on the math scores of traditionally disadvantaged subgroups than on other students. Black students in particular demonstrate large relative gains in math achievement prior to entering a middle school but then suffer larger drops both at and following the transition. Again, however, we find only small and statistically insignificant differences between the effects estimated for students of different ethnicities in reading. We find no differences in the effects for girls and boys.

Potential Explanations

Our results confirm that transitions into both middle schools and high schools cause drops in student achievement but that these effects are far larger for students entering middle schools. One possible interpretation of this pattern is that school transitions are more disruptive for younger students, perhaps because they are more susceptible to the negative influence of older students. Yet our estimates suggest that the effect of middle-school entry on student achievement is larger for students entering in grade 7 than for students entering in grade 6. Moreover, the fact that relative achievement continues to decline after students’ initial entry into middle schools suggests that average educational quality in Florida is lower in stand-alone middle schools than in schools serving grades K–8.

To explore why this might be the case, we first examine several characteristics of Florida elementary, middle, and K–8 schools. The most striking difference across school types involves cohort sizes (the average number of students in each grade). Although middle schools offer far fewer grades than K–8 schools, Florida middle schools on average enroll 146 more students than their K–8 counterparts; as a result, typical grade cohorts are almost three times as large. Florida middle schools also spend 11 percent less per student and have higher student-teacher ratios than K–8 schools, suggesting a potential role for differences in available resources. In contrast, we find no evidence that differences in observed teacher characteristics could explain our findings. Average teacher experience and average teacher salaries are similar across school types, while the share of the school’s instructional staff without prior experience is modestly higher in K–8 schools.

We conduct two analyses to shed light on whether these observed differences between middle schools and K–8 schools are likely to contribute to differences in school quality. First, we rerun our test-score analysis while controlling for these differences and find a similar pattern of results. Second, we examine whether the size of the drop in relative achievement suffered by students entering middle school in grade 6 varied with the characteristics of the middle school they attended. The results of this analysis again provide little evidence that low middle-school quality stems from differences in the school characteristics we can observe.

Middle schools could also differ from K–8 schools in their educational practices in ways that lead to lower student-achievement gains. To explore this possibility, we draw on a unique survey of Florida school principals conducted in 2003–04 to document responses to the state’s high-stakes accountability system. Confidentiality requirements preclude us from linking survey responses to specific schools, but we can document any differences in the average responses offered by principals of different school types.

We find few significant differences in the educational practices of the two groups of schools in our study. In particular, we observe no differences in the length of the school day or in measures of the extent to which schools had adopted specific policies to help low-performing students, policies to improve the performance of ineffective teachers, and incentives to reward highly effective teachers. If anything, these measures suggest that middle schools are more likely to have policies aimed at improving student achievement. We also find no differences across school types when we measure the degree of teacher autonomy.

A final set of survey items asked not about specific policies or practices but about the school’s overall climate. On these items, middle-school principals expressed significantly lower levels of agreement with statements indicating that their new and veteran teachers were excellent. This suggests that teachers in these schools may be less well equipped to deal with the challenges presented by their students. More middle-school principals also agreed with the statement that parents are worried about violence in the school. Although differences on the remaining items were statistically insignificant, they consistently point in the direction of middle schools having less-favorable school climates than K–8 schools.

In short, we find little evidence that the negative effects of attending a middle school are attributable to differences in resources, cohort sizes, or educational practices. We do, however, find suggestive evidence that the overall climate for student learning is worse in middle schools than in schools that serve students from elementary school through the 8th grade. This suggests a final potential interpretation of our results that is directly related to the choice of grade configuration: students may benefit from being among the oldest students in a school setting that includes very young students, perhaps because they have greater opportunity to take on leadership roles. This interpretation could account for both the gains in relative achievement made by students in K–5 and K–6 schools prior to entering middle schools and the superior performance of K–8 students relative to their peers in middle schools. A possible, if unlikely, alternative explanation is that students entering schools with different grade configurations have different growth trajectories for reasons having nothing to do with their schooling environment.

Taken as a whole, our results suggest that school transitions lower student achievement but that attending middle schools in particular has adverse consequences for American students. Especially when considered along with those of other recent studies, our findings clearly support ongoing efforts in urban school districts to convert stand-alone elementary and middle schools into schools with K–8 configurations. They are also relevant to the expanding charter-school sector, which has the opportunity to choose grade configurations without the disruption caused by school closures. More research is needed to see whether policy or pedagogical innovations can mitigate the effects of middle school. In the meantime, policymakers should exercise caution before extending the middle-school experiment to school districts that still enjoy the K–8 configuration.

Martin R. West is assistant professor of education at the Harvard Graduate School of Education and deputy director of the Program on Education Policy and Governance (PEPG) at Harvard’s Kennedy School. Guido Schwerdt is a researcher at the Ifo Institute for Economic Research in Munich, Germany.

Any time a high school student takes a course from a district other than the one where they live, a portion of Utah’s state aid shifts from the home district to the district providing the course online.

A district with a brilliant slate of online suddenly has the chance to solve its fiscal problems the easy way.

I learned about the Utah experiment at a conference held at Stanford University’s Hoover Institution, and sponsored by Harvard’s Program on Education Policy and Governance. While the details of the Utah experiment were not discussed, the basic idea is certainly intriguing.

No longer must students in rural Utah be denied the opportunity to take physics, chemistry, computer science or an esoteric language simply because the local district cannot afford teachers for courses with small enrollments.

No longer must a student in Utah take a social studies course from a teacher the student finds boring and unhelpful.

No longer must a student who cannot attend school on a daily basis—either because he or she is sick, or pregnant, or feels bullied, or wants to train for an Olympic sport—be denied the opportunity to maintain a regular schedule that will lead to a timely graduation.

Some find the policy unfair to smaller school districts, which lack the resources to create online courses.  To keep the playing field level, they say, each district should be allowed to provide online courses only to their own students. That way state aid would continue to flow to the district bearing the expenses associated with facilities management, extracurricular activities, transportation, the school lunch program, the guidance counselors, and much more.

If only a few students take just one or two online courses, the new policy may not pose too heavy a burden, but if student demand for courses outside their own high schools escalates rapidly, the inter-district competition could prove to be seriously disruptive for some districts.

One solution would be for the state to fund online courses outside the home district at something other than the full amount—perhaps at the 50 or 60 percent level.  The remainder would go to the home district. If Utah is not doing that already, it might consider an amendment along these lines.

If small districts want to keep all of their state aid, they should be able to save on upfront costs by contracting their online courses offerings out to other providers.  Florida Virtual School is already marketing such courses nationwide, and both commercial and university providers can be expected to follow, if they are compensated for each course taken.

Of course, there could be a race to the bottom, as each district looks for the cheapest provider.  If tests are easy, some students might be tempted to take a course no matter how poorly it is constructed.

Clearly, some kind of industry or state vetting of courses is needed if online learning is not to become the latest fad to go wrong.

Exactly how Utah is solving these problems is something I plan to share with you in a future post.  For now, I simply want to herald the idea of inter-district competition in the online world.  Whatever problems it may pose for some districts, it is hard to see why district needs should be put ahead of student ones.

If digital learning is to advance beyond the pilot stage, it needs to work within the current system of public education, not against it.  Public school districts have a legitimacy unrivalled by any other institution in American education. Whether digital learning is blended into the classroom or offered online, or both, districts have to be part of the action.

The solution is to put districts into competition with one another within an overall framework that maintains course quality.  If that is done, then it will only take two or three entrepreneurial districts to convince the remainder that they need to adjust if they are to keep their students from slipping away, one by one, course by course.

I shall report later on the specifics of the Utah experiment.  For now, I simply want to herald the general concept.  Putting districts in charge of online learning, while allowing them to contract out to private providers if they wish, creates a competitive marketplace within a legitimate political framework.  If properly implemented so as to maintain course quality and integrity, it can give all students, no matter what their racial, ethnic, or religious background, no matter what their place of residence, an opportunity to take well-designed courses offered under the direction of truly high quality teachers, to be taken by students each at their own pace.

-Paul Peterson

Still, the chatter in various social media about the announcement extend two faulty themes that needlessly limit educational technology discussions.

The first misguided frame, expressed by Core Knowledge’s Robert Pondiscio in USA Today, is whether technology, in this case digital textbooks, is a “magic bullet.” Pondiscio is right: Of course it’s not and anybody who claims so is foolish. But debating this point gets us nowhere.

What’s most important to understand about the digital textbook effort is that it’s an opportunity to open up a large amount of existing public money that has been locked into use by a very small and closed set of publishers. Opening up classrooms to new technologies in no way guarantees that textbooks or digital instructional materials will be better. But, it does provide the opportunity to shift power to educators, offering the possibility for not only more customization by teachers, but also access to a greater array of better materials. And, smaller publishers, including those who offer free content, such as Core Knowledge, may finally have a chance to enter classrooms based on the strength of their content, rather than their distribution and sales teams.

The second faulty frame is the conspiratorial suspicion of nefarious intent: any technology initiative is just a cover for private profit-seeking. But let’s be serious. We wouldn’t be having this discussion around school modernization. Construction companies make a lot of money on educational projects. We understand though, that this is a reason to exercise strong oversight of public funds. It’s not a reason to oppose modernizing crumbling facilities.

In reality, opposition to digital textbooks cements corporate control of instructional materials. This is about technology-driven industry change. Again, our K-12 schools already spend billions each year on textbooks — almost all purchased from the same small set of publishers. New companies are surely aiming at these dollars, just as Google, Facebook, and Craigslist have siphoned off newspaper ad revenues. And, this industry change also opens the doors for open educational resources (OER) that can be freely shared and modified. This is the real battle, between new and old ways of doing business, open and closed, as seen in the recent debate over SOPA. If there’s a critique here, it’s that there was little sign of the OER community in either the FCC’s announcement or the “Digital Textbook Collaborative” that it convened.

Two more things you may have missed:

• TASC continued its Digital Learning Beyond School effort with a white paper and video that makes the case for using technology to help community educators and teachers engage students in learning anywhere at any time.
• My favorite article from yesterday’s coverage describes a collaboration between the University of Utah and Google that is helping kids with autism spectrum disorders to shine. (h/t @mcleod)

-Bill Tucker

It’s no surprise that, 28 years after the publication of A Nation at Risk, school-reform efforts have generated so little effect. Our schools have proven, over the past century, quite adept at resisting change.

Recent attempts to inject accountability and innovation have brought us to an important opportunity. No Child Left Behind helped add transparency, and Race to the Top (RttT) motivated states to rethink teacher evaluation, charter limits, and more. The Investing in Innovation fund (i3) has seeded some promising innovations and helped attract more private investment to public education.

But none of these initiatives hits at the reasons that education has proven itself so innovation-resistant: governance and compensation. Further, there is good reason to believe a third impediment—the absence of useful data—will persist even through the Common Core State Standards initiatives.

Finland serves as a model for many reformers. There is a single curriculum; teachers are well educated and well respected. Their system reflects Finnish ideals and builds on Finnish strengths, and their students score at the top of international tests like PISA (Program for International Student Assessment) and TIMSS (Trends in International Mathematics and Science Study).

But a top-down system will continue to be the wrong approach in this country, whether on a national or state level. It doesn’t reflect American values or culture, nor does it address the size, diversity, or income disparity of the United States. (Finland has half as many students as New York City, and only 13 percent live in poverty.) In a country of 300 million people, a top-down approach makes substantive change virtually impossible. To fix our schools, states have to stop trying to fix them; the quickest way to raise performance is command and control, but over the long run martial law does not even work well for generals.

States can create a more agile, more American, system of governance that eliminates impediments to improvement, empowers schools to innovate, and uses data to help families find the right schools for their children. The federal government should encourage them to do so.

None of the proposals below address the role of profitmaking companies in K‒12 education (though my bias might be clear, as I have run education companies for 30 years). It is important not to conflate marketplace with for-profit. It is also important to recognize that it takes time for deregulation and a newly formed marketplace to work. The breakup of AT&T and the telecommunications bill of 1986 did little to help consumers in the very short term, but they cleared the path for lower costs and technologies including the Internet and the cellphone. Occasionally efforts to create a marketplace don’t work at all, as happened with banking deregulation. As education is a public good and requires public funding, proposed structures should be measured by the incentives they will create for schools, districts, and teachers to produce great student outcomes at reasonable expense.

Empower Schools

Although our ultimate goal is a system of schooling that naturally evolves and improves, it’s important to keep in mind that the capacity for experimenting and innovating resides in individual schools, not in central offices. Under the current system of governance and funding, schools have too few resources and too little discretion for experimentation. Without the dollars to implement novel ideas and to discover what works and what doesn’t, most schools look for, at most, incremental improvement.

Right now, every state distributes state and federal funds to districts; in turn, the districts distribute funds to schools. Imagine that states instead channel funds directly to schools and require that the schools contract with a school support organization (SSO) for an array of services similar to what its district’s central office now provides (see Figure 1). There are many ways to implement such a plan, but the recent transition of New York City schools to its empowerment model might serve as a useful example, even though the city may be losing its resolve to change.

Ideally, existing school districts would be spun off as independent nonprofits and freed to compete with other districts, as well as with the new SSOs in the nonprofit and for-profit sectors, for schools and dollars. University of Washington research professor Paul Hill and others have proposed variants of this concept.

Since most schools (especially those in small and wealthy districts) would probably keep their existing districts as their service providers at first, the initial shift would be subtle. But before long the roles and behavior of schools and districts would begin to change. Freed to choose a district or other SSO based on service, cost, and philosophy, schools would demand more for less, and SSOs would step up to pull schools away from their local districts and compete by differentiating themselves from their competitors. Perhaps they would charge less for similar services; perhaps they would deconstruct the services, providing only busing, technology, or financial/purchasing support. Eventually, districts and SSOs would also vie for schools based on their track records of learning outcomes.

Under this proposal, districts would become providers of services rather than owners of geographic zones. With their schools acting as clients rather than dependents, districts would be forced to compete for them, thereby becoming more innovative and cost-effective.

Concrete results would take a while to materialize, but they would come. The current system of big-district purchasing, for example, favors large textbook publishers, which play it safe. School-level purchasing—with proper financial controls—would allow smaller, more responsive companies to compete for business.

Charter schools are the one reform initiative of the past three decades that has addressed the issue of K–12 governance and gained some traction (some 5 percent of public schools are now charters). This proposal builds on some of the lessons learned from the charter school movement and would allow effective charter networks like Green Dot, KIPP, and North Star to operate as school support organizations on a level playing field with districts, with equal funding and authority. A great deal of innovation today is coming from charter networks; this change would encourage districts to match them.

Most states would need to implement significant initiatives to prepare school principals for their new role, and to recruit new principals with the right skills; education schools and programs like New Leaders for New Schools could participate in this effort. Further, states would need to balance power between districts and schools; for example, districts should have the power to reject association with a poorly performing school. Both schools and districts should be pushed to improve themselves and their products and services.

Accountability would become simple (and imperative) under this model. The newly empowered schools should live or die by their performance; similarly, SSOs would lose their customers if they proved unable to support high achievement (which is how the stock of K12, Inc., lost 40 percent of its value following a single critical article in the New York Times). Accountability goes hand in hand with empowerment; promoting one without the other will not succeed.

Empowering schools would also mean encouraging parental choice. After the district’s monopoly is broken up, it would be critical that states create intelligent, consumer-friendly systems to support parents in choosing their children’s schools. Any number of successful models exist, all of which would provide transparency and could be used to balance families’ desire for schools within reasonable distance with their desire for the right outcome.

This is not an easy change; further, many districts are already well run and don’t need change at all. But this proposal would remake the relationships between schools, districts, and states into a far more efficient and effective model, one that would increase agility and remove regulations that limit the autonomy of school leaders. (As Arizona congressman Jeff Flake once asked, “Who out there can sing their district fight song?”)

Offer Teachers a New Deal

Once we’ve empowered schools, we’re ready to address teacher compensation. Many people believe that teachers unions are a major cause of whatever they think is wrong with our schools. It’s not that simple; plenty of research suggests that districts without unions do not perform better than those that have unions, and are only slightly less expensive.

To be sure, pensions and tenure are huge impediments to organic change. But two parties signed the contracts putting them in place: the union, whose job is to get its members more pay for less work, and the district. It was the side representing kids—the districts and state legislatures—that failed. Demonizing unions and teachers is unfair and counterproductive.

The problem isn’t the total compensation; if anything, teachers are underpaid. It’s the structure of that compensation, a series of long-term obligations that severely limit agility while creating off–balance sheet debt that would make Wall Street blush. (According to district budget figures, New York City, for example, spends as much on teachers who no longer teach as on those who still do.)

Ending tenure without ending the current pension system would create some impossible pressures; teachers nearing certain vesting thresholds, for instance, would have a target on their backs. To create an agile system, states must end both tenure and pensions. We can take a big step down this road without reneging on commitments made to a generation of teachers who have accepted lower base salaries for long-term benefits. The starting point, in fact, is something many teachers would embrace.

States should give each teacher the right to choose an alternative contract that contains terms and benefits consistent with those in the private sector (e.g., an at-will contract with standard health-care benefits, 401k, etc.), and sits outside of the existing teacher pension system. Choosing this alternative contract would convert any existing pension to a lump-sum 401k contribution. In return, the new contract would have a far higher base salary; in fairness, states should require districts to hire an auditor to determine the savings that can be expected from each alternative contract teacher, and give that savings to the teacher as increased pay.

Under this plan, no current teachers would be forced to change their contracts. If a state chooses to implement this policy change on a school-by-school basis, teachers who choose the current traditional contract might be offered a transfer or be grandfathered, that is, allowed to continue under their current contract. But the alternative contract could be attractive: depending on the state or district, the expected pension-related savings over a standard contract could be as much as $25,000 per year per teacher. In New York City, for example, a teacher might choose her current contract and a $65,000 salary, or the alternative employment terms with a $90,000 salary but with no tenure guarantees. This change would not reduce costs overall, but it would begin to curb the practice of paying operating expenses with long-term, off–balance sheet debt.

Conversion specifics will vary by state; obviously, those with huge unfunded liabilities will have a tougher time finding an elegant solution to converting past pension obligations for teachers nearing vesting milestones. Some percentage of teachers will refuse to switch; every teacher who does switch, though, will reduce the scope of the long-term problem. Many teachers will prefer to have their retirement funds fully in their control, along with a higher base salary, over a pension subject to fierce political pressure.

So which teachers might choose the alternative contract? My hunch is that newer teachers, who would appreciate the extra cash, and high-performing teachers, who would be unconcerned about the decreased job security, would be likely converts. If that’s true, it’s probable that schools with the highest-need students (who traditionally have the least-experienced faculty) would be most likely to convert over to the new contract, and might thereby be able to attract higher-performing teachers.

Schools operating under the alternative contract would be free to evaluate teachers based on student performance and evaluation, as well as classroom observation and other evidence. These teachers could be empowered to shape their schools, by taking part in choosing the curricula they use in their classrooms and the formative assessments they use to measure student progress, for example. Giving teachers a voice in decisions that affect their work is a logical complement of recognizing and compensating them as professionals rather than as assembly-line workers.

Does this proposal solve the compensation problem? Not entirely, though it would take us halfway there. If we also clean up our accountability systems, we could compare the performance of teachers under each contract and adjust the compensation system to include performance metrics as appropriate.

Align Assessment to Curricula

For all their deficiencies, assessments of student learning are an indispensable component of an evolving school system. Without accurate assessments aligned with curricula and standards, education innovators would be flying blind.

The multistate Common Core State Standards project is an improvement over the patchwork of past state standards. But the standards are not the source of flaws in state accountability systems; the culprits are the state tests.

Tests used by international organizations, like TIMSS and PISA, and also our own NAEP (National Assessment of Educational Progress), can measure performance because they’re both broad and deep; they use a reasonable number of items (many of which are constructed responses) against a large number of standards. But that design makes those tests too long to give to one student. Instead, they’re matrixed; 10 students might each take one-tenth of the test. A few thousand well-selected subjects might give us an accurate picture of 4th graders in a state, but these types of tests cannot be used to measure the performance of a student or school.

A state or national test, on the other hand, can only last an hour or two in each subject. Because such tests must contain several items per standard to be accurate, it will measure only a fraction of the standards. And since a test must be reliable from year to year, it will measure that same subset every year. This limitation encourages schools to narrow their curricula to only those standards likely to be measured and gives rise to illusory performance gains. At present, various groups of states are trying to work out this problem. In the end, they’ll trust that the testing companies will solve this problem, and once again, they’ll be disappointed. There’s a better path.

Imagine if states stopped commissioning their own tests and instead created a small set of requirements for each curriculum provider:

• Adopt or create a secure summative test for each grade level. This test should align closely to the curriculum, and every school using that curriculum would use that test to measure student performance.

• Work with client schools to administer NAEP (or some other matrix-based test aligned to the standards) to 2,000 students each year in key grade levels; use their performance to set the curve for the summative test (think of this as “Curriculum NAEP,” the equivalent of the current state NAEP testing).

• Set the curve for tests on a standard score range that facilitates value-added analysis.

This new way of thinking about summative testing would retain the advantages of the Common Core project and the best state tests while eliminating most of the disadvantages. States would retain the authority to determine the curricula they might subsidize or even allow; they might adopt only one for some subjects and grades (say, for K–6 math); in this case, the world would look a lot like it does now. States would be better off, however, allowing schools to adopt curricula, along with the corresponding summative tests, that best fit their students’ needs. Again, it makes sense to empower schools at the same time that we hold them accountable for student performance. Either way, states could continue to compare schools, since each curriculum would be scored on the same curve and the scores equated through Curriculum NAEP.

This proposal would eliminate most gaming around test scores. There would be no incentive for a provider to dumb down its test, since Curriculum NAEP scores (and therefore the curve) would leave scaled scores unchanged. Moreover, the proposal would create a true alignment between curricula and tests, by removing the state as intermediary. Rather than teach to the state test, schools would teach a curriculum, and then test students accordingly.

Best of all, this regimen would encourage differentiation and competition among curriculum providers. In the end, the curriculum generating the best results for a particular cohort (say, middle-school Latina students) would likely be adopted by schools with large groups of those students.

That competition would extend to the tests themselves. A test should be judged not only by its accuracy, precision, and reliability, but also by its ability to promote learning. Many educators believe that authentic assessment (asking students to perform complex tasks rather than answer multiple-choice questions) encourages better teaching and learning; if this proves true, then curriculum providers using authentic assessments would dominate the market, despite their higher costs.

Finally, this approach would save money. Curriculum providers will find much more agile ways to connect to assessment providers than any state consortium has found so far.

Let the Data Flow

If our schools are to continually improve, we need to gather data and make it available not just to schools, school districts, and parents, but also to independent researchers, who can comb the databases for correlations and any underlying causal connections. Our goal should be to create a veritable education genome project open to all appropriate parties, with proper security measures to address privacy concerns.

We currently gather data through a 1970s-era approach that is slow and expensive. As data move from classroom to school to district to state to the federal government, the details that would allow us to draw meaningful conclusions about things like the effectiveness of a textbook, a supplemental services provider, or an afterschool program are lost. Meanwhile, Google and others manage much more data with far less cost and difficulty. We need to adapt their processes to education data.

The testing companies already collect data from individual schools, as they send and collect test booklets either directly or through the district. These vendors are technically savvy and have the incentive to maintain participation in a lucrative assessment market. States should require their testing vendors to collect data from each school in a standard format, including at least the curricular materials used in each classroom, the calendar and schedule in use at that grade level, the background of the teachers, and any academic interventions used for particular students. The companies should be required to then forward these instructional data, along with test scores, subscores on specific components of the test, and student demographic information, to the state in a standardized format. The state, in turn, should publish a database with accounts allowing schools, districts, education consumers, and (in a privacy-ensured format) researchers to access at will.

There are obvious privacy concerns about publishing personal data in a state database. However, these data are far less sensitive than other data that are commonly secured and made widely available. (Just what would someone do with your son’s 5th-grade math grades?)

Thousands of researchers would surely exploit the resulting database. Curriculum providers would look for evidence of their (or their competitors’) effectiveness. Policymakers would examine the results of various interventions, including afterschool programs, changes in class and day length, or class-size reductions. Teacher preparation and in-service training programs would know whether and where they were having an impact. Parents would be able to make informed choices about where to send their children to school.

Most states would save money by making use of this more efficient way to collect data. At the same time, it would spawn a wave of innovation, as various players start using the data.

Innovation and the New ESEA

All four of these proposals would move us away from a command-and-control education system, and toward an agile education marketplace that encourages innovation and excellence. But even if these proposals sound reasonable to you, you’re probably still wondering how and when they might ever come to pass.

The answer is through the next iteration of the Elementary and Secondary Education Act (ESEA); by attaching the mind-set of RttT and i3 to the billions of dollars of annual education aid to states, we can use incentives to encourage the right behaviors quickly and inexpensively. Title I channels $14 billion per year to states, which pass it along to districts along with their own funding. Imagine if the new law leads states to channel that money, along with their own funds, directly to schools, and discourages them from holding to the status quo. With a small tweak (for example, an increase or decrease in funding of 10 percent), the feds would give states a $3 billion push in the right direction.

The language enabling schools to choose a district or SSO should be simple. Each state should find its own path to empowering schools. Perhaps some states would empower high-performing schools first, while others might put failing schools into governors’ districts like the one currently proposed in New Jersey. Perhaps states with higher population density would create statewide choice systems, while others would favor parents who sought short travel times. There are many mechanisms imaginable for allowing a school community to vote on its district or SSO affiliation and for states to license and monitor school support organizations.

Similarly, Title II provides roughly $3 billion per year for professional development. The federal government could limit those funds to states that give teachers the right to choose the alternative contract. Again, though, the new ESEA should allow states great latitude in structuring that right (for instance, they could give that choice to individual teachers, or allow a school-by-school vote); regardless, each state will have to figure out what to do with its pension obligations to teachers who switch to the new contract.

The process by which Common Core states are creating math and English tests is well under way; it may result in top-notch exams that lead to dramatic performance increases. The easiest place to implement an assessment marketplace, then, is in science, history, and language courses. ESEA should establish a group that registers curricula in those areas; if this marketplace proves effective and states struggle with the Common Core tests, this marketplace can easily expand to incorporate math and English.

The accountability provisions of ESEA should require testing companies to phase in collection of school-level instructional and background data. Initially, the testing companies could provide the data to client states for analysis; perhaps down the road, states or foundations will find it useful to run studies across multiple data sets.

None of these proposals is expensive; in fact, most will save money in the short and long term. And although some might be politically inexpedient, none would have the natural and well-funded opponents of other commonsense reforms. Further, this is not an exhaustive list. Every reader of this article could probably come up with additional reforms that would create a more responsive education system.

This plan places a great deal of faith in competition and innovation, though within the construct of a robust public school system. As I’ve noted, this faith could be misplaced: perhaps education truly is different, and there simply is one immutable right way to run schools. But there is something to be said for empowering our schools with transparency, choice, and agility. American ideals shouldn’t just be taught in the classroom; they should shape that classroom.

John Katzman is the executive chairman of 2tor, Inc.

Behind the Headline
The Case for Special EducationVouchers
Education Next | Winter 2010

Former State Assemblyman Michael Benjamin makes the case for special ed vouchers in New York City in an op-ed appearing in today’s Post. Jay Greene and Stuart Buck explained how special ed vouchers work and dispelled myths about the vouchers in an article appearing in the Winter 2010 issue of Ed Next.

Just under 18 months ago, the U.S. Department of Education awarded over $330 million to two state consortia, PARCC and Smarter/Balanced, representing 45 states and the District of Columbia, to design and implement new student assessment systems. Two smaller state consortia, Dynamic Learning Maps (DLM) and the National Center and State Collaborative (NCSC), received an additional $67 million to develop new assessments for students with significant cognitive disabilities. The new assessments, offered mostly online, will replace the current state tests given to millions of students each year in reading and math. At the time, Secretary of Education Duncan called these initiatives an “absolute game-changer” and pledged tests of “critical thinking skills and complex student learning that are not just fill-in-the-bubble tests of basic skills.” In short, it’s an all-out effort to significantly improve one of the weakest — and most despised — aspects of our nation’s current educational system.

But, while it’s easy to think of the consortia as “building tests,” the more apt description is that they are attempting to re-invent, with heavy use of technology, the entire process of assessment. They are developing new types of assessment questions to go beyond multiple choice in conjunction with new methods to deliver, administer, score, and report on these assessments. They will delve deeply into professional development. And, together, they are also adopting common performance standards so that proficiency, which now means different things in different states, is a consistent standard across states.

Officially, the new assessments, including formative and interim tools, will not launch until the 2014-15 school year. In reality, though, most of the work needs to be fully-baked for field-testing in the 2013-14 time frame. That means the real work will take place over the next 18 months. This timeline will increasingly drive both decision-making and expenditures. Even though the consortia have generous grants, doing something quickly, for the first time, and in collaboration across many diverse states costs much more.

Many schools and districts, but not all, will struggle to develop the raw capacity – hardware, software, bandwidth, and tech support – to deliver online testing. Since it takes time for budgeting and procurement, districts want to know right now what the “requirements” are going to be. Yet, there’s a chicken/egg situation because the consortia don’t yet know the content/item types, so they can’t say whether to prepare for bandwidth-hogging simulations, graphics, etc.

At the same time, we have a limited sense of schools’ and districts’ actual capacity. When pushed, they may find a way: As one official at a recent State Education Technology Directors Association (SETDA) event noted, in his state districts and schools felt like they were being pushed off the cliff when online testing was implemented, but in reality, the cliff was only a couple of feet high. While the consortia are developing a “readiness tool” to assess the state of technology down to a school level, they’ll soon have to make a guess as to how ambitious the tech specs will be and that will then become a major constraint to development. And, that guess will have to be made in 2012 about 2015 technology. (iPads were not even around when the Department announced the grant competition.) Lower tech requirements will make schools’/districts’ lives easier, but may limit amount of innovation in item types, data collection, etc. Too far towards the other extreme increases the capacity problem.

From an instructional technology and content standpoint, the enormous scope means that the process by which the consortia do their work may have large implications. For example, if the consortia specify that you must have a device with at least a 13” screen size, good luck selling a 10” iPad tablet. More importantly on the back-end, decisions about the underlying technology architecture and standards for data/content transport will also have implications for both the vendor marketplace and integration of all sorts of other data systems (reporting, analytics, student information systems, formative assessments, content repositories, learning management systems, etc.). In other words, the consortia have the potential to exert a fair-amount of market power in a market that is currently dysfunctional. Whether the consortia choose to wield that power, and whether they do it as a force for good, remains to be seen. Ideally, this will all be done with a keen eye towards interoperability, openness, and extensibility, a system design principle where the implementation takes into consideration future growth. But, designing with the future in mind may take more time, could cost more, and often entails risk – presenting a dilemma for high-stakes development on a tight timeline.

The consortia provide a real opportunity to both understand and upgrade schools’/districts’ technology capacity. As a technology director told me, “they’ll buy for the testing mandate.” Yet, whether this capacity will have dual-use for instruction remains to be seen. Schools could get just enough bandwidth to support testing, but have to shut down any other uses for multiple weeks throughout the year. They could also decide to acquire “secure” computer labs, but isolate these from day-to-day classroom instruction. On the good side, one of the hopes of the new assessments is that they will point instruction to more cognitively challenging and beneficial methods. To the extent that these are technology-based, students must have access not just for testing, but also for instruction.

This may all seem to be too far in the weeds to pay attention. But like it or not, how we measure matters. The next generation of assessments will go a long way towards determining whether digital learning actually fulfills its immense promise. And this may be the best chance to get it right.

UPDATE: Smarter/Balanced and PARCC release statement announcing the new technology readiness tool.

- Bill Tucker

To become proficient at dividing a one-place number into a three-place number, a girl with blue-painted fingernails wrote in her math journal.

To become proficient in multiplying decimals, wrote a dark-haired boy. To become proficient at subtracting one four-place number from another. To become proficient in arithmetic word problems. To complete an exercise in the properties of numbers, like (4 + 9) + 5 = ? + (9 + 5).

No two youngsters seemed to have quite the same math goals because, of course, no two youngsters are quite alike when it comes to learning. That’s why Los Altos is betting the future on an online math program from Khan Academy, and why scores of other schools and districts are clamoring to include Khan Academy in their math curriculum.

For the next 45 minutes, Julian met individually with his 5th graders to refine their goals. (In November, Julian left Los Altos to become assistant principal in the Milpitas Unified School District.) Everyone else logged onto the free Khan Academy web site and called up the “module,” or math concept, that fit their goals. Some watched short video lectures embedded in the module; others worked their way through sets of practice problems. I noticed that one youngster had completed 23 modules five weeks into the school year, one had finished 30, and another was working on his 45th.

As youngsters completed one lesson, an online “knowledge map” helped them plot their next step: finish the module on adding decimals, for example, and the map suggests moving next to place values, or to rounding whole numbers, or to any of four other options.

Julian, meanwhile, tracked everyone’s progress on a computer dashboard that offers him mounds of data and alerts him when someone needs his attention. He showed me, for example, the data for a child who had been working that day on multiplying decimals. The child had watched the Khan video before answering the 1st practice problem correctly, needed a “hint” from the program on the 3rd question, got the 7th wrong after struggling with it for 350 seconds—the problem was 69.0 x 0.524—and got the 18th correct in under a minute.

But just as powerful are the data kids have on themselves. The Covington youngsters regularly pulled up an array of charts that showed them which math concepts they had mastered and which they were working on, needed to review, or were stumbling over.

The classroom buzzed with activity, and amazingly, all the buzz was about math.

Salman Khan (on left) and the team at the Khan Academy.

Khan’s Rise

By now, more than 1 million people have watched the online video in which Salman Khan—a charming MIT math whiz, Harvard Business School graduate, and former Boston hedge-fund analyst—explains how he began tutoring his New Orleans cousins in math by posting short lessons for them on YouTube. Other people began watching the lessons and sending Khan adulatory notes (“First time I smiled doing a derivative,” wrote one) or thanking him for explaining fractions to an autistic son.

Khan quit the hedge fund, moved to Silicon Valley, and in 2009, with funding from a constellation of technology stars (Bill Gates’s children were using the videos), launched the nonprofit Khan Academy. A year later, Mark Goines, a member of the Los Altos school board and a legendary Silicon Valley investor, introduced Khan to the district’s new superintendent. Los Altos already ranked among the best-performing districts in the state, but it had set itself a goal of improving individual achievement, and “capturing data at a granular level” on each student was proving difficult, Goines told me.

A few weeks later, in November 2010, Los Altos agreed to pilot Khan Academy with two classes of 5th graders and two classes of 7th graders and provide Khan with feedback to refine the web site and tools. By summer 2011, some 250 school districts, charter schools, and independent schools were asking to be part of the pilot—Khan chose only a dozen—and have Khan staff work with them to integrate the videos, data dashboard, and other tools into their curriculum.

Salman Khan’s short videos remain the centerpiece of Khan Academy (there already are 2,576 of them and counting). In each one, Khan’s voice describes a discrete math concept, such as solving a quadratic by factoring or interpreting inequalities, while only his hand-scribbled formulas appear on-screen. Khan’s idea was that youngsters would watch the videos at home and work on problems in class, essentially “flipping” the classroom (see “The Flipped Classroom,” What Next, Winter 2012). But teachers told me that youngsters also are using the videos as a just-in-time solution when they’re stumped on a problem in class, or to move ahead when they feel ready.

The data that the web site churns out and the site’s gaming features seem to be the real learning motivators. Youngsters become “proficient” in a concept by answering a “streak” of 10 consecutive computer-generated questions: miss one and the computer sends you back to the start. Youngsters earn “energy points” for correct answers, and badges for accomplishments as diverse as working speedily (that’s a meteorite badge) or becoming proficient in the Pythagorean theorem (that’s a moon badge).

Ted Mitchell, president of the NewSchools Venture Fund and a Khan Academy board member, told me that Khan developers “were blown away by how important” the games and badges seem to be in giving kids a sense of accomplishment and progress. Even older kids, for whom badges are ho-hum, “are instantly motivated” when they complete a streak, and the program acknowledges their accomplishment, says Brian Greenberg, who until recently was chief academic officer of Envision Schools. “What’s brilliant about Khan Academy is the instant feedback,” Greenberg told me.

Envision runs four charters in Northern California, including one that piloted Khan Academy with a small program for remedial-algebra students last summer.

Los Altos has extended the Khan Academy program to all of its 5th and 6th grade classes, and to its 7th graders who were achieving at grade level and below.

The Teaching Curve

From Covington Elementary, I dropped in on Courtney Cadwell’s 7th-grade pre-algebra class at Egan Junior High. She, like Julian, piloted Khan Academy last year. Based on that first-year success, Los Altos extended the program to all of its 5th- and 6th-grade classes, and to its 7th graders who were achieving at grade level and below.

Cadwell, a 17-year teacher who was wearing University of Texas orange for her alma mater, calls Khan just “one resource we use.” The previous night, she had assigned worksheet homework; she began the class with a textbook lesson. Math projects ringed the classroom, a reminder that Khan Academy doesn’t include project-based lessons. That night’s homework included a reading on the origin of zero: Cadwell, among others I spoke with, said Khan’s weakness is that it “is not great at helping kids conceptualize math.”

Khan’s strength became clear a few minutes later when the students opened their laptops. Cadwell strolled the room with an iPad in hand, tracking the youngsters as they moved through problems and modules, and intervening with a quick one-on-one when the data identified a student who was stumped. “I’m getting data in real time about each student instead of assuming the entire class needs intervention,” she explained afterward. Khan “lets me use my class time more wisely.”

It also means that teachers have to figure out new ways to work. “Teachers have to be willing to escape from the role of standing in front of the class” and flexible enough to group kids based on need, said Julian, who was a math coach in New York for 20 years and retains his big-city bustle.

As I watched Julian, Cadwell, and later Ruth Negash at Oakland’s Envision Academy of Arts and Technology, they seemed to be always on the move—meeting individually with children, tutoring small groups, and occasionally addressing the whole class. “I actually work harder” with Khan Academy, Julian said. “I’m up and around more, meeting with kids more.” That gives time back to students and, as Cadwell said, makes them “take ownership of their learning” by setting their own goals.

It also means a new level of classroom collaboration: youngsters can look at each other’s data and identify “coaches” among their classmates. Julian urged his 5th graders to ask the Khan program for a hint, watch a video, or ask a coach for help before coming to him. “Show him how to do it, don’t walk around the class giving answers,” he admonished would-be coaches. Pretty soon, a girl in a pink T-shirt turned to a girl in purple for coaching, and the two worked meticulously at solving 1.94 x 5.52.

The Khan Academy provides data in real time about each student, resulting in more efficient class time management for teachers.

Making It Work in Oakland

Los Altos is an affluent, tech-savvy community; I next wanted to see how Khan Academy could work in an inner-city classroom. So two days later, I visited Envision Academy, a downtown Oakland charter school, and Ruth Negash, an intense 4th-year teacher with wild, curly hair and two education degrees from San Francisco State University.

In 2011, Negash taught two summer-school classes of 9th, 10th, and 11th graders who had failed Algebra I. One randomly assigned class used Khan Academy; the other was a traditional math class. The results were promising enough that Negash now is using Khan in all of her 9th-grade algebra classes.

On the day I visited, Negash started both of her classes with a minilecture on linear equations, and then had her students solve for x in 7x + 4 = 18. The classes quickly became fidgety, first as Negash explained the problem, and then as youngsters finished at different speeds. Negash had to urge them to “respect the community of learning.”

But that changed a few minutes later when the youngsters opened their computers—I had noticed the same change in Cadwell’s class—and worked on Khan Academy for the next 75 minutes. I heard an occasional groan of exasperation. “They threw a trick question at me and sent me back to the beginning,” one boy moaned when his streak was broken. But the energy now was directed toward everyone’s screen.

Although everyone in Negash’s classes had taken, and presumably passed, algebra in 8th grade, their math competence ranged from marginal to impressive. In both periods, three or four youngsters claimed a table in the hallway, where they worked silently at lessons on quadrilaterals and complementary and supplementary angles, typical geometry exercises. But other students struggled with addition and subtraction, and one quarter don’t know their multiplication tables, Negash told me. (To keep those youngsters from falling even further behind, she gives them a reference sheet with the multiplication tables on it.) Negash told both classes to work on the Khan module on solving for a variable—a continuation of her minilecture—but Khan’s online prompts were urging most youngsters to first review lessons on lower-level skills.

Some of these youngsters simply “feel safer” doing arithmetic and will move on when they’ve experienced some math “success,” Negash predicted. Other educators had similar takes: Khan “takes away a lot of the fear about math” by letting kids backfill their gaps and then move ahead at their own pace, said Sandra McGonagle, the principal of Santa Rita Elementary in Los Altos, which also is using Khan Academy in its 5th and 6th grades.

“You don’t have to worry about getting something wrong in front of the whole class,” one of Julian’s 5th graders, the girl with blue nail polish, told me.

But in Negash’s classes, the wide range of math abilities is clearly a challenge. Negash sat with one low-performing student for much of the first-period class and with three others in the second period, hoping to encourage some of that “success.” Meanwhile, other students were calling for her help. Two boys were stumped by “adjacent” in a word problem; language issues crop up “every day,” Negash said.

When Negash finally had a moment to consult her Khan dashboard at the end of second-period class, she saw that one youngster had spent 62 minutes solidly working on math, but another had spent only 14 minutes. “It’s hard to figure out a different plan for 25 kids every day,” she sighed.

Gia Truong, superintendent of Envision Schools, said Khan Academy developers had urged her to let Negash’s students “start where they were” in math and move forward. But that’s creating a conflict when some kids are so far behind, she told me: “If you do that, you might never get to the algebra standards” that California students must pass in order to graduate.

“You’re in the new paradigm, but the grading standards are in the old paradigm,” she added.

Getting to Results

Test results at both Los Altos and Envision—the only two pilots to have any results so far—suggest that Khan Academy is working. Los Altos says that among the 7th graders who used the program in 2010–11—all remedial students—41 percent scored “proficient” or “advanced” on the California Standards Test compared to 23 percent the year before. Among 5th graders, 96 percent using Khan were proficient or advanced compared to 91 percent in the rest of the district.

At Envision’s summer-school program, the youngsters in the Khan Academy class spent only half their time on algebra—the rest of their time was on lower-level math skills—and yet still slightly outscored the traditional class, which spent all of its time on algebra.

Both districts are quick to say that it’s far too early to claim success: there were only 115 youngsters in the Los Altos pilot and just 20 at Envision. “It’s enough to say this is promising; it’s not enough to say this is the future,” former Envision Schools officer Brian Greenberg said.

Most observers of the Khan experiment agree that the measure of success must be student achievement. Otherwise, “I’m not very sympathetic,” said Michael Horn of Innosight Institute. As teaching is increasingly differentiated, however, schools may need a different kind of assessment. California’s year-end test can tell which 5th graders meet the state’s math standards; it can’t tell if some of those 5th graders have progressed to trigonometry or pre-calculus, as two Los Altos kids did last year.

But several experts also suggested measuring Khan’s impact by also looking at changes in the distribution of test scores. Khan Academy isn’t likely to close the learning gap because some kids, freed from the teach-to-the-middle plod of the usual classroom, gallop ahead. But Khan would be a success if low-performing kids move ahead too and “shift the bell curve to the right,” said the NewSchool Venture Fund’s Ted Mitchell.

Some other Khan watchers gave a surprisingly strong endorsement to such measures as student engagement and self-confidence, and to soft skills like goal setting and teamwork. “I don’t look at it as just based on the data,” said Mark Goines, the Los Altos school board member whose high-tech background (he helped develop and run TurboTax for Intuit, Inc.) suggests a fine reading of the data. “The kids seem to be happy about learning. That makes me excited,” he said.

What about increasing class size, I asked: Should Khan’s success be measured in part on its ability to increase teacher productivity? In elementary schools, where students generally spend the day with one teacher, increasing class size because of Khan would mean bigger classes in every other subject, too. And Goines, who said he has viewed “hundreds” of online programs, cautioned that there aren’t any comparable products in other subjects, especially in writing.

A fear among advocates of online learning is that slow learners will be abandoned in front of a computer, and a large classroom increases those chances. “It would then become a babysitting tool,” said McGonagle, Santa Rita’s principal.

At Envision Academy in Oakland, teachers say Khan takes away a lot of the fear about math by letting kids backfill their gaps and move forward at their own pace.

Blending Khan

Finally, I asked for “takeaways” from the Khan Academy experience. Greenberg told me that it’s more important that teachers be “nimble” and “entrepreneurial” than that they be tech wizards. All three teachers said they felt comfortable with technology, but that, more importantly, they were risk-takers. Even before she began piloting Khan Academy, Cadwell asked her PTA to buy classroom laptops for the youngsters in her remedial math class. “I figured if I could get them onto some practice sites, I’d figure things out from there,” she said.

Santa Rita’s McGonagle said it was “crucial” to have pilot teachers like Cadwell who can act as avatars for the rest of the district as it expands its blended learning. Cadwell is mentoring other Los Altos teachers this year. They “don’t need training as much as they need time” with the program, she told me (the data are fairly easy to use, but she and Julian asked Khan’s engineers for so much of it that both say they don’t always use it all).

The schools, meanwhile, are holding rollout meetings for parents and are urging parents to join the web site, where they can see the same data as the teachers, including whether little Bobby is really working on math up in his bedroom as he says he is. “It’s not just training the teachers; it’s training the community,” Goines said.

That training shouldn’t end with just learning to manipulate the data, though. It also means learning how teachers can use their time differently, how to work with youngsters who have different abilities, and how to blend Khan into the curriculum, not substitute for it, everyone told me. Cadwell and Negash said that they find gaps in the Khan curriculum, and that it isn’t completely aligned with either California or core-curriculum standards, although Khan is adding lessons to fill the holes.

“You can’t just put a kid down in front of a computer,” Goines said, although the kids I saw in Julian’s, Cadwell’s, and Negash’s classes sure seemed to enjoy it.

June Kronholz is an Education Next contributing editor.

The Chetty study shows that over a ten year period, the payoff for the students of a very effective teacher amounts to a total of $2.5 million. The harm done by a very ineffective teacher is the same. So if we could replace a terrible teacher with a great one, it would be worth $5 million total for all those kids affected by the switch.  And losing a great teacher, only to hire a bad one, would cost the same.   That’s convincing evidence for those who want to limit the tenure of non-performing teachers while giving the excellent ones their just reward.

But unions want to protect teacher tenure and pay all teachers the same, regardless of effectiveness.  So denying the Chetty study is absolutely crucial.

Though he lacks the necessary econometric skills, Michael Winerip takes up the assignment, claiming the data on teacher effectiveness, which comes from student testing during the 1990s, is too old to tell us anything.

But to ascertain the impact of teaching on student earnings that occur much later in life, it is of course necessary to look at those educated in the 1990s.   Those students have now finished high school (or not), gone to college (or not), and entered the work force (or not).  For today’s students, no one has that information–for the obvious reason that they are still too young.

Aha! says Mr. Winerip. That is the fatal flaw. Back in the 1990s, when students took standardized tests, No Child Left Behind did not exist, so “whether those results are applicable to our post-2004 high-stakes world, we cannot tell.”

If we are to buy this argument, the data will always be too old to tell us anything.  To learn what works we have to wait twenty years, and when that data is available, it will be just too old.

But is it?  Why should we assume that the tests taken back in the 1990s were more accurate than the post-NCLB tests given in 2005, when both teachers and students took them more seriously.  Student performance is more accurately measured when students take a test seriously and when teachers make sure the students understand the testing procedures to be followed. All that is more likely when tests count for something.

So if Chetty and his colleagues could identify large impacts of effective teaching using data from the 1990s, his successors will probably find even larger impacts from more accurate information gathered in the first decade of the 21^st century.

Of course, I cannot prove that, but it is certainly more likely than Winerip’s counter-hypothesis.  While he admits the 1990s tests were accurate, he claims tests today no longer are.  Only if Winerip is willing to make the astounding claim that most teachers today are cheating deliberately and systematically does that assertion hold. Otherwise, we can characterize his argument in one word:  Silly.

- Paul Peterson

When former governors Jeb Bush and Bob Wise strode to the stage at the 2011 Excellence in Action National Summit on Education Reform in San Francisco last October, Sal Khan had just shown the 750 attendees his vision of the digital future.

Khan is the former hedge-fund analyst turned education rock star who started Khan Academy, a nonprofit that reaches millions through its free online lessons and assessments. Tools like these, said Khan, can catapult education from its time-based roots toward a competency-based model in which students progress upon actual learning—mastery—instead of seat time.

At the same conference a year earlier, the two former governors, cochairs of Digital Learning Now!, released “10 Elements of High Quality Digital Learning.” This year, Bush and Wise said they had evaluated each of the 50 states against the elements and explained the assessment methodology they had used: states were judged against 72 individual metrics. (Disclosure: I was one of many who provided feedback on how different states ranked on the criteria and serve as a “digital luminary” for the Digital Learning Now! effort.) Rather than announce where the states fell in the ranking, the governors gave the crowd a preview of their “Roadmap for Reform,” a guide to help states navigate different paths toward changing their online education policies (see sidebar).

With the road map in place, one might assume that moving into the future will be a straightforward exercise: the pieces are all there and model legislation is forthcoming, so state policymakers just have to enact the 10 Elements.

Of course, things are never so simple, and many questions remain.

Some questions reflect legitimate disagreement over Digital Learning Now!’s recommendations, even among those who agree with its broad vision. An obvious flash point will be the idea that states require students to take at least one college- or career-prep course online to earn a high school diploma.

One argument in favor of the requirement is that the outcome from taking an online course—gaining the skills to succeed in a digital environment and perhaps become more self-driven—is valuable in a world in which postsecondary education and workforce training are increasingly done online. Yet some see this as yet another input-based requirement in a system already overburdened with mandates, and in conflict with the spirit of digital learning: if the experience is so important or compelling, won’t students naturally flock to online learning, particularly given Digital Learning Now!’s recommendation that dollars follow students to the online course of their choice?

Another consideration is that elementary-school students don’t take courses—at least in the sense that high-school and middle-school students do—and so ensuring that elementary-school students have access to online learning at the course level seems to miss some fundamental principle. According to the state report cards, though, several states have achieved their goals at the elementary-school level, which only raises more questions.

Many of the pieces that Digital Learning Now! casts as critical to the endeavor are not yet in place, and therefore no one actually knows how they will work in practice. For example, Digital Learning Now! has hitched its wagon to the enactment of the Common Core standards and accompanying next-generation assessments that should be in place by 2014. Whether these assessments will facilitate a competency-based learning environment unburdened by time—or lock in today’s system—is yet to be seen. States may abandon the digital effort when they see the up-front costs of implementing an online assessment system. And if they do, what will that mean for a plan that rests on paying for achievement instead of seat time? Valid, reliable, authentic, on-demand, and independent assessments are critical to moving to a system based on student learning outcomes. What about those courses that don’t fall under the Common Core? Does an outcome-based funding system require extending the Common Core to all subject areas, or will states create unique standards for subject areas other than math and English? Could entrepreneurs develop competency badges for their students that the public would recognize as legitimate? How would such competency measures be accredited?

A number of operational challenges need to be worked out as well. Utah, for example, passed in the spring of 2011 Senate Bill 65, based on the 10 Elements of High Quality Digital Learning. Utah state senator Howard Stephenson declared that the bill ends the “tyranny of time and place” in education by allowing dollars to follow high school students to their online course of choice. The legislation calls for the state to withhold 50 percent of the provider’s fee until the student successfully completes the course.

Not surprisingly, the devil has been in the details. Crafting a viable funding model for online courses that makes sense for districts and providers alike has not been easy. Even more challenging is helping schools and districts transition to a world in which students still need some of the services they provide but take most of their courses online. How does funding work in this model? How do schools create the flexible schedules and offer the critical services—many of which may be nonacademic—to accommodate students’ varying needs? How do they transition to this service—or community center—model?

A related set of issues plagues the funding model from the state’s fiscal perspective. If students progress based on competency instead of cohort, the state should presumably reward schools and providers that help students progress faster. And Digital Learning Now! suggests that it should reward those providers that help students make the most growth. Set aside for a moment the demands on state data systems created by an outcome-based system that rewards growth and the fact that these systems are not in place today. If this policy were in place, the state would be on the hook for paying for a student who masters, say, 20 half-semester courses in a given year, rather than a more conventional 12 or 14. How will states deal with this fiscal uncertainty? Holding back students seems like a poor choice, as does punishing schools that can educate students faster with less revenue.

And what if a student masters the high school curriculum by the time she is 15, as many students undoubtedly could? Does she go to college? Does she take time off? Or does she stay in high school with her friends but take college courses? If so, who pays?

Suggesting that a road map document could tackle such complexity isn’t fair. But a glimpse into the exciting— and uncertain—future presented by Digital Learning Now! does raise many legitimate questions. That’s no reason to delay implementing its recommendations though; innovation is never perfect right out of the box. Iteration in practice is critical. With the “Roadmap” coming on the heels of Khan’s conference presentation, surely some in the audience wondered whether innovations yet to come might even clear away many of the familiar roadblocks.

Michael Horn is cofounder and executive director of education at Innosight Institute.

The market is plenty motivated to create disruptive products and services to serve the public education system, but today’s policies and regulations don’t incentivize and reward those products and services that best serve students. As a result, philanthropic dollars are critical to help create the correct conditions such that those products that are efficacious and serve a higher end—student learning—are the ones that gain share.

As we’ve argued, public policy should reward those providers that best deliver student outcomes—and punish those providers that do not serve the public good.

There is one area, however, where I think philanthropic dollars should probably fund products and services, which is in the category of assessments. If we’re going to have a system that pays providers on how students do on outcome measures, we need robust assessments that are authentic and that people trust. The political incentives—for a variety of reasons—to create high-quality assessments aren’t particularly strong, so having philanthropists invest dollars to create these assessments and continue to push innovation is critical.

This is why yesterday’s announcement that The William and Flora Hewlett Foundation will award a $100,000 prize to the designers of software that can reliably automate essay grading for state tests to drive testing of deeper learning is so important. Open Education Solutions and The Common Pool designed and will be managing the competition.

The Hewlett Foundation’s leadership in creating better assessments to measure critical reasoning and writing is a big step forward—and its use of Kaggle, a platform for predictive modeling competitions, to host the competition is clever.

According to the press release, “The automated scoring competition intends to solve the longstanding problem of high cost and low turnaround of current testing deeper learning such as student essays. The goal is to shift testing away from standardized bubble tests to tests that evaluate critical thinking, problem solving and other 21st century skills.”

In addition, the competition is being conducted with the support of the two state testing consortia that are currently designing the next-generation assessments for the Common Core. Having this buy-in and collaboration gives the competition serious validity and the potential to have real impact.

-Michael Horn

The top-line findings, that blended learning models cost an estimated $8,900 per pupil (+/- 15%) and fully online schools cost $6,400 (+/- 20%) — compared to traditional expenditures averaging $10,000 — will surely be repeated in statehouse policy battles throughout the country. But, those who actually read the short brief will quickly realize that the authors have bent over backwards to caveat their findings in multiple ways. The most important of these caveats? The author’s cost figures reflect estimates of what online and blended schools are currently spending, rather than what they should be spending. In other words, since we have little understanding of how spending relates to student outcomes, the authors cannot say much about either the effectiveness or productivity of this spending. Is it the right amount? We just don’t know.

Still, readers of the paper will better understand the various components of costs in blended and fully online programs – and how they differ from one another and with traditional instruction. These insights should inform those looking to evaluate digital programs by helping them ask better questions about the choices these programs have made and how they align with an overall instructional philosophy. For example, online programs could spend relatively little on content, relying primarily on their teachers to adapt free and open educational resources. In that case, the program would instead need to invest in its educators, ensuring that they have both the support and expertise needed to assemble and modify curriculum. Likewise, programs investing in sophisticated adaptive content will likely pursue a different instructional model.

Finally, one part of the paper will hopefully improve the overall dialogue around potential “cost savings” from digital innovations. The authors correctly note the wide variations in types of blended and online programs, along with the many different reasons that educators and policymakers pursue these programs. Often, advocates confuse attempts to reduce overall costs with efforts to re-allocate the same costs into a different instructional model (i.e., Rocketship). The first results in lower total expenditures. While the latter may mean lower expenditures in certain areas, such as facilities, those savings are put back into different areas in an attempt to be more productive or focus resources on a particularly vexing instructional problem.

As debates around digital learning become increasingly prominent across the country, it would behoove advocates on all sides to better understand the economics behind these programs. This paper is a helpful start, not only for its content, but also for highlighting the ongoing need to better understand the student outcomes that result from these public expenditures.

-Bill Tucker

California policy has done its best to create a byzantine—some might say bizarre—set of regulations to frustrate the power of online learning to do just that. From geographic barriers that limit the ability of students in certain locales to access online learning to restricting blended learning in some unfortunate ways, California has created a maze to frustrate would-be innovators.

There have been some attempts by legislators over the last couple of years to begin to rectify some of these problems, but they have only stalled. Although some charter school operators, such as Rocketship Education and KIPP Empower, as well as some school districts, like Riverside School District, have created stellar blended-learning models, the most advanced school districts in California in online and blended learning have seen their efforts frustrated and curtailed. Even the exciting emerging blended-learning models appearing throughout California in response to tight budgets are limited in how innovative they could be by California’s regulatory landscape.

Against this backdrop, a group called Education Forward has introduced “The California Student Bill of Rights Act”—a proposed ballot initiative that would unlock some of the most onerous barriers to online and blended learning in California. But it would do so in an indirect way.

The initiative is actually not about online or blended learning per se; instead it’s designed to solve one of the most pressing problems facing California students today.

That problem is this: a stunning 1 million high school students in California—roughly 50 percent of the state’s high school student population—attend schools that do not offer the full slate of courses required for admission to the state’s university systems. This means that in many of California’s public high schools, students can graduate, but they won’t be able to get into a UC or CSU college even if they have a good GPA and good test scores.

The initiative solves this problem by creating a mechanism to move beyond simple seat-time funding and instead offer fractional funding to the course level, so students can take courses from an outside institution if their home school doesn’t offer a certain course. The initiative also stipulates that a school or district cannot deny students access to the courses needed for admission to the University of California and California State University systems, including college prep and Advanced Placement courses—a statement of a student’s basic educational rights.

If the initiative gathers the requisite number of signatures to be on the ballot, with a single vote this November, California’s voters could eliminate one of the most egregious examples of inequity in its educational system—and it won’t cost taxpayers any additional funds to do it. This fact alone should allow people from all sides to come together and get behind this.

The initiative certainly isn’t perfect—no initiative or bill is. It leaves a lot of discretion up to several entities, from the departments of education and finance to potentially the legislature—to create the mechanisms to make this all work well. If it passes, the “real” work would likely begin afterward. Some of the organizers behind Education Forward have some clever ideas about how to fund the online courses a student might take, for example—by offering 50 percent of funding to the provider up-front for enrollment, 25 percent for the student passing the course, and the last 25 percent upon successful passage of the state final exam—but this idea, which moves the focus to student outcomes, isn’t codified explicitly in the initiative (although the notion of competency-based learning is, which might lead to such an outcomes-based funding system).

But what successful passage of the measure would do is assert the voice of the people of California as a means to pressure the stalled legislature to do the right thing. And in so doing, it could do more than just solve the problem of equity to high-quality educational opportunities in the state, it also creates a mechanism for competency-based learning, establishes a strong grounding for what online learning and blended learning are, and eliminates the outmoded geographic barriers that prevent students from being able to access high-quality learning opportunities no matter where they originate in the state.

As such, it’s a much-needed breath of fresh air for a state that has been stuck for years now when it comes to education policy—and it could lead the way to bigger and better things ahead.

-Michael Horn

But just as these criticisms aren’t going away, neither is online learning itself. The genie is out of the bottle. So how can we go about drafting policies that will push digital learning in the direction of quality?

This is something we at Fordham are thinking a lot about, and we’ve published three papers (so far) in our series, Creating Sound Policy for Digital Learning: Rick Hess on quality control; Paul Hill on funding; and Bryan and Emily Hassel on teachers. And in January, we’ll publish an analysis by the Parthenon Group of what high-quality fulltime online learning really costs.

I’ll leave it to others to rebut the Times’ extremely selective use of data, expert opinion, and evidence. Where the article landed a punch, in my view, was around the perverse incentives at play today. Clearly K12, and its well-paid CEO, Ron Packard, face strong incentives to boost enrollment at their schools. Unfortunately, states haven’t figured out a way to create similar incentives around quality. And that needs to change.

First, a short digression. I worked at K12 a long, long time ago, just after its creation. (I believe I was employee number 10.) I needed a job, and I convinced Bill Bennett to create a role for me (the august-sounding Vice President for Community Partnerships) in which I would figure out how to take K12’s rich resources and make them available for poor kids. Our basic assumption was that K12’s model—which relied on parents or other caretakers doing most of the instruction—wouldn’t be feasible for kids living in poverty, most of whom would need the custodial care offered by traditional public schools.

To be honest, I didn’t make much progress. The learning materials weren’t even created yet, and so I had few “partnerships” to offer to communities. I left after 9 months for an appointment in the George W. Bush administration.

But what a difference a decade makes. One of the real surprises of the online learning movement is that lots of poor families are choosing to give it a try, and that explains (to a large degree) why K12’s test scores are lagging. (Yes, poverty and achievement are linked, at least for now.)

And the impression painted by the Times article is that online education companies like K12 have every reason to sign up as many parents as possible—poor, rich, whatever—regardless of how prepared they are to tackle the challenge of home-based instruction. Because of some states’ sloppy finance systems, the schools can keep the money if the families change their minds and head back to traditional schools. And, as has been true for all public schools since the beginning of time, the online schools get paid whether their students are learning or not.

Fixing the payment problem is a no brainer. (Schools of all kinds should only get paid for the days of instruction that kids actually show up for.) But is it time to consider performance-based funding, too? To pay companies like K12 more or less depending on how their students perform on state tests or depending on their graduation rates?

In his paper for Fordham, Paul Hill dismisses the idea, arguing:

Pay for performance would create a harsh environment for all education providers. Conventional, virtual, and hybrid schools might spend money on a student’s instruction for a whole course or semester yet receive nothing in return. Online vendors of all kinds, who have little control over their students’ effort or persistence, could be even more at risk. In general, this approach would limit the unproductive use of public funds and quickly destroy any vendor that could not demonstrate good results. It would favor providers with deep pockets, e.g., district-run schools and online vendors supported by large foundations. Performance-based payment as defined here could create a lethal environment for smaller-scale innovators.

He’s probably right about smaller-scale innovators, but I still think it’s worth a try, at least for full-time online schools. (It might be harder in the “blended learning” setting, where a child might be taking just one or two subjects online.)

What if K12 only got paid for every student that made at least a year’s worth of progress on the state test? Some argue that this would create its own perverse incentives, encouraging the company to cherry pick students who are most likely to succeed. But if the measure is student growth, and the test being used is a good one (a big if, admittedly), then all kids but those with severe cognitive disabilities should be seen as contenders.

Instead of indiscriminately signing up students willy-nilly, K12 would then have a reason to vet each family’s situation to make sure they are ready for the rigors of online learning. They would invest, up-front, in assessing whether the child’s parents or other caretakers are up to the task of instructing the student, and whether they have a home situation conducive to success. And then K12 would work like the dickens to make sure every student was making strong progress over the course of the year.

Personally, I’d like to see performance-based pay for all schools. That won’t fly anytime soon, but performance-pay for online learning (at the least the full-time, virtual charter school version) could. Which state is ready to give it a try?

-Mike Petrilli

Kipp shares her experience teaching 11th and 12th grade English online while she resides with her family in rural Colorado. Though not physically in a classroom, Kipp has been able to successfully engage students through live class sessions, emails, instant messaging, and texting. Kipp used to teach in a traditional classroom setting but says that despite some of the unique challenges teaching virtually presents, she finds the online teaching experience more rewarding and in many instances more effective.

My take is threefold. (An earlier version of this originally appeared as an Ed Week commentary, but I thought it worth sharing a tweaked version here.)

First, claims that teachers are, in Duncan’s words, “desperately underpaid,” are a familiar refrain. Yet, given that we’ve steadily boosted staffing and after-inflation spending in recent decades to little obvious effect, and that states and districts are wrestling with structural shortfalls, it’s healthy to question such orthodoxies. Biggs and Richwine remind us that the costs of teacher benefits dramatically inflate the cost of compensation, even if the results aren’t always obvious when scanning a paycheck. Recall, for example, that University of Arkansas economist Bob Costrell pointed out during the Wisconsin collective bargaining fight earlier this year that the average Milwaukee teacher earned a salary of $56,500 but, due to benefits, actually cost the district $100,005 in total compensation. This ought to be of particular concern to educators eager to see more of their compensation show up in their pay stubs. In light of that, I’m disappointed (if not surprised) that most of the responses I’ve seen to Biggs and Richwine have been ad hominem, with Duncan declaring in the Huffington Post that the study “insults teachers and demeans the profession.”

Second, their analysis is intriguing, but it rests upon assumptions and data which deserve to be carefully scrutinized. For instance, Biggs and Richwine rely upon SAT and GRE scores to measure cognitive ability. It’s fair to ask both how good those metrics are and how much they may say about teaching ability. And it’s worth noting that their cognition data are nearly two decades old; if the makeup of the teaching force has changed significantly in that time, it would obviously change the outcomes. Similarly, the job-security and benefits data don’t reflect more recent developments or the fact that teaching positions may be less secure going forward; it will be interesting to see how such changes might impact the underlying data. At the same time, it’s important to note that Biggs and Richwine penned for the HuffPo what I thought was a pretty compelling response to the two methodological criticisms that Duncan had raised.

Third, I ultimately think the are-teachers-overpaid-or-underpaid question is just not that interesting or helpful to those of us in the fields of schooling and education. It’s a useful question for policymakers who must decide how to allocate dollars for highways, health care, and schooling, but for those of us working in the K-12 arena, the more relevant question is: How do we most wisely spend the dollars we have?

For what it’s worth, I’m firmly convinced that, today, some teachers are underpaid and others are overpaid. When I am asked the long-standing question about whether teachers are underpaid or overpaid, my consistent refrain is, “Yes.” I’m much more interested in the broader issue of how we can rethink the profession, make fuller use of talented teachers, and wisely spend the dollars we do have than in debating what the “right” wage level should be.

Under today’s step-and-lane pay scales, the primary way we determine how much teachers are worth is how long they’ve taught and how many graduate credits they’ve accumulated. Now, there’s nothing innately wrong with step-and-lane compensation. Indeed, when introduced in the early 20th century, it was a sensible response to reflexive, sweeping discrimination under which women were routinely paid half as much as their male counterparts. When a captive market of women had few options except to teach, the benefits of this more equitable system outweighed its defects.

Today, however, the world has changed. Whereas limited professional options meant that more than half of women graduating from college became teachers in mid-20th-century America, the figure today is closer to 15 percent. At the start of the 21st century, new college graduates–both men and women–are much less likely to stick to a job for long stretches, the competition for college-educated talent has intensified, and we are becoming better able to track educational outlays and outcomes. All this adds up to a new environment in which step-and-lane industrial-era pay is ill-suited to attracting and retaining talent. The consequence of treating different employees similarly, despite their varying work ethics and skills, has become a growing burden.

As school systems wrestle with tough fiscal decisions, it’s vital to understand that one-size-fits-all pay is insensitive to questions of productivity. Although the term “productivity” is typically regarded as a four-letter word in K-12 conversations, teacher productivity means nothing more than how much good a given teacher can do. If one teacher is regarded by colleagues as a far more valued mentor than another, or helps students master skills much more rapidly than another, it’s axiomatic that one teacher is more productive than the other. Yet, step-and-lane pay makes no allowance for such differences.

Today, we’re paying the most productive employees too little, paying their less productive colleagues too much, or, most times, a little of each. In a world of scarce talent and limited resources, this is a problem. School systems casually operate on the implicit assumption that most teachers are similarly adept at everything. In a routine day, a 4th grade teacher who is a terrific English language arts instructor might teach reading for just 90 minutes. This is an extravagant waste of talent, especially when one can stroll down the hallway and see a less adept colleague offering 90 minutes of pedestrian reading instruction.

One approach to using talent more wisely might entail overhauling teacher schedules and student assignment so that an exceptional 4th grade English language arts instructor would teach many more students. Colleagues, in turn, would shoulder that teacher’s other instructional responsibilities. An essential component of such rethinking is to adjust compensation to recognize the importance of their various roles.

After all, we pay thoracic surgeons much more than we do pediatric nurses–not because we think they’re better people or because they have lower patient-mortality rates, but because their positions require more sophisticated skills and more intensive training and because surgeons are harder to replace. Salary should be a tool for solving problems by finding smarter ways to attract, nurture, and use talent; it should not be an obstacle to doing so.

Almost any effort to really rethink staffing and pay entails some educators earning more–probably, a lot more–and other educators earning less. That sounds about right. The real question isn’t whether we should pay all teachers more or less; it’s how to pay the right teachers more, in a way that serves students and maximizes the bang we get for the educational buck.

-Frederick Hess

This post also appears on Rick Hess Straight Up.

Stanford’s move is significant. But, unless it goes further, Stanford University Online High School is still just a small, selective program for gifted students. Here are two ways to have real impact:

1. Scale the program, allowing tens of thousands of students to participate. At this point, though, the university seems reluctant to grow the school much beyond the size of a typical elite independent school.
2. Generate research and knowledge, helping to define what quality high school online education looks like, what works for whom, what implementation practices matter, and why.

Perhaps Stanford’s move will push other institutions to consider the real game-changer – offering elite quality education, at an affordable cost, on a more massive scale. When will the University of Michigan, UVA, UNC, Berkeley, or any of our other great public universities do this for an entire state?

If the national movement to “reform” public education through vouchers, charters and privatization has a laboratory, it is Florida. It was one of the first states to undertake a program of “virtual schools”—charters operated online, with teachers instructing students over the Internet—as well as one of the first to use vouchers to channel taxpayer money to charter schools run by for-profits….

In Florida, only fourteen months after Crist handed a major victory to teachers unions, a new governor, Rick Scott, signed a radical bill that could have the effect of replacing hundreds of teachers with computer avatars. Scott, a favorite of the Tea Party, appointed Levesque as one of his education advisers. His education law expanded the Florida Virtual School to grades K-5, authorized the spending of public funds on new for-profit virtual schools and created a requirement that all high school students take at least one online course before graduation….

A combination of factors has made this year what Moe calls an “inflection point” in the march toward public school privatization. For one thing, recession-induced fiscal crises and austerity have pressured states to cut spending. In some cases, as in Florida, where educating students at the Florida Virtual School costs nearly $2,500 less than at traditional schools, such reform has been sold as a budget fix.

But, here’s what the article left out: Florida Virtual School, which is prominently connected with privatization in four separate paragraphs of the article, is not a private corporation. It is, instead, a state-owned and state-run institution. There are no shareholders. There are, though, real, live teachers. Led by a former elementary school teacher, the school employs over a 1,000 state certified teachers, almost all of whom have also taught in traditional classrooms. It is fully accredited by two major agencies: The Southern Association of Colleges and Schools (SACS) and The Commission on International and Trans-Regional Accreditation. And, while it is not a charter school, it was the country’s first state-wide Internet-based public high school and has enrolled hundreds of thousands of public school students since 1997.

Florida Virtual School is, in short, a poster child for public sector innovation.

But none of that fit into author Fang’s narrative. It would have made a simple story into the complex one that it is.

K-12 online learning is vast and varied, crossing both political and ideological lines. Programs range from Stanford Online High School to specialized dropout prevention high schools run in partnership with community-based nonprofits toRocketship, which plows savings from technology into extended learning opportunities and higher teacher salaries, to the North Carolina Virtual Public School, a signature program of Democratic Governor Bev Purdue (launched when she was the state’s Lt. Governor). Full-time virtual schools, the majority of which are run under contract to a for-profit schooling company, are part of this landscape. So, too, are numerous traditional school districts — including those who run their own programs and those who oversee contracts with private providers.

Within this landscape, as in any new arena, there are areas of serious concern. Among full-time online learning programs, what we know of performance is decidedly mixed. And, Fang’s article is correct to point out the moneyed influence and lack of transparency from operators like Ohio’s White Hat Management. We’ve been writing about these issues for several years.

But, just as alternative energy should not be defined by Solyndra, neither should online learning be defined by White Hat. Strong oversight to ensure both high quality learning experiences and accountability for public funds is essential. So, too, is knowledgeable and objective reporting.

Schools will not need as many conventional teachers as they did yesterday, but those they need will be able to tap top-notch technology and instructional support teams to achieve excellence at scale. They’ll get paid more, too, potentially a lot more. And all this can be done within tight budgets so long as education systems judiciously blend technology and people.

Digital learning has the potential to transform the teaching profession in three major ways:

• Extending the reach of excellent teachers to more students.
• Attracting and retaining more excellent teachers.
• Boosting effectiveness and job options for average teachers.

Extending the reach of the best. In the digital future, teacher effectiveness will matter even more than it does today. As digital learning spreads, students worldwide will gain access to core knowledge and skills instruction. What will increasingly differentiate outcomes for schools, states, and nations is how well responsible adults carry out the more complex instructional tasks: motivating students to go the extra mile, teaching them time management, addressing social and emotional issues that affect their learning, and diagnosing problems and making the right changes when learning stalls.

The top 20 or 25 percent of teachers already meet these challenges. But in traditional classrooms, they only reach 20 to 25 percent of students. That’s where digital learning can help.

Digital technology, along with changes in teacher roles and schedules, should make it possible for top teachers to assume responsibility for all students, not just 20 or 25 percent of them

For example, by replacing 25 – 50 percent of teaching in some subjects, digital instruction can free excellent teachers’ time, enabling them to take responsibility for more students – keeping similar class sizes and gaining planning time. These “time-technology swaps” are already used in top-performing schools that combine digital learning with excellent teachers to boost results.

Digital tools can also connect excellent teachers working live with students across the hall, state, or nation – using web cameras and email. Shy instructional masters can help design smart software to personalize learning. Star performing content masters can go viral on digital video, and someday holograms, to millions of students anywhere, who with excellent teachers can convert that access into stellar learning.

Attracting and retaining the best. Digital learning will also transform career opportunities for excellent teachers. As they reach more students, they should earn more out of the per-pupil funds generated by the expanded number of students. The chance of enhanced advancement and pay will, in turn, make the profession a more attractive long-term career for top performers, wooing unfulfilled engineers and lawyers into a better life.

-Bryan Hassel and Emily Asycue Hassel

This blog entry also appears on Flypaper. It is based on “Teachers the Age of Digital Instruction,” a paper published this week by the Fordham Institute as part of its Creating Sound Policy for Digital Learning series.

“Teachers,” written by Public Impact’s Bryan and Emily Hassel, opens with an important and refreshing perspective: “that digital education needs excellent teachers and that the teaching profession needs digital education.” Rather than replacing teachers, the authors see digital learning as transforming teaching — both by offering tools for traditional classroom teachers and by enabling entirely new ways of teaching. Often missing from conversations around technology, the paper outlines the varied roles that teachers play, including helping with motivation, social and emotional support, and stretching critical thinking and analytical skills. It concludes that the future is a much more differentiated field, with a smaller number of higher-paid, more empowered teachers acting in teams with a variety of specialized and lower-paid support personnel. (School of One offers one glimpse of this future.)

Some of the paper’s most interesting discussions touch on new administrative structures and the role of unions. The authors see today’s teacher evaluation battles as a relic of an old one-classroom, one-teacher model. Instead, they envision a different form of accountability, such as that in a small professional firm, where one person takes on both leadership and administrative responsibility to coordinate a variety of teaching personnel and supporting technology tools. They reject the notion that digital learning is necessarily a union-killer. Instead, they see a role for a new type of union, modeled perhaps after the Screen Actors Guild, which provides employment and pay security in increasingly differentiated teacher roles, but does not constrain top performers. One quibble though, is that when the paper discusses these new models, it too often uses a static, more-effective/least-effective teacher frame. A more helpful frame might place the same weight on effective teaching, but explore the interdependency between a teachers’ role and effectiveness.

Overall, the paper both rightly recognizes the fallacy of technology replacing teachers and appropriately posits that digital tools will be limited in potential if shoved into traditional teaching models. Additional exploration should go even further, contemplating how digital learning might also change and possibly more tightly align the roles of informal and out-of-school educators, including those in museums, cultural institutions, youth development programs, and of course, homes.

The second paper, written by Paul Hill, details how current school funding systems conflict with new forms of digital learning that cross school, district, and time boundaries:

The problem boils down to this: Our system doesn’t fund schools, and certainly doesn’t fund students. It funds district-wide programs, staff positions, and so forth. This makes it difficult, if not impossible, to move money from concrete facilities, established programs, and entrenched staff roles to new uses like equipment, software, and remote instructional staff. Yet to encourage development and improvement of technology-based methods, we must find ways for public dollars to do just that—and to follow kids to online providers chosen by their parents, teachers, or themselves.

Hill blames this funding rigidity — not the lack of ideas from teachers, principals, or innovators — for the relative scarcity of education innovation at scale. And, while states have developed workaround solutions, few go far enough. His solution: a new “follow-the-child” funding system. While many states already have what is often called weighted-student funding, where funding follows students to their educational institutions and is weighted to account for greater needs, such as those of an English language learner, a new system needs to go beyond “whole school” models. In other words, if digital learning “unbundles” school so that students can choose courses and learning experiences from multiple places, as in Florida and other states, then funding needs to be just as nimble. And, it even needs to accommodate parents who want to assemble their own learning experiences. Hill says:

Funds available for a child’s education must include all the taxpayer funds available to support students’ education. To make this happen, some government entity would need to assemble all of the funds available from all sources for K-12 education in a locality, keep an account for every student, and faithfully allocate its con-tents to whatever school or education program a student attends.

Hill goes on to discuss important implications of these ideas, including dilemmas around accountability and choice. And, while many might reflectively reject Hill’s ideas as a digital-age voucher, there’s also the kernel of another more radical idea. If taken to its logical extreme, localities might not just assemble K-12 funding, but also those for all sorts of other services, such as juvenile justice, mental health, out-of-school programs, etc., enabling an approach that just might resemble a digital-era Harlem Children’s Zone.

Faced with a $30 million shortfall in its $295 million budget for the 2011–12 school year, the Adams 12 school district in north Denver laid off custodians, furloughed teachers, trimmed programs, reduced benefits—and then took its budget scalpel to student activities.

The district dropped middle-school sports, cut back on travel for its high-school teams, and pared $500,000 from the $2 million budget that supports afterschool activities like the Math Olympiad and spelling bee at Centennial Elementary, the technology and drama clubs at Rocky Top Middle School, and the anime (Japanese animation) and Knowledge Bowl clubs at Mountain Range High.

Christopher Gdowski, superintendent of the 42,000-student district, talks hopefully of volunteers stepping in to fill some of the gaps. The YMCA has approached him about taking over some of the sports teams, even offering to buy the used school uniforms and the licensing rights to the school mascots. But some activities may have trouble finding sponsors, he concedes, and teachers union contracts may preclude others from turning to the community for advisors.

“We’re hoping for the best, but we’re fearing the worst,” Gdowski told me.

With school districts struggling to keep their noses above choppy budget waters and voters howling about taxes, should schools really be funding ping-pong and trading-card clubs? Swim teams, swing dancing, moot court, powder-puff football? Latino unions, gay-straight alliances, the Future Business Leaders of America, the French Honors Society, the jazz band, the knitting club? The barbell club at Adams 12’s Niver Creek Middle School?

As it turns out, maybe they should. There’s not a straight line between the crochet club and the Ivy League. But a growing body of research says there is a link between afterschool activities and graduating from high school, going to college, and becoming a responsible citizen.

“Honestly, the place that best prepared me for college was the hardwood court of men’s varsity basketball” in high school, Andrew Snow, a University of Michigan senior and pre-law major, e-mailed me recently. “That court taught me hard work, sacrifice, teamwork, humility…and leadership,” he added, plus, “how to deal with people in social situations” and “responsibility off the court [because] if you made a bad decision, someone would see it.”

Artists for Humanities serves 250 teens annually in an intensive arts micro-enterprise program.

Cause or Effect?

The U.S. Department of Education last compiled data on extracurricular activities a decade ago, when it reported that more than half the country’s high-school sophomores participated in sports, that one-fifth were in a school-sponsored music group, and that cheerleading and drill teams, hobby, academic, and vocational clubs each involved about 10 percent of kids.

At affluent suburban schools, the choice of activities can be dizzying. Walt Whitman High School in Montgomery County, Maryland, a Washington, D.C., suburb, offered 89 clubs (equestrian, Persian, unicycle…), 26 sports, seven choral ensembles, seven bands or orchestras, a newspaper, a literary magazine, and a yearbook last year.

Whitman’s feeder school, Thomas W. Pyle Middle School, offered even more: 100 activities, including a stock market club, cooking, a math team, and a magic club.

Whitman says that 96 percent of its students go to college; its SAT scores in math and critical reading are 250 points above the national average. That isn’t because it has an equestrian team and a Shakespeare club, of course. The education department data show that kids from families in the top third by income and education are half again as likely to take part in sports and almost twice as likely to participate in music as kids from the bottom third. Almost 80 percent of the adults in Whitman’s zip code are college graduates, and the median household income is three times the U.S. average.

The data also show that kids with the highest test scores are the most active in afterschool activities. Two-thirds of kids in the top quarter of test takers played sports, for example, compared to less than half in the lowest quarter.

So, is there a link? Did kids who joined afterschool activities become good students, or did good students join afterschool activities?

As with a lot of social science research, the findings about extracurriculars aren’t always consistent or conclusive: You can’t randomly assign kids to soccer, after all. But some researchers insist there is a cause-effect relationship between activities and academic success, not just the other way around.

Margo Gardner, a research scientist at Columbia University’s National Center for Children and Families (NSCF), is among them—and certainly not alone. Using data from the 1988 National Education Longitudinal Study (NELS), and controlling for poverty, race, gender, test scores, and parental involvement, Gardner has calculated that the odds of attending college were 97 percent higher for youngsters who took part in school-sponsored activities for two years than for those who didn’t do any school activities.

The odds of completing college were 179 percent higher, and the odds of voting eight years after high school, a proxy for civic engagement, were 31 percent higher.

Gardner repeated the analysis using propensity-score matching, that is, comparing kids whose profiles suggested they had a similar propensity either to join or sit out afterschool activities. Even within those groups of similar kids, those who participated in activities had better school success rates than those who didn’t.

The National Center for Education Statistics, in its own analysis of the longitudinal or NELS data, found that high-school seniors who were involved in school activities were less likely to cut class and play hooky than kids who weren’t involved. Three times as many had a GPA of 3.0 or higher; twice as many scored in the top quarter on math and reading tests. And 68 percent expected to get a college degree, compared to 48 percent of kids who weren’t involved in school activities.

Other researchers have approached the question differently, but come up with complementary results. Angela Duckworth, a University of Pennsylvania psychologist, looked at college activities as a predictor of success. She rated the résumés of recent graduates who were applying for their first teaching jobs. She gave the highest scores to those people who had been in a college activity for several years, any college activity, and who had attained a level of leadership or achievement (say, MVP on the softball team).

Those with the highest “grit” scores, as she calls them—with the most persistence—turned out to be the best teachers, based on the academic gains of their students. As an added bonus, the “grittiest” scorers also were more likely to stay in their jobs rather than quit midyear.

Duckworth attributes the difference to perseverance rather than talent: There wasn’t any significant difference in teacher effectiveness based on the SAT scores and college GPAs of the job applicants, she calculated. This isn’t just about whether teachers are new, Duckworth told me: People who are persistent and passionate about something, whether cross-country or baton twirling or spelling bees, will carry over that enthusiasm to other parts of their lives.

Similarly, Betsey Stevenson, an assistant professor at the University of Pennsylvania’s Wharton School of business and currently the chief economist at the U.S. Department of Labor, has found a link between high-school sports and girls’ success. Stevenson compared the college-going and labor-force rates between girls who attended high school before the 1972 passage of Title IX and those who attended after. Title IX, an amendment to the Civil Rights Act of 1964, required high schools and colleges to offer girls and boys the same opportunities to play sports.

Again controlling for age, race, and their state of residence, Stevenson calculated that for every 10-percentage-point rise in the number of girls playing high-school sports in any state there was a 1-percentage-point increase in those going to college and a 1- to 2-point rise in those with jobs. Title IX led to a 30-percentage-point rise in girls’ sports participation, she adds.

The National Center for Education Statistics found that high-school seniors who were involved in school activities were less likely to cut class and play hooky than kids who weren't involved. Three times as many had a GPA of 3.0 or higher.

Engaging Students

Temple University psychologist Laurence Steinberg, whose book, You and Your Adolescent: The Essential Guide for Ages 10–25, discusses afterschool activities. He suggested two more reasons for what he believes is a causal link between activities and academic success.

Kids who are involved in clubs and sports spend an extra couple of hours a week with an adult, usually a role model like a drama director or a football coach. “They don’t want to disappoint the coach,” Whitman’s principal, Alan S. Goodman, told me. All he has to do to straighten out a misbehaving athlete is to threaten to talk to the coach, he said: “‘Oh no, don’t talk to the coach,’ they tell me.”

Extracurriculars also make school more palatable for a whole lot of kids who otherwise find it bleak or unsatisfying, Steinberg said. Grades improve not because of what kids are learning in the video club, but because the video club is making them enjoy school more, so they show up more often, find a circle of like-minded friends, and become more engaged in school.

Christopher Gdowski, Adams 12’s superintendent, echoed Steinberg when I asked him what he meant by “fearing the worst” if some afterschool activities are canceled. His district polled thousands of taxpayers as part of its budget process: A huge majority opposed eliminating all activities, but most agreed on trimming the number of activities each school could offer.

Gdowski said he worries that for “some meaningful number of kids,” those activities are what brings them to school. “That’s the hook,” he said, and budget cuts could leave that hook unbaited.

Penny-wise?

After years of steady increases in education spending, and with the expiry of federal stimulus funds, school districts are facing some unaccustomed belt-tightening this year. K–12 spending rose 39 percent between the 1989–90 and the 2007–2008 school years, according to the U.S. Census bureau, and hit $605 billion in 2009, the latest year for which it has reported numbers.

But the National Business Officers Association has calculated that spending is expected to be off $2.5 billion this year from a year earlier. Florida’s 2012 budget cut K–12 spending by 8 percent, or about $540 a student. Arizona cut $183 million from K–12; New York cut more than $1 billion, and Colorado cut $250 million, according to the National Conference of State Legislatures.

The Center on Education Policy surveyed districts in the spring and found that 46 percent expect funding decreases of 5 percent or more in the 2011–12 school year (the poll asked districts about their “total funds available” for the year, excluding federal stimulus monies).

Staff salaries and benefits are taking much of the hit. But as bus routes, textbook purchases, and even cleaning supplies come under budget scrutiny, it’s no surprise that extracurriculars are in for some pain, too.

Diane M. Place, superintendent of the 1,700-student  Towanda, Pennsylvania, school district, told me she received hate mail and “horrendous calls” when she recommended a $30-a-household tax increase to close a $2.2 million gap in her $24 million budget. Instead, she cut the instruction budget by 9 percent and then went after extracurriculars. She eliminated the rifle and junior robotics clubs, JV soccer, majorettes and one cheerleading squad, and halved the funding for the forensics team and Future Business Leaders.

The 1,000-student Salida, Colorado, school district, facing at least a $500,000 budget gap, moved to a four-day week, and then announced plans to cut Key Club, Math Counts, jazz, and weight lifting. Coos Bay, Oregon, planned to let go a Knowledge Bowl coach in the middle school and a forensics coach in high school after the district chopped $44,000 from its activities fund. Cincinnati is thinking of shifting all of its extracurriculars onto a community group, a move it predicted will save $250,000 a year, largely in teacher coaching stipends.

Extracurriculars teach a lot of the skills you need as an adult: time management, leadership. self-discipline, and persistence.

Or Pound-Foolish?

There’s no ready estimate of how much districts spend for extracurriculars: Districts account differently for teachers’ afterschool pay (it can be lumped in with merit pay, says Stephen Frank of Education Resource Strategies), whether they include team buses in the extracurricular budget, how much they depend on parents and booster clubs for field maintenance and stage-set construction, if and how much they charge students to participate, whether they use federal Title I funds for afterschool enrichment, and so on.

Marguerite Roza, who studies school finance at the University of Washington, calculates that districts spend about the same to suit up a youngster to play a sport as to enroll her in a semester of, say, history. A difference is that there are three seasons for sports, but two semesters for history.

Districts increasingly are depending on kids and their parents to fund extracurriculars. State laws, not national policy, determine which school expenses must be taxpayer-funded and which can be charged to students as user fees. California recently settled a lawsuit brought by the American Civil Liberties Union against dozens of Golden State schools that levied fees for classroom materials, lab fees, and afterschool activities.

But elsewhere, the Pennsylvania School Boards Association counts 33 states where at least some school districts charge athletes anywhere from $25 to $1,500. The band fee at Medina Senior High in Ohio is $200. Arlington, Massachusetts, public schools charge youngsters $405 to join the cheerleading squad and $480 to wrestle. Lakeville, Minnesota, charges $190 to join the debate team and $110 for the chess club.

Many of the best student-athletes, musicians, actors—even cheerleaders and debaters—already are paying lots more than that for private lessons. And some of the most talented spurn their school’s programs in favor of club soccer teams and community orchestras, arguments that budget cutters sometimes cite for trimming extracurriculars.

But Steinberg counters that no one suggests eliminating math classes for mediocre students, or depending on private tutors for calculus. “You could extend that argument out to its illogical extreme,” he said.

At Whitman High, where kids pay a $40 district-wide activities fee, Goodman told me he would rather increase class size than eliminate activities. “You can cope with an extra kid in your class, but at 2:10” when school lets out and intramural basketball is canceled, “what do they do?”

Police statistics offer one answer: Juvenile crime peaks between 3 p.m. and 6 p.m. Education department data offer another: 31 percent of high-school seniors watched three or more hours of television every weekday in 2004, the last time the department ran the numbers, up from 9 percent in 1992.

Students write songs in an after-school program run by ZUMIX. The program offers young people the opportunity to travel thoughout New England, performing their original songs and engaging with other musicians.

Lessons That Last

Tony Wagner, codirector of the Change Leadership Group at Harvard’s Graduate School of Education, told me he did a focus group a decade ago with college students who graduated from a leading public high school in New England. He asked them what “important things” they remembered about high school, three to five years after leaving.

“They described all their experiences in extracurricular activities and sports. This went on for an hour,” he said. But about what the remembered from their academics, “they said, ‘you basically start over.’”

The takeaway, Wagner said, is that extracurriculars “teach a lot of the skills you need as an adult: time management, leadership, self-discipline, and persistence for doing work that isn’t extrinsically motivated.” That dovetails with Wagner’s academic work, which defines the “skills of the future” as including adaptability, leading by influence, and initiative.

“Kids who have a significant involvement in an extracurricular activity have a capacity for focus, self-discipline, and time management that I see lacking in kids who just went through school focused on their GPA,” he told me. Like Gardner and Duckworth, he doesn’t single out football players over the engineering team, or vice versa. The kind of activities “seems not to matter; what matters is the level of engagement,” he said.

I tested Wagner’s conclusion using an updated version of the focus group: I posted a question on the Facebook pages of my college-going sons. I asked their friends what they learned in high school that best prepared them for college, and received answers that were carbon copies of Wagner’s.

No one dumped on high school—“It’s not that I didn’t have fine teachers,” Andrew Snow e-mailed me—but no one credited AP chemistry with preparing them for college, either. In fact, no one mentioned classes at all. Instead, they wrote that extracurriculars introduced them to new ideas and interests, taught them to study more efficiently, developed their social skills, and exposed them to caring adults. “Coach was a maker of honorable men,” wrote Snow.

Justine Mrosak, a first-year medical student at the University of Minnesota, wrote that high school taught her “how to balance my academics with other passions.” Basketball and choir took time, she wrote. “But I didn’t want to give up doing the things that I loved just to get good grades, so I really learned how to schedule my time, prioritize my activities, and make my studying [as] efficient as possible.”

Steven Zuckerman, a pre-law major at the University of Michigan, wrote that “the most valuable thing” he learned was “to challenge my inhibitions by trying new things.” That meant playing sports “I had never tried before,” joining clubs “about things that I never thought would interest me,” and, inevitably, meeting “people with whom I never saw myself connecting.” That curiosity has followed him into college, where he has worked on political campaigns, he says.

I’d rise to the defense of Algebra I any day, and I assume any social scientist would, too. But, leadership, adaptability, social skills? Try a couple years on the school newspaper to learn that.

June Kronholz, a contributing editor, spent four years on her high-school newspaper and 30 years at the Wall Street Journal.

To get those challenging, high-wage jobs, today’s students are going to need college courses, vocational certification, or job training, many of which will be offered virtually. For perspective, nearly 30% of college and university total enrollment is online enrollment.

Schools must equip students with the knowledge and skills they will need for future success, and today that means using a technology-inclusive education. Yet less than 10 percent of our nation’s students are benefiting from digital learning.

Thankfully, state leaders are recognizing the power of digital learning and the impact it has on students’ future success. Michigan and Alabama were the first states to require an online learning experience in order to graduate high school. New Mexico requires students to either take an online course, a dual-enrolled course or an Advanced Placement course in order to graduate high school. This year, Florida passed an online course requirement.

Because of the leadership of Governor Otter and Superintendent of Public Instruction Tom Luna, Idaho is the latest state in a nationwide movement to use technology to prepare students to achieve in and outside of the classroom. By requiring students to take at least two credits online to graduate, the state is arming its kids with the knowledge and skills they will need to thrive in our increasingly digital world.

Equipping every student with a personalized digital learning experience is a must in today’s digitally-driven society. Digital learning can customize education with high expectations and ensure that all students graduate from high school with the knowledge and skills to succeed in college and careers. It leverages the power of technology to give students the ability to learn in their own style, at their own pace, providing all students the opportunity to achieve

Idaho’s actions are trailblazing a path of bold reforms that make systemic changes in education and extend customized digital learning to all students.

-Jeb Bush

As a “practitioner” of sorts, I’ve wondered the same thing for 15 years. The K–12 school sector generates little empirical research of any sort. And of this small amount, most is targeted to policymakers and superintendents, and concerns such matters as the effects of class size reduction, charter school attendance, or a merit-pay program for teachers. Why is there virtually no empirical education research meant to be consumed by the nation’s 3 million teachers, answering their questions?

Those 3 million teachers generate about 2 billion hour-long classes per year. We do not know empirically which “teacher moves,” actions that are decided by individual teachers in their classrooms, are most effective at getting students to learn. Why doesn’t this kind of research get done?

Mr. Gates has part of the answer. Money. For 2011, the Microsoft R&D budget is $9.6 billion, out of total revenue in the $60 billion range. The U.S. Department of Education’s Institute of Education Sciences (IES) represents only a fraction of total education research, but its budget gives some perspective: IES spends about $200 million on research compared to more than $600 billion of total K–12 spending. So, 15 percent to upgrade Microsoft, 0.03 percent to upgrade our nation’s schools. And while Microsoft’s research is targeted to the bottom line ($8.6 billion is on cloud computing, the profit center of the future), IES spends almost nothing examining the most important aspect of schools: the decisions and actions that individual teachers control or make.

One IES project is the What Works Clearinghouse (WWC), established in 2002 to provide “a central and trusted source of scientific evidence for what works in education.” The WWC web site lists topic areas like beginning reading, adolescent literacy, high school math, and the like. For each topic, WWC researchers summarize and evaluate the rigor of published studies of products and interventions. One might find on the WWC site evidence on the relative effectiveness of middle-school math curricula or of strategies to encourage girls in science, for example.

But there is almost nothing examining the thousands of moves teachers must decide on and execute every school day. Should I ask for raised hands, or cold-call? Should I give a warning or a detention? Do I require this student to attend my afterschool help session, or make it optional? Should I spend 10 minutes grading each five-paragraph essay, 20 minutes, or just not pay attention to time and work on each until it “feels” done?

And the WWC’s few reviews of research on teacher moves aren’t particularly helpful. A 63-page brief on the best teaching techniques identifies precisely two with “strong evidence”: giving lots of quizzes and asking deep questions. An 87-page guide on reducing misbehavior has five areas of general advice that “research supports,” but no concrete moves for teachers to implement. It reads, “[Teachers should] consider parents, school personnel, and behavioral experts as allies who can provide new insights, strategies, and support.” What does not exist are experiments with results like this: “A randomized trial found that a home visit prior to the beginning of a school year, combined with phone calls to parents within 5 hours of an infraction, results in a 15 percent drop in the same misbehavior on the next day.” If that existed, perhaps teachers would be more amenable to proposals like home visits.

By contrast, a fair number of medical journals get delivered to my house. They’re for my wife, an oncologist. They’re practical. In each issue, she learns something along these lines: “When a patient has this type of breast cancer, I currently do X. This study suggests I should do Y.” There is a bit on medical policy, but most of the information is meant for individual doctors in their day-to-day work.

That’s not to say that we shouldn’t conduct research on education policy. My own work has certainly benefited from it. For example, the quasi-experimental study by economists Tom Kane and Josh Angrist on Boston charter schools, which compared the winners and losers of charter admission lotteries, helped change the Massachusetts law that had blocked the creation of new charters. The change enabled me to help launch a new charter school, MATCH Community Day. My point is simply that relative to education policy research, there is very, very little rigorous research on teacher moves. Why? Gates knows it’s more than a lack of raw cash; it’s also about someone taking responsibility for this work. “Who thinks of it [empirical research on teachers] as their business?” he asked. “The 50 states don’t think of it that way, and schools of education are not about [this type of] research.”

I agree, but I contend there are a number of other barriers. The first is a lack of demand.

The Demand Side

Why aren’t teachers clamoring for published research? One reason is that researchers generally examine the wrong dependent variable. Researchers care about next August (when test scores come in, because they can show achievement gains). Teachers care about that, too, but they care more about solving today’s problems (see sidebar, page 26).

A second issue is that researchers don’t worry about teacher time. Education researchers often put forward strategies that make teachers’ lives harder, not easier. Have you ever tried to “differentiate instruction”? When policy experts give a lecture or speak publicly, do they create five different iterations for their varied audience? Probably not.

The return on investment for teacher time and the opportunity cost of spending it one way rather than another is rarely taken into account. In what other, valuable ways could teachers be spending the time taken up with building “differentiation” into a lesson plan? They could phone parents, tutor kids after school, grade papers, or analyze data. Much research implies that teachers should spend more time doing X while not indicating where they should spend less time.

Teachers don’t trust research, and understandably so. There’s a lot of shoddy research that supports fads. Experienced teachers remember that “this year’s method” directly contradicts the approach from three years ago. So they’d rather go it alone. Newer teachers pick up on the skepticism about research from the veterans.

Unlike medical research, teacher research rarely examines possible side effects, and whether they are short-term aggravations or can be expected to persist. Imagine that a teacher reads an article arguing that students benefit from being asked “higher-order questions.” She begins doing that. Some students, surprised at this new rigor, are frustrated. Some students throw up their hands and give up. Misbehavior ensues.

Student frustration is probably a fairly predictable short-term side effect of asking higher-order questions. If she isn’t being properly warned, a teacher might quickly abandon this technique.

For all these reasons, the 3 million teachers aren’t forming picket lines to demand research.

Do We Know What Works?

Neither policy camp, reformers nor traditionalists, care much about research into teacher moves, either. Some traditionalists see teaching as an art, one that cannot be subjugated to quantitative analysis (“every teacher is different”). Others aren’t averse to research; they simply don’t see it as a priority. They’d prefer that limited resources be used to fight poverty, not to improve students’ day-to-day classroom experiences.

Meanwhile, some reformers argue “we already know what works,” and we just need to scale it.

As part of the “reformer” community, I find this troubling. From charter opponents like Diane Ravitch to supporters like education secretary Arne Duncan, there’s agreement that “some charter schools work.” Furthermore, there’s strong evidence that the charters that succeed tend to be “No Excuses” schools. So do we know what works?

I’m the founder of one of those charter schools; our high-school students have the highest value-added gains of all 340 public high schools in Massachusetts. I’m also the founder of a small teacher residency program that supplies teachers to schools like KIPP (Knowledge Is Power Program). Many of us would agree to a very different proposition: We know teacher moves “that work” to some extent, enough to create very large achievement gains, but we don’t know teacher moves well enough to get our college graduation rate near where we’d like it to be. Nor do we know how to help teachers do these moves more efficiently, so that their jobs are sustainable.

Without a massive uptick in our knowledge of teacher moves, we’ll continue on the current reform path. That path is a limited replication of No Excuses schools that rely on a very unusual labor pool (young, often work 60+ hours per week, often from top universities); the creation of many more charters that, on average, aren’t different in performance from district schools; districts adopting “lite” versions of No Excuses models while pruning small numbers of very low performing teachers; and some amount of shift to online learning. Peering into that future, I don’t see how we’ll generate a breakthrough.

Bridging the Divide

The final barrier to research on teacher moves is the divide between practitioners and researchers. My analogy is a 5th-grade dance. Boys stand on one side. Girls stand on the other. There is very little actual dancing. In this case, teachers are off to one side, and quantitatively oriented researchers are on the other.

After a while, the boys go into the hallway and talk about video games. Similarly, quantitative researchers find the transaction costs of setting up experiments are too high and give up on doing research about teacher actions. They take their problem-solving marbles and find other data sets to crunch.

Girls see that the boys aren’t around anymore. So they dance with each other. Teachers and school leaders, if they like to learn, do so through observation of and conversation regarding perceived “best practices.” There aren’t many practitioners who care about rigorous empirical research.

With all these barriers, is there much hope? There’s not going to be a pot of gold in this funding environment. If research on teacher moves matters, we need to be more creative about catalyzing the low-hanging fruit. That would mean identifying practitioners who are unusually interested in randomized research, and connecting them with doctoral students who are unusually interested in teachers and teaching.

What does it look like when practitioners and researchers dance together? Here is one example.

In July 2010, I asked Harvard economist Roland Fryer for some help. My research question was fairly simple: Do teacher phone calls to parents “work”?

In our school, teachers proactively phone parents. Typically, the parents have not been heavily involved in their children’s previous schools. We believe that phone calls to parents help teachers generate improved decorum, effort, and ultimately learning from students. (Sometimes the calls to parents are supplemented with teacher calls to students) These parent relationships seem to be linked to very high parent-satisfaction ratings, and in turn we have thought those were related to our high test-score growth. Truth be told, however, we just don’t know whether this is a productive use of teachers’ time.

Fryer enlisted two doctoral students, Shaun Dougherty and Matt Kraft, from the Quantitative Policy Analysis in Education program at the Harvard Graduate School of Education. These two did an amazing job, operating skillfully within our school to do the randomized study. From their findings:

“On average, teacher-family communication increased homework completion rates by 6 percentage points and decreased instances in which teachers had to redirect students’ attention to the task at hand by 32%.”

This collaboration worked for several reasons. First, we have a teacher residency embedded in our charter school, so I had 24 student teachers who could be fairly easily randomized during the summer school session. Second, a professor I trusted chose the graduate students who would conduct the research. These guys were, in my view, dispassionate. I’ve tried to work before with grad students who have strong preexisting beliefs about what they’ll find (typically with a “progressive” lens), and it was difficult to gain real knowledge. (Researchers often feel the same way about practitioners, that we’re searching for marketing, not truth). Also, Fryer paid them a stipend; in my experience, graduate students working for free, and only for credit of some sort, don’t always follow through.

The cost of the two graduate students was not the only expense. In our experiment, at any given time, there were 16 classrooms in action. The researchers needed to hire 16 observers to carefully code student behavior for a few weeks. The total bill was around $10,000. Kraft and Dougherty found a Harvard grant of $1,000. The rest I needed to pay.

Once we’d designed the experiment, I needed to explain it to my team: the principals of our high school and middle school, and the student teachers who were involved. These are people I know well, and they generally trust me. Still, this buy-in phase required expending both time and “relationship capital,” a resource that gets spent down and must be built back up over time. Using student teachers was also of benefit. It would have been tough to randomize our regular teachers. Their belief in the efficacy of parent communication is so strong I suspect many would have doubted the value of changing their normal routines.

There were other costs to the experiment. The head of our teacher-prep program spent many hours handling the experiment’s complex logistics, including a permission slip for parent consent. He could have spent those hours coaching these student teachers, which is the main task I was paying him to do.

All of these issues reflect transaction costs: finding the right people and then doing the right study well takes time, effort, and money.

Researching Teacher Moves

Think of the Human Genome Project. When the project started, scientists didn’t know how many genes there were; now they believe the number is 20,000 to 25,000.

We don’t know how many teacher moves there are. The number is certainly high but not infinite, maybe 200, 2,000, nobody knows. Presumably, there are some unusually high-yield teacher moves across all contexts, some moves that are high yield but only in specific situations or contexts, and other less powerful moves. There is undoubtedly lots of interaction effect among many moves. Mapping all of this might be called the Teaching Move Genome Project, and at the beginning it would be a scary undertaking.

Absent this work, what do we have? Perceived best practices, often buttressed by observation or nonrandomized studies. In his best-selling book Teach Like a Champion, Doug Lemov describes 49 teaching moves he has observed in the nation’s top charter schools. At the University of Michigan, Deborah Ball and her colleagues are close to unveiling a list of 88 math teacher moves. Lee Canter’s Assertive Discipline and Jon Saphier’s Skillful Teacher discuss scores of moves, like the “10-2” rule (have kids summarize for 2 minutes in small groups after 10 minutes of teacher-led instruction), much of it supported by nonrandomized research. On the basis of its observations of effective teachers, Teach For America (TFA) promotes 6 teacher behaviors and 28 component parts, like “plan purposefully” or “set big goals”; none are specific moves.

What would a series of randomized trials look like? Let’s apply it to Lemov’s 49. Imagine a group of trials that would ask the questions, Do all of the moves work? Are any particularly successful? How does the degree of teacher buy-in interact with effectiveness? What are the “costs” of these moves?

An example from Lemov is “Right Is Right.” The idea is that when a kid gives an answer that is mostly right, the teacher should hold out until it’s 100 percent correct. Lemov describes various tactics the teacher can use to elicit the 100 percent right answer from the student (or first from another student, before having the original student repeat or extend the correct answer).

The obvious cost of implementing this move is time. These back-and-forths add up to lost minutes each period when other topics are not being discussed. A less skillful teacher might be drawn into a protracted discussion, when her next best alternative (simply announce the 100 percent right answer, and move on) might work better. We just don’t know.

Back in 2003, education researchers David Cohen, Stephen Raudenbush, and Deborah Ball argued that “one could make accurate causal inference about instructional effects only by reconceiving and then redesigning instruction as a regime, or system, and comparing it with different systems.” That suggests “a narrower role for survey research than has recently been the case in education, and a larger role for experimental and quasi-experimental research. But if such studies offer a better grip on causality, they are more difficult to design, instrument, and carry out, and more costly.”

Still, we need a better grip on causality. So who would undertake this cost?

A Proposal

Once again borrowing some terminology from medicine, I propose a typology of trials, delineating phases in a continuum.

Phase 1 trials would be small, nongeneralizable empirical studies of teacher moves. These could be randomized, single-subject, or regression discontinuity, but the dependent variable would not be year-end test scores. Instead, we’d look for next-day or next-week outcomes: measurable effects on student behavior, effort, or short-term learning.

Who would decide what moves to test? Some would be proposed by established authors and thinkers in the teaching field. Some would come from the nation’s 3 million schoolteachers, possibly with crowd sourcing to identify the most-promising ideas. Some would come from academic researchers, particularly those from other fields, like psychology, who may offer unusual insights. But for the next level, testing competing ideas, I’d suggest we draw heavily on teacher opinion, particularly a group of teachers selected for their stated willingness to try new methods (if they are supported by research).

Phase 2 trials would test promising teacher practice from Phase 1 on a larger, more varied teacher pool to see if the next-day outcomes held up, probably across different types of schools. Again, the dependent variable is short-term student response.

Phase 3 trials would be randomized trials in which teachers combine multiple moves that emerge from Phase 2. In the end, our bottom line is student learning, and Phase 3 trials are combinations of moves that are measured to see if they bolster year-end student learning gains.

Medical researchers have found that treating some illnesses requires a drug “cocktail,” that is, no one medicine by itself works as well as the combination of several. The same approach might work in education: it could be that individual teacher moves by themselves cannot create measurable year-end achievement gains in students, but combining many together can.

My proposal is that each of the nation’s 1,200-plus schools of education and teacher prep programs conduct one randomized trial on a teacher move each year: Phase 1, Phase 2, or Phase 3. They’d do that by recruiting alumni into a network of experienced teachers willing to participate. The advantage is that once you pay the one-time transaction costs of finding these teachers, the ongoing expenditures related to persuading them to participate, and securing permission from families and principals, decline.

Once that network existed, it would function like a laboratory. Various Phase 1 experiments could be run through it, with small numbers of teachers at first, so that many experiments could be run concurrently. Larger numbers of teachers would be included in more promising Phase 2 validation experiments. Of course, there would be selection bias in terms of which teachers are willing to be participate in this sort of work, and other imperfections. But in the end, experiments could build on proven results from previous ones. Multiple ed schools would combine their networks for Phase 3 trials.

By itself, no single experiment would be that important. Instead, it would be like cancer research: thousands of people each trying to answer small questions in a very rigorous way…which would add up to promising treatments.

The goal is an affordable system for conducting teacher research that teachers would actually consume, that would address both the implementation challenges and the high transaction costs for researchers and practitioners in creating such research. Until that exists, I’ll see you at the 5th-grade dance.

Michael Goldstein is the founder of MATCH Charter School and MATCH Teacher Residency, in Boston.

But the state shouldn’t be shocked by the report. As the truism goes, you get what you pay for.

Colorado’s policy environment incentivizes exactly what it’s getting from its full-time virtual schools—and arguably not just its virtual schools, but all of its schools statewide.

The biggest problem is this: It pays a school all of its funds on a “count day” on October 1 based on the number of students enrolled on that day. If students leave afterward, the original school keeps the funds. If students enroll elsewhere, the new school receives no funds.

This incentivizes providers to enroll students, but there are few incentives in place to focus on what happens after that. As a result, a significant number of online providers seem to have followed these incentives and done exactly what Colorado paid them to do. The end result isn’t pretty for students, as a great number of them allegedly leave soon after the count day and enroll back in district schools if they enroll elsewhere at all.

Some are using this to bash all online learning, as well as for-profit providers that are seizing this revenue-making opportunity (as many such providers did in higher education), but in so doing, these critics are missing the point.

As I’ve written numerous times, studying whether online learning is more or less effective than traditional learning is invariably asking the wrong question. Online and blended learning have the potential to dramatically transform our education system by being able to individualize for each student’s distinct learning needs (just look at the results from Carpe Diem, KIPP Empower, or Rocketship Education), but whether it does so will have a lot to do with policy—whether we change the incentives and focus not on merely serving students and micro-managing the inputs, but instead focusing on the student outcomes and leaving behind an antiquated factory-model system for a student-centric one. Ultimately we want a system that can deliver the right learning experience for each individual student when he or she needs it—whether that be an online or offline activity.

And just because many studies show that on average online and blended learning work better than does face-to-face learning, this does not mean that just because a program is online that it will be good. There will be both good and bad online programs, just as there will be good and bad face-to-face ones. Good programs, however, that do customize for these different learning needs and lead to increased student engagement and time on task, should be easier to scale in a digital world as opposed to an analog one.

Similarly, an oft-leveled charge at for-profits in education is that they only care about their shareholders, not about their customers. This is absurd. The way companies create shareholder value is by serving their customers. The problem here is that what the customer—the state of Colorado—is incentivizing is blatantly misaligned with what its students need.

As I wrote in a paper for the American Enterprise Institute titled “Beyond Good and Evil: Understanding the Role of For-Profits in Education Through the Theories of Disruptive Innovation,” for-profit companies are not inherently good or evil. Rather, successful companies do what their customers offer incentives to do—not much more or less. To say that for-profits are evil or poor quality misses the point because quality is defined by what a customer will pay someone to do. Blaming for-profits for doing what we have asked and paid them to do from the outset makes little sense.

What’s interesting in this particular case, however, is that a successful for-profit, K12, Inc., does apparently defy its incentives to some extent. According to Jeff Kwitowski, K12’s VP of Public Affairs, “K12 invoices the school for student-related expenses based on the number of students who are enrolled each month, not based on the October 1st enrollment count” despite the policy in place.

That’s good and smart of K12 to observe the spirit of the law, not just the letter. But policymakers must do better and create a system that does not rely on heroes and anomalies.

Given that, as I also wrote in the AEI piece, for-profits scale faster on average than do non-profits, they tend to be aggressive in seizing these policy opportunities, so policymakers need to fix bad policies quickly before an industry coalesces around a faulty value proposition and stands in the way of changing those policies with lots of money to back it up (as has happened with the players—mostly non-profit and government-run—that make up the country’s current factory-model education system).

There is a second problem with Colorado’s policy environment as well, which creates problems for truly judging how online learning programs are performing and could create incentives to avoid serving the hardest-to-serve and most vulnerable students. This one lies in the way our education system—across the nation—calculates graduation rates.

Although I’m still working out my thoughts on this, here’s the dilemma. The common graduation rate formula that has just recently been adopted across the country calculates a state’s graduation rate based, in essence, on the percentage of an entering freshman class that finishes high school with a regular diploma four years later. This may make sense as a way to judge a state (although there may be legitimate questions if it makes sense when moving to a competency-based learning system and away from one based on time), but it creates some problems for judging schools to where students are transferring in the midst of their high school experience.

The reason is this: Picture a school—like many of the online schools in Colorado—that enrolls a student in the fall who is classified as, say, a junior, based on his age. The graduation rate calculation says he should graduate in two springs from now; if he doesn’t, the school’s graduation rate falls. But say that student is many credits behind—let’s pretend for simplicity’s sake a year behind—and not on track to graduate “on time.” If the school manages to accelerate the student and the student graduates only a few months late—the summer after and not an entire year behind—shouldn’t that school get credit for accelerating the student? With today’s measurement systems, it is penalized.

The reverse scenario also exists today, as we give credit to schools that may have merely enrolled advanced students.

This doesn’t make sense. When a student transfers schools, he ought to be recognized based on the credits he brings and where he truly is academically, not based on his age—and the success of the school calculated accordingly.

This points to a desperate need to move toward a competency-based learning system that measures and rewards individual student growth, as well as an underlying shared learning infrastructure that allows the country to identify each unique student in a consistent way—so that when he or she moves geographies, the student’s record does as well—and to keep track of what that student knows and can do in a consistent way across geographies. Even moving past the question of calculating accurate graduation rates, unless this occurs, it remains challenging to figure out whether a school is helping a child academically with just a snapshot view.

Until we fix these problems, we shouldn’t be surprised at stories like the one unfolding in Colorado.

The biggest shame in all of this? By focusing on the wrong part of the story, it may set back our opportunity to leverage the rise of digital learning to transform our system into the student-centric one that each student deserves.

On student/teacher engagement:

The part that is often missed when discussing these concepts is that it’s a strategy for learning that humanizes the classroom. Building and growing teacher-student relationships is essential to improving student learning outcomes. When a teacher has the opportunity to speak to each student and assess their progress every day, students feel that learning matters. They feel challenged and supported. Again, it’s not about the technology or the devices, it about shifting our pedagogy to put each student first.

I use this tool for all the reasons stated and more, particularly the opportunity to spend more class time addressing the higher order thinking skills. Could it be that this is the point that critics are missing? The term ‘ flipped classroom’ places too much emphasis on a tool used by students to prepare for class and clouds the fact that teachers are developing fuller, richer learning cycles with their new time. Let’s call it the ‘flip-tool’ and start to write more about the consequences that is the rich learning cycles we have been able to develop for our classrooms.

On technology and ensuring equal access:

I have vids on flash drives and DVDs for kids w/ no internet access/digital tools.

[Teachers] don’t need the internet to do this! They can create videos, and save them on students’ machines quickly and easily. That way students just watch them from the computer without having to worry about connecting to the internet. We’ve also been able to repurpose old laptops for just this use. Since all the computers need to do is play a few videos, old laptops are perfect for this task.

On managing time and motivating students to do at-home activities:

The flipped class allows for learners to customize when and where they learn. I have plenty that spend class time learning and practicing, and at home, they don’t worry about chemistry. There is no such thing as “homework” anymore. If they work, they can use class time to front-load and account for their work life. We went through how to budget time and use the resources so they don’t overwhelm themselves through the year…. Most of what my kids (and many flipped kids) do is use the videos as A) remediation if they need it, or B) pre-learning for use in class the next day. The videos I put out are less than 10-minutes in length, so the time at home is considerably LESS than a “normal” homework assignment. Plus, they aren’t sitting at home struggling with a worksheet or book assignment, so their mental stress is also alleviated to a degree with a flip.

one of the most surprising things I learned when a colleague and I went to using videos to deliver most of the “content” of our class was that when forced to boil down the content to the most important concepts in order to create the videos, we ended up with a total of 8 videos of about 10-15 minutes each for our 10 week course in microbiology. In the past, we wold have spent FAR more time delivering the same content in class. Now, class time is spent exploring the content in context, the students are in the lab more often and the class time is a far more collaborative endeavor for the students. We have been able to do more higher-order thinking projects with the “found” time. Also, the students really like being able to control the pace of the delivery of the content in the videos. We provide them with sheets to take notes on while watching the videos so it is not simply a passive activity. Flipping has definitely resulted in more engaging and enjoyable class time for the students and the teachers.

Finally, as chemistry teacher Aaron Sams explains, it’s important to emphasize that there’s no single model, with most teachers figuring this out, adapting, and improving their practice as they go.

It is becoming increasingly obvious that digital learning systems can be tailored to the specific interests, learning styles, and levels of accomplishment of each student.  As digital curricular materials employ ever-more-sophisticated technologies—3-dimensional videos, game playing, interactive exercises, real-time provision of information on student performance to teachers and students alike, and more—they will be seen as essential 21^st century learning tools.

But we can expect a strenuous, highly politicized debate over the way in which digital learning should be provided.  On the one side will be those who propose that most digital learning in K-12 public education be of the “blended” variety, that is, take place within public school classrooms under the tutelage of a highly qualified teacher.

On the other side, “online” proponents will argue that blended learning alone is not enough.  American education can be transformed only if the power to drive change is placed in the hands of students, who are offered a choice of providers that include not only the blended classroom but also those who offer products  exclusively online, supplementing asymmetric video presentations of online materials with interactive systems that employ such tools as Skype, interactive games, social networking, email communications and phone conversations.

All of this became clear at the conference sponsored in San Francisco last week by the Foundation on Excellence in Education, the nonprofit headed by former Florida Governor Jeb Bush, who is promoting a strikingly innovative, bipartisan reform agenda that combines the Common Core standards promoted by the Gates Foundation and the Obama Administration with the accountability and choice principles to which he was committed during his eight years as Florida’s governor.

It is digital learning that holds together and gives spark to Bush’s agenda.  Common standards provide a nationwide platform upon which next generation curricular materials can be built; choice allows students to pick the courses most suited to their needs, abilities, and interests; and accountability ensures that learning is genuine.

Bush put on an impressive show.  His self-deprecating wit, extraordinary command of the subject, and undeniable passion generated a level of enthusiasm seldom found outside the confines of a well-orchestrated campaign event. When the former governor interviewed Melinda Gates about her support for Common Core standards, she relaxed noticeably, revealing a personal warmth and depth of knowledge less well displayed in her formal presentation.

But the true star of the show was Sal Khan, a former venture capitalist turned curriculum specialist, who has become a rock star of digital education. Unlike some other proponents of digital learning promoting their wares at the conference, Kahn taught his audience by both precept and example.  Not only did he advocate next-generation learning, but, in so doing, he blended a sweater-casual speaking style with a smoothly offered, high-tech digital presentation that was little less than astounding.  When he finished, only the most hard-nosed of skeptics walked away unconvinced that Khan had invented the one-and-only way to teach math to young people.

For Khan, next-generation learning combines simple, short, witty videos with problem sets that must be mastered before one moves to the next stage of instruction.  To motivate students, he uses, surprisingly, nothing more than badges and other phony rewards reminiscent of the stars that old-fashioned elementary school teachers used to post next to the names of high achievers.  Real-time data on success and failure is provided simultaneously to teachers, students, parents and anyone else authorized to access that information. You can learn all about the Khan method by looking at his videos on YouTube.

Yet Khan leaves the debate over blended versus online learning wide open.  On one side, the power of online learning is demonstrated by videos that are being viewed by Khan’s distant cousins as well as by the next generation of the Melinda and Bill Gates family, a saxophone player who is self-educating himself into an electrical engineer, and millions of young people in developing countries across the globe.

But the “blenders” will undoubtedly point to certain in-classroom keys to his accomplishments in the public schools of Los Altos, California.  There, student success at problem-solving is monitored in real time by teachers, serving as coaches, who intervene when videos are not enough. For blenders, the keys to the intervention’s apparent success include the use of real-time performance information by qualified teachers, not just the videos and problem sets.

Apparent success, it must be said, because the impact of neither the blended nor the online version of the Khan intervention has yet to be documented by a randomized trial.  Still, Los Altos school authorities are impressed enough to allow Khan Academy to expand from just a couple of demonstration classrooms to middle schools throughout the district.  And other charter and district schools are climbing on board this fast-moving train.

But the debate between blended and online learning will continue.  Too much politically is at stake for it to be otherwise.  It is not yet clear that blended learning a la Khan Academy will be any more efficient than the current bloated system of public education.  At a time of extreme fiscal exigency, legislators will look for ways in which technology can save money, not for new ways to add costs.

Meanwhile, school districts and teacher unions can be expected to fight publicly funded online learning that offers students a choice of taking courses outside their local district school.  If online learning should prove to be more effective than the learning that takes place within classrooms, it would provide a serious challenge to the school district-teacher union duopoly that blended learning does not.

-Paul E. Peterson

Is there a bolder alternative that might actually induce our nation to achieve widespread learning excellence?

Here’s a simple idea: put excellent teachers, the top 20 to 25 percent who achieve well over today’s “year of learning progress,” in charge of every child’s learning—consistently. Even with solid teachers who achieve a full year of progress, students who enter school behind stay behind, and those in the middle do not leap ahead. Moreover, the current teacher pool feeds the anemic principal pipeline, meaning excellent teachers are often pulled from instruction—or forced to work under inadequate leaders.

In our recent report, Seizing Opportunity at the Top, we suggest three major ways to generate the significant will needed to put excellent teachers in charge of every child’s learning. We must, at the federal or state level:

• Limit who can teach to top high school graduates, with further screening for behavioral competencies of excellent teachers;
• Offer large financial incentives from regular funding streams, for districts, schools, and teachers when they produce high-growth learning, up to and far beyond standards; and/or
• Create a new civil right to excellent teachers, one that parents and students can enforce legally when a child is behind standards, not making a full year of progress annually, or has not had an excellent teacher in a subject for two years running.

The only way to implement any of these reforms successfully, within budget and at scale, is to help excellent teachers increase their productivity: swap portions of excellent teachers’ time with digital instruction so they can teach more classes with similar or even smaller group sizes; let them delegate nonessential tasks to other adults; use digital tools to save time on instructional monitoring and planning; put them in charge of other teachers; and let the willing have more students to nurture under their strong wings. Find more discussion of these options in Opportunity at the Top and 3X for All. This is not new: Other excellent professionals, whose jobs and pay aren’t frozen into molds, started making these changes for themselves a half-century ago, developing differentiated teams and using technology to save time and eliminate error from routine work, leaving the best free to do the most complex tasks.

Public Impact, with help from teachers and others, will soon begin releasing designs that clarify how to make these changes in schools, within budget, and pay excellent teachers more for the additional children they reach. “How to” models will help, but without major policy changes to induce the will, all evidence is that schools simply won’t budge—not even the ones that already can (e.g., charter schools).

ESEA could help. At a minimum, it could:

• Require states to identify excellent teachers immediately (even if full-blown evaluation systems take longer to develop);
• Require reporting of the percentage of students reached by teachers at each effectiveness level, not just the percentage of teachers at different effectiveness levels—rewarding places that put excellent teachers in charge of more students, directly or through managed teams; and
• Make federal funding contingent on clearing barriers that keep excellent teachers from reaching more students, such as limits on their pay, class sizes, and non-teaching staff who could monitor digital instruction.

Seizing Opportunity at the Top lists the basic policies that states must change. Absent will-inducing provisions, though, even ESEA and basic state policy changes combined will be inadequate.

Policy and political leaders at all levels: We must stop haggling over how to pack our saddlebags while other nations board helicopters. Our nation needs us to step up. We need major action to induce the will to put excellent teachers in charge of every child’s learning.

- Emily Ayscue Hassel and Bryan C. Hassel, Public Impact

In the tiny Florence School District outside Pueblo, Johnson was one of 39 students who left Florence High School last year to sign up for online classes with GOAL Academy, one of the largest online schools in Colorado…

Johnson said she signed up for GOAL in July after her former science teacher promised free college classes. But she was back at Florence High School by January with no credits earned.

“I feel like I wasted an entire semester of my life,” said Johnson, now working overtime to boost her grades in hopes the gap in her transcript will be less noticeable to colleges.

So, a trusted former teacher, perhaps at a free BBQ in a local park, told Johnson about a wonderful new way to learn. Good for GOAL, which got a year’s worth of funding. And bummer for Florence High, which lost Johnson’s state dollars.

But the real risk — and real consequences — were borne by Johnson. It may be true that Johnson made a poor decision when she decided to enroll in GOAL in the first place. But, a system that offers little guidance and no safety nets for ill-informed high school students making big educational decisions is almost certain to produce many more stories of seventeen year-olds wasting a semester of school at the worst possible time. If we are going to offer students new options — and we should — policymakers must first do whatever they can to mitigate the risks borne by students.

Wrestling with the Mobility Issue

Data from both Ohio and Colorado show exceptional levels of mobility among full-time online students. In Ohio, state data show that about a third of students were enrolled for less than a year. Ed News Colorado found that of 10,500 students in the largest online programs in fall 2008, more than half – or 5,600 – left their virtual schools by the fall of 2009.

Representatives of online schools claim that this mobility is not necessarily a bug, but a feature. And, they’re not entirely wrong:

Reasons for the turnover include working with an at-risk student population that sees online learning as their last resort, students who use online as a brief experimentation with a new learning process, and parents not being able to stay home to oversee their children’s studies, said Heather O’Mara, executive director of Hope Online, one of the state’s largest online programs.

“We are all so different, we are serving different audiences and students are enrolling for very different reasons,” O’Mara said. “At Hope, we particularly target kids who are at risk, who have not been academically successful, not only at their previous school, probably several schools before that.”

But it’s also likely that high mobility is a sign of dissatisfaction or misaligned expectations about what online learning would really entail. And, since we don’t accept “we serve difficult students” as a blanket excuse when evaluating traditional school districts, we shouldn’t accept it for online schools either.

Most importantly, we need to align the incentives so that schools are compelled to share the risk with students — even if it slows growth. Here, Branson Online High offers a hopeful example, as its recent focus on ensuring families understood the online program before enrolling appears to be leading to more successful outcomes.

These are sticky, complex issues. The National Association of Charter School Authorizers, in its recent brief on virtual charter schools, acknowledges that high mobility impacts instruction and makes it challenging to evaluate virtual school performance, but offers no prescriptions.

I’ll offer up my ideas over the next few days. But it’s time to stop ducking this issue. I hope providers, advocates, and accountability hawks will respond with ideas of their own.

-Bill Tucker

It’s called “the flipped classroom.” While there is no one model, the core idea is to flip the common instructional approach: With teacher-created videos and interactive lessons, instruction that used to occur in class is now accessed at home, in advance of class. Class becomes the place to work through problems, advance concepts, and engage in collaborative learning. Most importantly, all aspects of instruction can be rethought to best maximize the scarcest learning resource—time.

Flipped classroom teachers almost universally agree that it’s not the instructional videos on their own, but how they are integrated into an overall approach, that makes the difference. In his classes, Bergmann says, students can’t just “watch the video and be done with it.” He checks their notes and requires each student to come to class with a question. And, while he says it takes a little while for students to get used to the system, as the year progresses he sees them asking better questions and thinking more deeply about the content. After flipping his classroom, Bergmann says he can more easily query individual students, probe for misconceptions around scientific concepts, and clear up incorrect notions.

Counterintuitively, Bergmann says the most important benefits of the video lessons are profoundly human: “I now have time to work individually with students. I talk to every student in every classroom every day.” Traditional classroom interactions are also flipped. Typically, the most outgoing and engaged students ask questions, while struggling students may act out. Bergmann notes that he now spends more time with struggling students, who no longer give up on homework, but work through challenging problems in class. Advanced students have more freedom to learn independently. And, while high-school students still occasionally lapse on homework assignments, Bergmann credits the new arrangement with fostering better relationships, greater student engagement, and higher levels of motivation.

Once Bergmann’s and Sams’s lessons were posted online, it wasn’t long before other students and teachers across the country were using the lessons, and making their own. Across the country in Washington, D.C., Andrea Smith, a 6th-grade math teacher at E. L. Haynes, a high-performing public charter school, shares Bergmann’s enthusiasm, but focuses on a different aspect of the flipped classroom. Smith, who has taught for more than a decade in both D.C.’s public charter and traditional district schools, immediately saw the benefit for students, but says she was most captivated by the opportunity to elevate teaching practice and the profession as a whole. As Smith explains, crafting a great four- to six-minute video lesson poses a tremendous instructional challenge: how to explain a concept in a clear, concise, bite-sized chunk. Creating her own videos forces her to pay attention to the details and nuances of instruction—the pace, the examples used, the visual representation, and the development of aligned assessment practices. In a video lesson on dividing fractions, for example, Smith is careful not to just teach the procedure—multiply by the inverse—but also to represent the important underlying conceptual ideas. Like Bergmann, she makes it clear that the videos are just one component of instruction. She’s keen on the equivalent of a motion picture’s “director’s cut,” where a video creator might explain the reasoning behind the examples chosen and how she would extend those activities into class time.

“Flipping” is rapidly moving into the mainstream. Bergmann and Sams have completed a book, are in high demand across the country at educator conferences, and even host their own “Flipped Class Conference” to train teachers. The chief academic officer at Smith’s school, Eric Westendorf, is taking the tools he has piloted at the school and building them into a platform for teachers everywhere to create and share videos. Most notable, though, is the emergence of the Khan Academy, an online repository of thousands of instructional videos that has been touted by Bill Gates and featured prominently in the national media.

Given education’s long history of fascination with new instructional approaches that are later abandoned, there’s a real danger that flipping, a seemingly simple idea that is profound in practice, may be reduced into the latest educational fad. And, in today’s highly polarized political environment, it also runs the risk of being falsely pigeonholed into one of education’s many false dichotomies, such as the age-old pedagogical debate between content knowledge and skills acquisition.

But the ideas behind flipping are not brand new. For over a decade, led by the National Center for Academic Transformation (NCAT), dozens of colleges have successfully experimented with similar ideas across math, science, English, and many other disciplines. NCAT’s increasingly impressive body of practice shows that thoughtful course redesigns lead to improved learning. Carol Twigg, NCAT’s president and CEO, says there is no magic: course redesign is “a hard job.” She’s not assuming students love homework. But redesign offers an opportunity to reengage students and improve their motivation, while setting proper expectations and monitoring to “push school to the top of the list.” And while many course redesigns focus on incorporating more project-based learning opportunities, Twigg’s experience leads her to quickly dismiss pedagogical extremes: “If you don’t have basic math skills, you can’t do an interesting physics project.”

There is also some danger that the flipped classroom could be seen as another front in a false battle between teachers and technology. Yet Bergmann and Sams emphasize that the “only magic bullet is the recruiting, training, and supporting of quality teachers.” And while Khan Academy’s prominence engenders fear of standardization and deprofessionalization among some critics, Bergmann, Sams, and Smith see instructional videos as powerful tools for teachers to create content, share resources, and improve practice. Smith admits that if such tools were available when she first started out, she “would have run to this every week when planning.”

It seems almost certain that instructional videos, interactive simulations, and yet-to-be-dreamed-up online tools will continue to multiply. But who will control these tools and whether they will fulfill their potential remains to be seen. As Scott McLeod, one of the nation’s leading thinkers on educational technology and the director of the UCEA Center for the Advanced Study of Technology Leadership in Education, observes, the “reason Sal Khan is so visible right now is that nobody did this instead. It would have been great if the National Council of Teachers of Mathematics had been doing this, but someone from the outside had to fill the vacuum.” His guidance to educators: “Start making!”

Bill Tucker is managing director of Education Sector.

My classmates complained that her class was too hard.

One of my other instructors spent class sessions badly summarizing the readings, instigating awkward and often one-sided class discussions, or trying to explain the homework assignments and projects she thought up. When she assigned one of her own articles for us to read, it became clear that despite having completed a doctorate at our university, she could not write a coherent academic article.

Desperate for a more challenging academic experience, I increased my course load for the second semester and handpicked my instructors. I actually enjoyed most of my classes that semester, but it was at this point that I began to deeply question the university’s approach to preparing future teachers.

It baffled me, for example, that I could get a master’s degree in teaching English to speakers of other languages (TESOL) after having completed only one rudimentary course in linguistics and one in English grammar. Almost all of my classmates struggled greatly in these two courses, leading me to wonder whether perhaps the admission requirements might also need refining. A class in adolescent development was useful, but the program offered no course in child development, despite the fact that my certification would be for grades K–12. It seemed that they were skimming over the important topics while bogging me down with courses in “theory and practice,” which did little to make me feel prepared to begin teaching on my own.

The focus of the third and fourth semesters was student teaching. My first placement was in high-school foreign language, for which I was also receiving certification. I was fortunate to work with a relatively strong supervising teacher; the infuriating aspect of this first placement was how I was evaluated. A supervisor from the university observed me during three lessons over the course of the semester. After each observation, she completed a write-up and made a few minimally helpful suggestions. During the final observation, she leaned over to my supervising teacher and casually asked, “So, what grade would you give her?” No criteria for evaluation, no request for a report on what I needed to work on. Fortunately, I did receive some valuable feedback from my supervising teacher that semester; I cannot say the same about my English as a Second Language student-teaching placement the following semester.

The final task I was asked to complete for the program was an “individualized project,” which sounded to me like a dumbed-down version of a thesis or capstone project. I have to confess that I took the easy way out. I knew I wasn’t going to get the kind of academic support I would need to complete an actual thesis, so I settled for designing a unit based on what I was already working on with my ESL students. After meeting with the professor a few times and receiving some vague suggestions, I handed in a project that earned me the last of a full transcript of easy As, with a friendly note on the cover and not a single comment or suggestion for how the unit could have been improved.

After observing and teaching in a variety of classroom settings over the course of my graduate studies, I have concluded that good teaching depends on three things: mastery of the subject, a keen understanding of how children learn, and an ability to maintain a disciplined yet positive learning environment. It is hard for me to express how disheartening it is to have spent two years and more than $80,000 in student loans on a program that did justice to none of those objectives.

The author earned a masters degree in education at a private university in the Northeast. Julia Harvey is a pseudonym.