Brookings, Baseball and Value Added Assessments of Teachers



By 11/22/2010

5 Comments | Print | NO PDF |

I have always been suspicious of consensus documents with multiple signatures.  They have the patina of authority but usually produce pabulum. So it was a pleasant surprise to read the latest consensus document from the Brookings Institution on “the important role of value added” when assessing teacher performance.

The merit pay movement depends on value-added testing to make its case that teacher salaries need to be based on performance, not experience.  But the opposition to value-added testing is potent, as teacher unions have been fighting virtually any kind of  performance-based evaluations of their members. Testing has also been attacked relentlessly in education schools across the country.  Just last Thursday, I attended a seminar at Boston College where David Kirp championed his forthcoming book, Kids First. Not a person in the room could find anything good to say about using tests to hold schools or teachers accountable.  The well-known Princeton economist Jesse Rothstein has done much to advance his career by purporting to show that value-added measures get it wrong.

Given this kind of organized opposition to testing, one would expect Brookings, the very definition of a middle-of-the-road think tank, to produce a consensus document that balances the arguments carefully, saying “yes, testing has some value,” but “no, there are a lot of problems with it.”  But, on the contrary, Grover Whitehurst, director of the Brown Center at Brookings, has put a group together that not only makes a clear case for value-added measurement of teacher performance but states that case simply and persuasively.

The group admits that test-based measures of teacher effectiveness correlate, on average, for any given teacher, from one year to the next, at no better than 0.35, well below the 0.90 correlation one would in principle like to have.  But the report then points out that the “between-season correlation in batting averages for professional baseball players is 0.36.”  In the best line of the report, they say:  “Ask any manager of a baseball team whether he considers a player’s batting average from the previous year in decisions about the present year.”   If general managers, in making salary decisions, look beyond years of experience, so should school districts. Thanks, Brookings.

-Paul E. Peterson




Comment on this article
  • stats geek says:

    You are mixing your statistics like you mix metaphors. A batting average is a percentage, so your example of batting average of 0.36–a very high average–means this player makes it on base approximately once per game. If teaching were baseball, then we really would have an education crisis. Only one-third of our students would contribute to society.

    A correlation examines the co-variance between two constructs, or the overlap between two hypothetical distributions. A correlation of .30 is considered small and poor predictive ability. The average correlation between the SAT and college freshman GPA is 0.50, a moderate correlation. Still, there are decades of argument that SAT scores have poor predictive validity, because bigger factors contribute (i.e., financial support and individual motivation). SAT scores then become one criterion of many when admissions committees examine applications. College applicants submit a portfolio for consideration.

    Hmmmmm, teacher portfolio…

  • [...] Yes, that’s right. Writing on the Education Next blog, Harvard professor Paul Peterson brought my attention to a great new consensus report from the Brookings Institution on the role of value-added in [...]

  • JD says:

    Roger Sweeny left a comment on Ed Policy Thoughts that bears quoting in full, as it is so witty:

    “I imagine a world … where it is illegal to keep statistics on baseball players–aside from whether they show up for work each day.

    “Instead, in order to play professional baseball, one must go to “baseball school.” There one takes courses in human physiology, the professional baseball rule book, theory of hitting, the history of baseball, etc.

    “Some schools require the aspiring player to play on a team for a period of several months, taking the place of a “master player” for parts of a game and listening to advice from the master player and a sponsor from the baseball school.

    “After graduation, the aspiring player can play for a real team but the decision on whether to keep the player has to be based on whether the newbie is “good in the clubhouse” and how well he hits at three carefully observed batting practices (“how well” to be determined partly by whether he does what the observer considers to be “best hitting practice”).

    “Meanwhile, some radicals have proposed that records should be kept on what the aspiring player does in real games. At the moment, the major proposal is that “at bats” and “hits” should be combined in a summary statistic called “batting average.”

    “To some people this seems like useful information but the people who work at baseball schools have made many arguments why it is far, far from perfect. Some say it is useless. Some say that allowing it to be calculated will actually make things worse. The statistic will be misinterpreted by decision-makers and players will care more about their individual statistics than about winning games.

    “A few baseball regulatory commissions are now allowing the use of “batting average” in addition to all the previous requirements.”

  • [...] looking at the report, I recommend you start with Paul Peterson’s post over at EducationNext. This entry was posted in Uncategorized and tagged Brookings [...]

  • [...] Okay, I have to confess. The main reason I wanted to share this passage is that today is Opening Day for many Major League Baseball teams. And any time someone can draw the connection between education reform and baseball, it makes me smile. And actually, the above argument builds off an interesting Paul Peterson piece about differential pay for educators and baseball players. [...]

  • Comment on this Article

    Name ()


    *

         5 Comments
    Sponsored Results
    Sponsors

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform

    Sponsors