How to raise children’s learning outcomes, especially among those most disadvantaged, is not just the focus of education policymakers in the U.S. It’s a global problem and has attracted much attention. Education is listed fourth among the United Nations’ Sustainable Development Goals, following poverty, hunger, and health. Today, nine in ten children in low income countries cannot read nor do basic math by the end of their time in elementary school. Business as usual will not deliver the transforming change needed.
The World Bank Group, a provider of financial and technical assistance to developing countries, knows this. As I’ve written previously, the World Bank understands the important connection between measuring progress and getting results. In developing countries, data on education quality are rare. There are few countries with comparable learning outcome measures and even fewer countries with reliable measures of teaching practice. The World Bank, noting this gap and the necessary connection between better teaching and better learning, has launched a new tool called TEACH to help define and measure effective teaching practices.
TEACH is an open source classroom observation tool designed to be used in low- and middle-income countries. It comes with an in-depth explanation of how instructional practices are to be identified and how teachers are to be scored.
There is much to like about the World Bank’s new measure of teaching practices. TEACH is an accessible, carefully constructed, and well-researched instrument. Equally important, the team provides high quality implementation resources, including thoughtful cost estimates and tools to equip school systems to train reliable classroom observers. I strongly believe better feedback can lead to better teaching [1].
Designing a great instrument, however, is only part of the challenge. If the instrument is to achieve the team’s goals and bring about improvement, the World Bank needs to help school system leaders view teachers as products of their system, not as causes of its failure. While shaming is not the intent of the TEACH measure, the World Bank’s introduction, which highlights three teachers’ malpractice, risks misinterpretation. It won’t be easy for school system leaders who read these examples to interrogate their own practices and processes as potential root causes of weak teaching. The more natural response will be to blame teachers.
This response would be missing the point of TEACH entirely. The World Bank team clearly intends TEACH to be used for system diagnostics to help governments judge the effectiveness of their policies to improve teaching practices and as a professional development tool to identify a teacher’s strengths and weaknesses. My experience co-directing the Gates Foundation’s Measures of Effective Teaching project (MET) taught me three things I believe could help the World Bank strengthen its guidance to school systems, avoid likely pitfalls, and realize its vision for this potentially powerful tool.
First, fear trumps improvement. I can get anxious when someone watches me work, especially if this doesn’t happen regularly. Knowing my observer holds a tool which promises to expose weakness in my work increases my discomfort. This could escalate to panic if I might suffer due to poor results. Even worse, if I don’t trust the observer, I add error to my worries, fearing that I might suffer due to someone else’s mistake or malice. Finally, whose tool is this anyway? I wasn’t trained to do these things. I’m not even sure these practices help kids learn better. I’m a wreck.
This isn’t paranoia, it’s quite rational. Healthcare improvement expert Don Berwick tells us most organizations, upon receiving new capacity for measuring work, attempt to improve quality through inspection [2]. He calls this approach the “Theory of Bad Apples”, whereby the organization believes it will get better by finding and removing bad apples. Too often, this approach produces defensiveness rather than improvement. If school systems use the TEACH instrument to discover which teachers need to improve or be removed, they should not be surprised when teachers, afraid to lose their jobs or reputation, find clever ways to thwart the measurement system. This can take various forms, notably the infamous “dog and pony show” where the teacher, faced with inspection, immediately shifts instruction to a tried-and-true lesson, sometimes combined with added incentives for extra student participation and good behavior during the inspection.
The World Bank has an opportunity to help school systems avoid repeating this error. By urging the various ministries of education away from using this tool for teacher accountability, it will preserve TEACH’s ability to produce accurate information about teaching practices. Once the dog and pony shows begin, it will be hard to trust the data.
Second, teaching is a shared responsibility. The World Bank naturally adopts the language used within the research and reform communities: “The difference between the impact of a great and a weak teacher on student test scores is equivalent to one to two years of schooling.” This language, squarely and too heavily, places responsibility for quality upon the shoulders of teachers. Yet, a moment of reflection is all it takes to realize how much of what contributes to effective teaching is out of the control of teachers.
A teacher’s job is easier (meaning students learn better, faster) when students arrive having learned the prerequisites in prior classes. It’s easier when the school climate supports learning. It’s easier when the curriculum is clear and supports student learning. Add to this list: a good principal, proper supports for high needs students, enough time to accomplish the learning goals, and aligned state and/or national assessments. There’s more, to be sure, but it’s clear from this list that teaching practices are just one of several important contributors to student learning. Teachers will reject the measurement system as unfair if they are held solely accountable for what they know to be a shared responsibility.
It’s a mantra within the improvement science community that a system is perfectly designed to get the results it gets. Understanding teaching as a product of the system is a necessary first step toward improving teaching. The approach of fixing teachers, one class at a time, simply cannot succeed. Teachers and teaching are products of a system, not the causes of its failure.
Third, better teaching practices, as important as they are, are much more powerful when other facets of teaching also improve. Effective teaching requires all of practices within the TEACH framework and more: students must receive the right content and tasks at the right time in the right order. To do this well requires good curriculum, aligned assessments, thoughtful student placement, and support for teachers to use these tools well.
MET researchers found something extraordinary as they examined the impact of teaching practices on student learning; For the vast majority of teachers, observed differences in teaching practices did not explain differences in student learning [3]. Malpractice harmed learning and excellent practice helped learning, but researchers observed very few teachers in these categories. For most teachers, the 80% between malpractice and outstanding, other factors besides teaching practices more powerfully explain student performance differences. School systems can expect rapid student performance gains by eliminating malpractice and promoting outstanding practice. Student learning gains will be far less dramatic for most teachers, whose practices, despite improvement, still fall short of outstanding.
School systems should consider improving other facets of teaching beyond practices alone. Unquestionably, better teaching practices help students learn better. The benefits multiply, however, when better teaching practices are combined with the benefits of adopting and guiding teachers to use better curriculum.
A recent meta-analysis of direct instruction [4] estimated its impact on student learning to be roughly five times greater than the best estimates [5] [6] of student learning gains from teaching practice improvements.
Still, there is much benefit to gain if school system leaders use TEACH to clarify expectations for effective teaching, to develop and enlist teachers to observe and provide peer feedback, and entirely avoid using TEACH for individual consequences or rewards. There’s some hopeful research behind this recommendation. The school district in Cincinnati, Ohio introduced a common teaching framework, without consequences, and saw teaching practice improve for teachers at all levels of experience [7]. In Jackson County, Tennessee administrators used a common observation instrument to enable strong teachers to coach struggling teachers. Both teachers’ students’ performance improved [8].
In promoting TEACH, I hope the World Bank can help school systems avoid placing unrealistic expectations on this measure. TEACH is a potentially useful tool, but as the TEACH team knows, a teaching practices measure by itself cannot improve instruction and, unless each school system wields it wisely, it will create more problems than it solves.
Dr. Steve Cantrell is Vice President for Measurement and Evaluation at Bridge and a Senior Fellow at Georgetown University’s FutureEd.
Notes:
1. Jeff Archer, Steve Cantrell, Steven L. Holtzman, Jilliam N. Joe, Cynthia M. Tocci, and Jess Wood.
Better Feedback for Better Teaching: A Practical Guide to Improving Classroom Observations.
San Francisco: Jossey-Bass
2. Donald M. Berwick, 1989 – New England Journal of Medicine 320:1 pp 53-56
3. Erik A. Ruzek, Christopher A. Hafen, Bridget K. Hamre, Robert C. Pianta
“Combining Classroom Observations and Value Added for the Evaluation and Professional Development of Teachers” (Pages: 203-233) in Thomas J. Kane, Kerri A. Kerr, Robert C. Pianta eds. Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project. San Francisco: Jossey-Bass (2015)
4. The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research
Jean Stockard, Timothy W. Wood, Cristy Coughlin, Caitlin Rasplica Khoury
2018; Review of Educational Research, pp. 479–507
5. An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement
Joseph P. Allen, Robert C. Pianta, Anne Gregory, Amori Yee Mikami, Janetta Lun
Science 19 Aug 2011: Vol. 333, Issue 6045, pp. 1034-1037
6. Teacher Quality and Learning Outcomes in Kindergarten.
Caridad Araujo Pedro Carneiro Yyannú Cruz-Aguayo Norbert Schady.
The Quarterly Journal of Economics, Volume 131, Issue 3, 1 August 2016, Pages 1415–1453
7. Taylor, Eric S., and John H. Tyler. 2012. “The Effect of Evaluation on Teacher Performance.” American Economic Review, 102 (7): 3628-51.
8. https://www.edweek.org/ew/articles/2017/10/25/teachers-pair-up-to-improve-practice.html
