The Best Foot Forward project at the Center for Education Policy Research at Harvard has been investigating the use of digital video to make classroom observations more helpful and fair to teachers and less burdensome for supervisors. In a randomized field trial involving 347 teachers and 108 administrators in Delaware, Georgia, Colorado and Los Angeles, teachers were given a special video camera and invited to collect multiple lessons. They could then choose a subset of their lesson videos to submit for their classroom observations. A secure software platform allowed administrators as well as external observers (selected for their expertise in a teacher’s discipline) to watch the videos and provide time-stamped comments aligned to specific moments in the videos.
In addition to giving teachers a reason and an opportunity to watch multiple instances of their own teaching, the videos served as the basis for one-on-one discussions between teachers and administrators and between teachers and the external content experts. The comparison teachers and schools continued to do in-person classroom observations. Although we’re awaiting data from a second year of implementation, we can report five preliminary findings so far:
• Despite teachers’ initial discomfort with collecting and watching video of their own instruction, the intervention did shift the mode of classroom observations, from in-person to video. Treatment teachers collected an average of 13 videos of their lessons. The average treatment teacher had 2.85 formal observations based on their submitted videos as well as two no-stakes observations by external observations. Treatment teachers also reported 1.06 fewer in-person classroom observations.
• Although their final official observation scores were no different than comparison teachers, treatment teachers perceived their supervisors to be more supportive and their observations to be fairer. They reported fewer disagreements over scoring and were more likely to be able to describe a specific change in their practice resulting from their post-observation conference. Treatment administrators reported that their post-observation conferences with teachers were less defensive.
• The opportunity to watch their own lessons seems to have made treatment teachers more self-critical. They rated their own instruction lower than comparison teachers, particularly in terms of time management and their ability to assess student mastery during class.
• We learned that the use of video did not save administrators’ time; in fact, treatment administrators reported spending more time on the observation process than the control group. However, the ability to watch video did allow supervisors to shift their observation duties to quieter times of the day or week: more than half of all log-ins occurred during lunch hour, the two hours immediately after school, evenings, weekends, and holidays.
• Although both treatment and control teachers volunteered to participate in the project, the treatment teachers were more likely to support use of video at the end of the first year, as a replacement for some or all of their in-person classroom observations.
In sum, the use of teacher-collected video in classroom observations did seem to improve the classroom observation process along a number of dimensions: it boosted teachers’ perception of fairness of classroom observations, reduced teacher defensiveness during post-observation conferences, led to greater self- perception of the need for behavior change and allowed administrators to time-shift observation duties to quieter times of the day or week. In coming months, we will provide evidence on whether or not these apparent improvements in the observation process were sufficient to generate improvements in student achievement.
For more on the project, please see: http://cepr.harvard.edu/best-foot-forward-project
– Thomas J. Kane and Miriam Greenberg
This first appeared on the Brown Center Chalkboard
