My teaching career began in Washington, D.C., at the same time the first pay-for-performance plan was implemented in the city. I saw up close how the focus on student output changed the behavior of school leaders and staff, sometimes for the better, but not always. In 2017 and 2018, journalists and a follow-up audit report disclosed that a third of D.C. public school graduates should not have received a diploma according to eligibility criteria. But with graduation rates scrutinized as a key indicator of success for schools and the district, the proper protocol fell by the wayside.
This isn’t unique to education or to Washington, D.C. What’s required to remedy this situation is re-elevating the process of achieving academic results, and using outcomes not for external judgment, but for internal calibration. This can happen by asking districts, schools, and teachers to clarify the inputs that are most important to getting their desired results and evaluating them on the quality of these inputs.
There are at least three distinct reasons for measuring inputs in addition to outputs. The first is to prevent misguided strategies for raising the outputs. Think about a person hoping to lose weight. A person could attempt to lose weight and stay focused strictly on the output, the number on the scale. Perhaps the goal is to lose ten pounds. The most efficient way to do that would be to fast for a few days or sweat off a few pounds in back-to-back Soul Cycle classes. The goal might be achieved, but only temporarily — much like students who are great at cramming for tests, but who retain a negligible amount when retested a few months later. This cramming, which is standard operating procedure in schools, is incentivized by a system whose sole concern is on outputs.
A second reason to measure inputs along with outputs is to facilitate learning about which inputs work. When you measure inputs, you start with a hypothesis about how change happens. In the weight-loss analogy, a person might believe that he loses weight when he shifts to a diet low in fat and sugar and is active for an hour a day. His success would be measured by the quality of what he is putting in his body. His weight would be a source of information that might cause him to adjust his inputs, but the weight is not what he is judged on; it is the quality of the inputs that constitutes whether he is succeeding or not. In this example, and in the case of educators, scores become a metric for internal information rather than external validation. The end result still has a role to play — it’s just not for the funder, district office, or state to use as the main grounds for judgment. It’s for the organization or individual to learn from and adjust around in a cycle of continuous learning and improvement.
If a school has high-quality inputs, but poor outputs, it may mean it has selected the wrong inputs; and when the school realizes these inputs aren’t leading to the results they expected, it owns that outcome and can use it as the starting point of a conversation about what to focus on next, rather than as an endpoint.
By evaluating the quality of select inputs and using outputs to inform practice, schools become learning organizations once again. Chicago Mayor Rahm Emanuel published a piece in The Atlantic describing how he came into the office with a narrow focus on testing gains and increased graduation rates, but after a series of setbacks and conversations with educators, his focus shifted to figuring out what inputs make up a great school system. Reflecting on Chicago’s progress over the last few years, Emanuel realized it was a handful of these inputs that made the difference. He specifically notes creating a pipeline of great principals, allowing these principals autonomy, wraparound/out-of-school services and a dual-credit program for high schoolers. Once he realized the need to focus on these inputs, the outputs started to follow.
A third reason to measure inputs in addition to outputs is that some outputs are difficult to measure. Some social-emotional skills, for example, are not easily quantified. To be sure, it can sometimes be complicated to collect the data around inputs, too, but the magic of what happens in schools is not a simple matter.
What would a shift toward measuring inputs look like in practice? For a school, this would mean judging success not on test scores, but on the conditions that go into creating academic results. For example, a school may say, “judge us on the number of high-quality teachers in the building,” “judge us on how much collaborative planning time teachers have,” “judge us on the ratio of student-to-teacher talk.” Schools will still administer tests, and they will still concern themselves with the students’ test results, but as a means of informing their own practice, not as a way of labeling their worth.
The other benefit to being evaluated on the quality of your inputs is that there is plenty of research on which inputs are effective. That research doesn’t always get as much attention as it should. For example, the London-based Education Endowment Foundation website has a fantastic tool that ranks every classroom intervention and its relative costliness. Actions the website highlights as moderate- to high-impact include providing effective feedback, facilitating collaborative learning, and requiring demonstration of metacognition, or helping students be aware of how they learn.
Measuring school success based on the quality of inputs encourages diverse approaches that are responsive to the community each school serves, but also does not limit the collection of standardized assessment data.
Ideally, a focus on inputs would mean more time spent thinking, preparing, and developing what goes into making effective schools and less time testing and re-testing students with the false hope that more scores will lead to better scores. Some caution will be required. A shift from a singular focus on outputs to a singular focus on inputs without acknowledging the connection between the two, and the bridge that culture plays in effective implementation, has the potential to replicate the original problem.
Intentionality and leadership are still required, and ideally reinforced, by an input-oriented approach to evaluation. The power of measuring educators on the quality of what goes into their programs is in the culture that this process supports. Evaluating inputs requires schools and districts to have, and be able to explain, a theory of change. It helps individuals and schools own their own results and trusts them to use the results as feedback in an iterative process of improvement.
Ultimately, evaluating inputs over outputs encourages a culture of learning, which should be at the heart of every education organization.
Orly Friedman is the founder of Red Bridge Education – a new model of K-8 school, opening its first location in San Francisco in fall 2020. She taught elementary school in the Washington, D.C., public schools for five years. She most recently was the Head of Lower School at the Khan Lab School in Mountain View, Calif., and part of the founding team there. She has spent the last year as an entrepreneur-in-residence with Transcend working on a blueprint for a school designed to build a sense of agency in students.
