Anna Egalite, Jonathan Mills, and I have a new study in the journal Improving Schools in which we administer multiple measures of “non-cognitive” skills to the same sample of students to see if we get consistent results. We didn’t.
How students performed on a self-reported grit scale was uncorrelated with behavioral measures of character skills, like delayed gratification, time devoted to solving a challenging task, and item non-response. These are all meant to capture related (although not identical) concepts, so they should correlate with each other. The fact that they don’t suggests that we still have a lot of work to do to refine our understanding of character skills and how best to measure them.
Angela Duckworth, who developed the self-reported grit scale, and David Scott Yeager, who is a pioneer in measuring growth-mindset, have been trying to warn the field that these measures are still in their infancy. They have an article in Educational Researcher and have been giving interviews emphasizing that while non-cog skills appear to be a very important part of later life success, our methods of measuring these concepts are still not very strong — certainly not strong enough to include in school accountability systems.
Our research showing the lack of relationship between behavioral and self-reported measures of character skills adds to the case for caution in using these measures for evaluation or accountability purposes. Remember, it took decades of research and practice to develop reliable standardized tests. A similar effort and patience will be required to develop reliable measures of character skills. And I suspect that even improved measures may be useful for research purposes but never robust enough to use for accountability.
Ed reformers can be dangerous if they are too much in a hurry. We unfortunately want to apply every new insight right away and lack patience for the careful development of policies and practices for long-term benefit. We also invest few resources in basic research that is essential for long-term gains. According to my analysis in a chapter in a new book edited by Rick Hess and Jeff Henig on education philanthropy, the largest education foundations only devote 6% of their funding toward research. And most of that research may really be short-term policy advocacy masquerading as research. The federal government is little better at making funds available for basic research.
Non-cog or character skills are incredibly important but if we are going to use these and other ideas to improve education, we are going to need a significant shift toward funding research and greater patience to bring those ideas to fruition.
—Jay P. Greene
