My student, Collin Hitt, and colleague, Julie Trivitt, have an amazing paper on how we can efficiently measure an important non-cognitive skill that is strongly predictive of later life outcomes. A growing number of researchers have come to realize that lifetime success is partially a function of traditional academic achievement (cognitive skills) and partially a function of what are called non-cognitive skills, such as hard work, self-discipline, determination, etc… Schools may play a central role in conveying both types of skills, but for the most part we have only been collecting information on cognitive skills in the form of standardized test results. The main difficulty in expanding the types of measures we collect to include non-cognitive skills is that we have not developed efficient mechanisms for doing so.
Hitt and Trivitt have taken an enormous step forward to solve this problem. They have discovered that student non-response on surveys (not answering questions or saying they don’t know) is an excellent measure of non-cognitive skills that are strongly predictive of later life outcomes. In particular they examined survey response rates from the National Longitudinal Study of Youth (NLSY) given to students ages 13 to 17 in 1997. The number of items that students answered was predictive of the highest level of education students attained by 2010, controlling for a host of factors including measures of their cognitive ability. If students care enough to answer questions on a survey they are more likely to care enough to pursue their education further.
They then examined another data set to see if they found the same relationship. They did. The number of items that students in Milwaukee answered in a survey when they were in 9th grade was predictive of whether they graduated high school and went to college later, controlling for their academic achievement and other factors.
If this holds up when examined with multiple data sets, it will be an amazing breakthrough for researchers. We will finally have a fairly easy to obtain measure of an important non-cognitive skill that is predictive of later life success.
When studying voucher or other school choice programs, for example, we have observed modest test score benefits for participants, but fairly large attainment benefits. This suggests that school choice has larger effects on non-cognitive skills, but up until now we haven’t been able to observe these non-cognitive benefits without waiting nearly a decade to see if students graduate high school and go on to college. With the Hitt and Trivitt measure, we will have an early warning indicator of whether students are acquiring non-cognitive skills and are more likely to have higher attainment later.
I am not suggesting that the Hitt and Trivitt measure can be used in an accountability system, since it is certain not to work once high stakes are attached. But for research purposes it could be incredibly useful.
Developing an accurate and efficient measure of non-cognitive skills is especially important because one commonly considered measure, the self-reported “grit scale” developed by Angela Duckworth, may not be holding up very well. In the recent Dobbie and Fryer evaluation of the Harlem Promise Academy, it actually appears that the Duckworth scale was a contrary indicator of later life success. That is, students who rated themselves higher on the grit scale were less likely to succeed. We have also tried the Duckworth scale in an experiment and found that it was uncorrelated with other, behavioral measures of non-cognitive skills, such as time devoted to a challenging task and delayed gratification. But the self-reported grit scale was related to a student self-assessment of honesty, suggesting that the Duckworth scale may really measure how highly students will rate themselves more than actual grit or other non-cognitive skills.
Of course, the Hitt and Trivitt measure requires a lot more testing and field research, but it is one of the more exciting recent developments in education research.
—Jay P. Greene
