The news is full of stories about incidents of cheating on various accountability tests. The Secretary of Education has urged all state commissioners to focus on testing integrity. In response, states are asking task forces to develop new security protocols, are hiring consultants to evaluate erasure patterns on test booklets, and are contemplating how they can change the pressures for cheating.
A more complete integration of testing, accountability, and teaching would be superior to dealing with the integrity of testing in isolation. Let’s put the tests out in the sun instead of trying to lock them up in more and more secure rooms.
The use of student outcome measures for accountability is now firmly entrenched and is not about to go away. But a variety of complaints about the current testing system exist. First, the tests tend to narrow the teaching to just what is expected on the tests – with excessive teaching to the test and drilling on practice tests. Second, the tests are too easy for students in some schools and too hard for others, either wasting the time of some or frustrating others. Third, using tests for accountability purposes encourages cheating and other ways to evade scrutiny.
To address these different issues, we need to think differently about aims and means. Here is a brief proposal to deal with all of the problems. It starts with developing a large item bank of test questions of varying difficulty. Imagine 1,500 questions for fourth grade math that cover the entire scope of appropriate material from basic to advanced topics. Next, make all of the test items – not just sample items – publicly available and encourage teachers to teach to the test, because the items cover the full range of the desired curriculum. Making the items public will also ensure the quality of the test items. One could invite feedback ratings or open sourcing to provide a path to improving the questions over time. Then, move to computerized adaptive testing, where answers to an initial set of questions move the student to easier or more difficult items based on responses. This testing permits accurate assessments at varying levels while lessening test burden from excessive questions that provide little information on individual student performance. Such assessments would not be limited to minimally proficient levels that are the focus of today’s tests, and thus they could provide useful information to districts that find current testing too easy. Students would be given a random selection of questions, and the answers would go directly into the computer – bypassing the erasure checks, the comparison of responses with other students, and the like.
This proposal actually follows the current testing by the FAA of knowledge needed to obtain a private pilot license. While there are commercial books on these tests, replete with questions and answers, the efficient way to prepare for the tests is simply to learn the underlying concepts. It is not to attempt to memorize the answers, because it is easy to confuse such an attempt.
What are the potential problems? Some say developing test items would make this too costly, but remember that it is only necessary to have one item bank, not the continually changed banks of today. Some worry about ensuring that sufficient computers are available in all schools, but with all of the digital devices currently in use, surely there are a range of possibilities to deliver the tests effectively and efficiently. There is the problem that the testing companies would not particularly like the proposal. They find they are happy with mindlessly developing slightly different variants of existing tests for different states, years, and administrations. But, maybe there are more productive ways for them to enter into the process.
The proposed system would yield quick and reliable feedback on student achievement, would deal with the various cheating and gaming issues, and would more effectively define what students should know than the currently available standards.
-Eric A. Hanushek
Eric Hanushek is a senior fellow at the Hoover Institution of Stanford University and member of the Koret Task Force on K-12 Education.
