Examinee Noneffort and the Validity of Program Assessment Results
- 26 March 2010
- journal article
- research article
- Published by Taylor & Francis Ltd in Educational Assessment
- Vol. 15 (1), 27-41
- https://doi.org/10.1080/10627191003673216
Abstract
Educational program assessment studies often use data from low-stakes tests to provide evidence of program quality. The validity of scores from such tests, however, is potentially threatened by examinee noneffort. This study investigated the extent to which one type of noneffort—rapid-guessing behavior—distorted the results from three types of commonly used program assessment designs. It was found that, for each design, a modest amount of rapid guessing had a pronounced effect on the results. In addition, motivation filtering was found to be successful in mitigating the effects caused by rapid guessing. It is suggested that measurement practitioners routinely apply motivation filtering whenever the data from low-stakes tests are used to support program decisions.Keywords
This publication has 13 references indexed in Scilit:
- Test-Taking Motivation and Mathematics Performance in TIMSS 2003International Journal of Testing, 2007
- Setting the Response Time Threshold Parameter to Differentiate Solution Behavior From Rapid-Guessing BehaviorEducational and Psychological Measurement, 2007
- Changes in Rapid-Guessing Behavior Over a Series of AssessmentsEducational Assessment, 2007
- Taking the Time to Improve the Validity of Low‐Stakes Tests: The Effort‐Monitoring CBTEducational Measurement: Issues and Practice, 2006
- An Investigation of the Differential Effort Received by Items on a Low-Stakes Computer-Based TestApplied Measurement in Education, 2006
- An Application of Item Response Time: The Effort‐Moderated IRT ModelJournal of Educational Measurement, 2006
- Low Examinee Effort in Low-Stakes Assessment: Problems and Potential SolutionsEducational Assessment, 2005
- Construct‐Irrelevant Variance in High‐Stakes TestingEducational Measurement: Issues and Practice, 2004
- Effects of Motivational Interventions on the National Assessment of Educational Progress Mathematics PerformanceEducational Assessment, 1995
- Raising the Stakes of Test Administration: The Impact on Student Performance on the National Assessment of Educational ProgressEducational Assessment, 1995