Nonignorable Consequences of (Partially) Ignoring Missing Item Responses: Students Omit (Constructed Response) Items Due to a Lack of Knowledge
Open Access
- 30 April 2023
- Vol. 3 (2), 215-231
- https://doi.org/10.3390/knowledge3020015
Abstract
In recent literature, alternative models for handling missing item responses in large-scale assessments have been proposed. Based on simulations and arguments based on psychometric test theory, it is argued in this literature that missing item responses should never be scored as incorrect in scaling models but rather treated as ignorable or handled based on a model. The present article shows that these arguments have limited validity and illustrates the consequences in a country comparison using the PIRLS 2011 study. It is argued that students omit (constructed response) items because they do not know the correct item answer. A different treatment of missing item responses than scoring them as incorrect leads to significant changes in country rankings, which induces nonignorable consequences regarding the validity of the results. Additionally, two alternative item response models are proposed based on different assumptions for missing item responses. In the first pseudo-likelihood approach, missing item responses for a particular student are replaced by a score that ranges between zero and a model-implied probability computed based on the non-missing items. In the second approach, the probability of a missing item response is predicted by a latent response propensity variable and the item response itself. The models were applied to the PIRLS 2011 study, demonstrating that country comparisons change under different modeling assumptions for missing item responses.Keywords
This publication has 80 references indexed in Scilit:
- Sensitivity Analysis When Data Are Missing Not-at-randomEpidemiology, 2011
- Bayesian estimation of the multidimensional graded response model with nonignorable missing dataJournal of Statistical Computation and Simulation, 2010
- An NCME Instructional Module on Booklet Designs in Large‐Scale Assessments of Student Achievement: Theory and PracticeEducational Measurement: Issues and Practice, 2009
- Partial and latent ignorability in missing-data problemsBiometrika, 2009
- Missing Data Analysis: Making It Work in the Real WorldAnnual Review of Psychology, 2009
- Models of Nonresponse in Legislative PoliticsLegislative Studies Quarterly, 2008
- Multiple imputation inference for multivariate multilevel continuous data with ignorable non-responsePhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2008
- An Essay on Measurement and Factorial InvarianceMedical Care, 2006
- Modelling non‐ignorable missing‐data mechanisms with item response theory modelsBritish Journal of Mathematical and Statistical Psychology, 2005
- The Impact of Omitted Responses on the Accuracy of Ability Estimation in Item Response TheoryJournal of Educational Measurement, 2001