A Comparison of Four Methods for Detecting Differential Item Functioning in Ordered Response Items

Abstract

Item bias is a major threat to measurement validity. Methods for detecting differential item functioning (DIF) are now commonly used to identify potentially biased items. DIF detection methods for dichotomous items are well developed, but those for ordinal items are less well developed. In this article, the authors compare four methods for detecting DIF in ordinal items: the Mantel, generalized Mantel-Haenszel (GMH), logistic discriminant function analysis (LDFA), and unconstrained cumulative logits ordinal logistic regression (UCLOLR). Factors varied include type of DIF, group ability differences, studied item discrimination, skewness in ability distributions, and sample size ratio. All procedures had good Type I error control as well as high power for detecting uniform DIF. However, the Mantel could not detect nonuniform DIF, and the LDFA also performed poorly in detecting nonuniform DIF, particularly when item discrimination was high. The UCLOLR and GMH performed extremely well under conditions simulated in this study. Implications for research and practice are discussed.