Subgroup Differences in Situational Judgment Test Performance: A Meta-Analysis
- 15 July 2008
- journal article
- review article
- Published by Taylor & Francis Ltd in Human Performance
- Vol. 21 (3), 291-309
- https://doi.org/10.1080/08959280802137820
Abstract
In this article, we describe a systematic review of mean race and sex differences in situational judgment test (SJT) performance. On average, White test takers perform better on SJTs than Black, Hispanic, and Asian test takers. Female examinees perform slightly better than male test takers on SJTs. We investigate two moderators of these differences: loading of g or personality on the SJT, and response instructions. Mean race differences between Black, Hispanic, Asian and White examinees in SJT performance are largely explained by the cognitive loading of the SJT such that the larger the cognitive load, the larger the mean race differences. Regarding the effect of personality loadings of SJTs on race differences, Black–White and Asian–White differences are smaller to the extent that the SJT is correlated with emotional stability and Hispanic–White differences are smaller to the extent that SJTs are correlated with conscientiousness and agreeableness. Cognitive loading has minimal effect on male–female SJT score differences; however, SJT score differences are larger, favoring women, when SJTs are correlated with conscientiousness and agreeableness. Concerning response instructions, knowledge response instructions showed greater race differences than behavioral tendency instructions. The mean correlations show that these differences are largely because of the greater g loading of knowledge instructions. A second study showed that when used in hiring, SJTs are likely to have adverse impact by race but not by sex.Keywords
This publication has 32 references indexed in Scilit:
- SITUATIONAL JUDGMENT TESTS, RESPONSE INSTRUCTIONS, AND VALIDITY: A META‐ANALYSISPersonnel Psychology, 2007
- Sex differences on the progressive matrices: A meta-analysisIntelligence, 2004
- Psychological Bulletin: Editorial.Psychological Bulletin, 2003
- The Development and Validation of a Situational Judgement Test for Security OfficersSecurity Journal, 2003
- Situational Judgment and Job PerformanceHuman Performance, 2002
- Gender differences in personality traits across cultures: Robust and surprising findings.Journal of Personality and Social Psychology, 2001
- Determinants, Detection and Amelioration of Adverse Impact in Personnel Selection Procedures: Issues, Evidence and Lessons LearnedInternational Journal of Selection and Assessment, 2001
- Use of situational judgment tests to predict job performance: A clarification of the literature.Journal of Applied Psychology, 2001
- A Nonparametric "Trim and Fill" Method of Accounting for Publication Bias in Meta-AnalysisJournal of the American Statistical Association, 2000
- Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions.Journal of Applied Psychology, 1997