A situational judgement test of professional behaviour: development and validation
- 1 January 2008
- journal article
- research article
- Published by Informa UK Limited in Medical Teacher
- Vol. 30 (5), 528-533
- https://doi.org/10.1080/01421590801952994
Abstract
Aims: To describe the development of a single best answer multiple choice situational judgement test (SJT) to assess medical students' ability to judge professional behaviour; to present results of an expert validation of the SJT; to describe experiences with two different validation formats. Methods: Based on educational objectives and work situations concerning professional behaviour, a SJT with 17 vignettes and 35 questions was developed. Best answers were developed according to available evidence. Forty-four experts validated the answers using either a rank order or a rating scale. Results: For sixteen questions in the rating group, thirteen questions in the ranking group and twelve questions in both groups more than two thirds of the experts agreed on a best answer. In the ranking group some experts found it difficult to commit themselves to a single best answer as instructed. In the rating group some experts did not mark any of the proposed answer options as adequate. Discussion: The definition of the construct of and experts for professional behaviour remain a challenge. Both validation scales have advantages and disadvantages. The two thirds criterion is somewhat arbitrary and lower or higher agreement may be acceptable or necessary depending on the purpose of the assessment.Keywords
This publication has 18 references indexed in Scilit:
- Situational judgment test research: Informing the debate on practical intelligence theoryIntelligence, 2005
- Achievement of non-cognitive goals of undergraduate medical education: perceptions of medical students, residents, faculty and other health professionalsMedical Education, 2004
- How to conceptualize professionalism: a qualitative studyMedical Teacher, 2004
- Essential Elements of Communication in Medical EncountersAcademic Medicine, 2001
- Situational Judgment Tests: A Review of Practice and Constructs AssessedInternational Journal of Selection and Assessment, 2001
- Use of situational judgment tests to predict job performance: A clarification of the literature.Journal of Applied Psychology, 2001
- Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions.Journal of Applied Psychology, 1997
- Developing key-feature problems and examinations to assess clinical decision-making skillsAcademic Medicine, 1995
- The Medical Council of Canadaʼs key features projectAcademic Medicine, 1995
- The assessment of clinical skills/competence/performanceAcademic Medicine, 1990