Who You Know or What You Know? Effect of Examiner Familiarity With Residents on OSCE Scores
- 1 October 2011
- journal article
- assessment through-simulation
- Published by Ovid Technologies (Wolters Kluwer Health) in Academic Medicine
- Vol. 86, S8-S11
- https://doi.org/10.1097/acm.0b013e31822a729d
Abstract
Despite the goal of objective structured clinical examinations (OSCEs) to be objective, examiner biases may influence scores. Examiner familiarity with candidates is a potential bias that has not been well studied. To determine the effect of familiarity, OSCE scores for 158 internal medicine residents were analyzed by whether examiners were familiar with them, based on previous clinical encounters, and if previous impressions were positive or negative. A hierarchical multivariable analysis of variance was performed to control for resident, examiner, and level of training. Across 480 interactions (50 examiners, 158 residents), multivariable analysis showed that positive familiarity was associated with a significant increase in ratings (+0.37 on a 5-point scale), comparable to the difference between first- and third/fourth-year residents. Familiarity with candidates is a significant source of examiner bias in OSCE scores. Consideration should be paid to the influence of examiners' previous knowledge of examinees and attempts made to mitigate this bias.Keywords
This publication has 15 references indexed in Scilit:
- Sources of variation in performance on a shared OSCE station across four UK medical schoolsMedical Education, 2009
- The effect of candidate familiarity on examiner OSCE scoresMedical Education, 2007
- Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEsAdvances in Health Sciences Education, 2007
- How specific is case specificity?Medical Education, 2006
- A Comparison of Physician Examiners and Trained Assessors in a High-Stakes OSCE SettingAcademic Medicine, 2005
- Objectivity in Objective Structured Clinical ExaminationsAcademic Medicine, 2003
- OSCE checklists do not capture increasing levels of expertiseAcademic Medicine, 1999
- Effects of Examiner Familiarity on Black, Caucasian, and Hispanic Children: A Meta-AnalysisExceptional Children, 1989
- Introduction of an objective structured clinical examination (OSCE) to an undergraduate clinical skills programmeMedical Education, 1982
- Assessment of clinical competence using objective structured examination.BMJ, 1975