Role of formative assessment in predicting academic success among GP registrars: a retrospective longitudinal study
Open Access
- 1 January 2020
- Vol. 10 (11), e040290
- https://doi.org/10.1136/bmjopen-2020-040290
Abstract
ObjectivesThe James Cook University General Practice Training (JCU GPT) programme's internal formative exams were compared with the Royal Australian College of General Practitioners (RACGP) pre-entry exams to determine ability to predict final performance in the RACGP fellowship exams.DesignA retrospective longitudinal study.SettingGeneral Practice (GP) trainees enrolled between 2016 and 2019 at a Registered Training Organisation in regional Queensland, Australia.Participants376 GP trainees enrolled in the training programme.Exposure measuresThe pre-entry exams were Multiple-Mini Interviews (MMI), Situational Judgement Test (SJT) and Candidate Assessment and Applied Knowledge Test. The internal formative exams comprised multiple choice questions (MCQ1 and MCQ2), short answer questions, clinical skills and clinical reasoning.Primary outcome measureThe college exams were Applied Knowledge Test (AKT), Key Feature Problems (KFP) and Objective Structured Clinical Examination (OSCE).ResultsCorrelations (r), coefficients of determination (R-2) and OR were used as parameters for estimating strength of relationship and precision of predictive accuracy. SJT and MMI were moderately (r=0.13to 0.31) and MCQ1 and MCQ2 highly (r=0.37to 0.53) correlated with all college exams (p<0.05to p<0.01), with R-2 ranging from 0.070 to 0.376. MCQ1 was predictive of failure in all college exams (AKT: OR=2.32, KFP: OR=3.99; OSCE: OR=3.46); while MCQ2 predicted failure in AKT (OR=2.83) and KFP (OR=3.15).ConclusionWe conclude that the internal MCQ formative exams predict performance in the RACGP fellowship exams. We propose that our formative assessment tools could be used as academic markers for early identification of potentially struggling trainees.Keywords
This publication has 32 references indexed in Scilit:
- The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort studyBMC Medical Education, 2013
- The remediation challenge: theoretical and methodological insights from a systematic reviewMedical Education, 2013
- Predictors of Attrition and Academic Success of Medical Students: A 30-Year Retrospective StudyPLOS ONE, 2012
- The use of progress testingPerspectives on Medical Education, 2012
- How well do selection tools predict performance later in a medical programme?Advances in Health Sciences Education, 2011
- Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 ConferenceMedical Teacher, 2011
- The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical SchoolBMC Medical Education, 2010
- Medical Student Distress: Causes, Consequences, and Proposed SolutionsMayo Clinic Proceedings, 2005
- The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.Journal of Applied Psychology, 2005
- Educational impact of in-training assessment (ITA) in postgraduate medical education: a qualitative study of an ITA programme in actual practiceMedical Education, 2004