Estimating Learning Outcomes From Pre- and Posttest Student Self-Assessments
- 1 March 2013
- journal article
- research article
- Published by Ovid Technologies (Wolters Kluwer Health) in Academic Medicine
- Vol. 88 (3), 369-375
- https://doi.org/10.1097/acm.0b013e318280a6f6
Abstract
Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course. In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives. Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains. The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.Keywords
This publication has 18 references indexed in Scilit:
- Towards outcome-based programme evaluation: Using student comparative self-assessments to determine teaching effectivenessMedical Teacher, 2011
- The Development and Implementation of a Health-System-Wide Evaluation System for Education Activities: Build It and They Will ComeAcademic Medicine, 2009
- Enhancing Evaluation in an Undergraduate Medical Education ProgramAcademic Medicine, 2008
- The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10Medical Teacher, 2008
- Accuracy of Physician Self-assessment Compared With Observed Measures of CompetenceJama-Journal Of The American Medical Association, 2006
- A longitudinal study of self-assessment accuracy.Medical Education, 2003
- Multidimensional Outcome Considerations in Assessing the Efficacy of Medical Educational ProgramsTeaching and Learning in Medicine, 2003
- The misunderstood limits of folk science: an illusion of explanatory depthCognitive Science, 2002
- Measuring Self-assessment: Current State of the ArtAdvances in Health Sciences Education, 2002
- A review of the validity and accuracy of self-assessments in health professions trainingAcademic Medicine, 1991