Validation of Two Instruments to Assess Technical Bronchoscopic Skill Using Virtual Reality Simulation
- 11 April 2008
- journal article
- Published by S. Karger AG in Respiration
- Vol. 76 (1), 92-101
- https://doi.org/10.1159/000126493
Abstract
Background: In an era of increased emphasis on patient safety and competency-based education, demonstration of significant variability in the teaching of flexible bronchoscopy has led to initiatives for new standardized curricula and assessment tools. Objectives: To evaluate the efficacy of such curricula and to assess bronchoscopic skill, 2 measuring instruments have been developed: the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT) and the Bronchoscopy Step-by-Step Evaluation Tool (BSET). We studied the validity and reliability of these 2 instruments. Methods: Two independent testers simultaneously scored 22 volunteer participants at 3 levels [novice (n = 7), fellow (n = 8) and attending (n = 7)] on a virtual reality bronchoscopy simulator using the 2 instruments; each participant was tested twice, in 2 separate sessions. Intertester and test-retest reliability were analyzed with intraclass correlations (ICC); ANOVA was used to assess concurrent validity based on the subjects’ expected skill level. Results: The ICCs between the testers were 0.98 for both the BSTAT and BSET. Comparison of the scores between the sessions showed high test-retest reliability by ICC (0.86 and 0.85 for BSTAT and BSET respectively), with a small yet statistically significant learning effect. The novice group’s scores were lower than the fellows’ and attendings’ (p < 0.001) for both the BSTAT and BSET; the fellows’ scores were consistently lower than the attendings’ on both tests, yet the differences were not statistically significant. Conclusion: This validation study of 2 objective tests of bronchoscopic skill demonstrated high reliability and concurrent validity. These instruments can now be used to evaluate the effectiveness of new competency-based bronchoscopy curricula.Keywords
This publication has 53 references indexed in Scilit:
- American College of Rheumatology guidelines for screening, treatment, and management of lupus nephritisArthritis Care & Research, 2012
- Mycophenolate Mofetil versus Cyclophosphamide for Induction Treatment of Lupus NephritisJournal of the American Society of Nephrology, 2009
- An Evidence-Based Virtual Reality Training Program for Novice Laparoscopic SurgeonsAnnals of Surgery, 2006
- From the President of the American Association for Bronchology, The Future of Interventional PulmonologyJournal of Bronchology, 2006
- Interventional Pulmonology Competency StandardsJournal of Bronchology, 2006
- The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculumSurgical Endoscopy, 2005
- Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic reviewMedical Teacher, 2005
- Evaluation of Virtual Reality Bronchoscopy as a Learning and Assessment ToolRespiration, 2003
- Residency Program Director Evaluations Do Not Correlate With Performance on a Required 4th-Year Objective Structured Clinical ExaminationTeaching and Learning in Medicine, 2001
- Competency-based instruction in critical invasive skills improves both resident performance and patient safetySurgery, 1998