VALIDATION OF WEB-BASED CALIBRATION MODULES FOR IMAGING SCORING SYSTEMS BASED ON PRINCIPLES OF ARTIFICIAL INTELLIGENCE: THE SPARCC MRI SACROILIAC JOINT INFLAMMATION SCORE
Background The application and appropriate use of imaging-based scoring instruments is usually based on passive learning from published manuscripts while real-time interaction with instrument developers is often non-feasible. Moreover, most instruments lack knowledge transfer tools that would facilitate attainment of pre-specified performance targets for reader reliability. Objectives 1. To develop a web-based calibration module for the SPARCC MRI SIJ Inflammation Score based on consensus scores from these instrument developers, experiential game psychology, and real-time iterative feedback. 2. To test the feasibility and attainment of pre-specified performance targets for reader reliability. Methods The scoring of inflammatory lesions of the SIJ on MRI using the SPARCC method is based on SIJ quadrants and the calibration module is comprised of 50 DICOM cases, each with scans from baseline and 12 weeks after the start of TNF inhibitor therapy. Scans are scored blinded-to-time-point. Continuous visual real-time feedback regarding concordance/discordance of scoring per SIJ quadrant with expert readers is provided by a color-coding scheme. Reliability is additionally assessed by real-time intra-class correlation coefficient with the first ICC data being provided after 20 cases. Accreditation for SPARCC BME score is achieved with status and change score ICC of >0.8 and>0.7 and is based on the final 20 cases. 26 readers scored the SPARCC BME module (7 rheumatology fellows, 2 chiropracters, 1 undergraduate, 8 rheumatologists, 8 radiologists) with 21 having no prior experience. Feasibility was assessed by 8-item survey. Results The majority of readers achieved accreditation for SPARCC BME score on the basis of sufficient reliability with instrument developers for both status and change scores, irrespective of prior experience (table 1). All readers who completed the module a second time, 6 months after the first exposure, achieved accreditation for SPARCC BME score. All readers rated the modules as easy and intuitive with average time for reading each case for SPARCC BME being 8 min. Conclusions Experiential web-based learning is an effective and feasible calibration tool to achieve proficiency targets in the scoring of MRI scans for SIJ inflammatory lesions. Disclosure of Interest W. Maksymowych Shareholder of: CaRE Arthritis, S. Krabbe: None declared, D. Biko: None declared, P. Weiss: None declared, M. Maksymowych: None declared, J. Cheah: None declared, G. Kröber: None declared, U. Weber: None declared, K. Danebod: None declared, P. Bird: None declared, P. Chiowchanwisawakit: None declared, J. Moeller: None declared, M. Francavilla: None declared, J. Stimec: None declared, T. Kogay: None declared, V. Zubler: None declared, M. Battish: None declared, N. Winn: None declared, D. Rumsey: None declared, R. Guglielmi: None declared, S. Pedersen: None declared, H. Boutrup: None declared, S. Shafer: None declared, J. Jaremko: None declared, F. Malik: None declared, E. Heffernan: None declared, M. Johansson: None declared, B. Trinh: None declared, J. Paschke: None declared, R. Lambert: None declared