Do as AI say: susceptibility in deployment of clinical decision-aids
Top Cited Papers
Open Access
- 19 February 2021
- journal article
- research article
- Published by Springer Science and Business Media LLC in npj Digital Medicine
- Vol. 4 (1), 1-8
- https://doi.org/10.1038/s41746-021-00385-9
Abstract
Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.Keywords
Funding Information
- Gouvernement du Canada | Natural Sciences and Engineering Research Council of Canada
- Microsoft Research
This publication has 41 references indexed in Scilit:
- Impact of a deep learning assistant on the histopathologic classification of liver cancernpj Digital Medicine, 2020
- Machine Learning in MedicineThe New England Journal of Medicine, 2019
- High-performance medicine: the convergence of human and artificial intelligenceNature Medicine, 2019
- Artificial intelligence in healthcareNature Biomedical Engineering, 2018
- Computerised interpretation of fetal heart rate during labour (INFANT): a randomised controlled trialThe Lancet, 2017
- Dermatologist-level classification of skin cancer with deep neural networksNature, 2017
- Effect of Robot Performance on Human–Robot Trust in Time-Critical SituationsIEEE Transactions on Human-Machine Systems, 2017
- Central Fetal Monitoring With and Without Computer AnalysisObstetrics & Gynecology, 2017
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsJAMA, 2016
- The Role of Explanations on Trust and Reliance in Clinical Decision Support SystemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015