Measuring the Accuracy of Diagnostic Systems
- 3 June 1988
- journal article
- research article
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 240 (4857), 1285-1293
- https://doi.org/10.1126/science.3287615
Abstract
Diagnostic systems of several kinds are used to distinguish between two classes of events, essentially "signals" and "noise". For them, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy. It is the only measure available that is uninfluenced by decision biases and prior probabilities, and it places the performances of diverse systems on a common, easily interpreted scale. Representative values of this measure are reported here for systems in medical imaging, materials testing, weather forecasting, information retrieval, polygraph lie detection, and aptitude testing. Though the measure itself is sound, the values obtained from tests of diagnostic systems often require qualification because the test data on which they are based are of unsure quality. A common set of problems in testing is faced in all fields. How well these problems are handled, or can be handled in a given field, determines the degree of confidence that can be placed in a measured value of accuracy. Some fields fare much better than others.This publication has 29 references indexed in Scilit:
- Teaching thinking skills.American Psychologist, 1986
- The Effect of Verification on the Assessment of Imaging TechniquesInvestigative Radiology, 1983
- Statistical versus clinical lie detection.American Psychologist, 1981
- Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic TestsThe New England Journal of Medicine, 1978
- MEASURES OF LANGUAGE EFFECTIVENESS AND THE SWETSIAN HYPOTHESESJournal of Documentation, 1975
- THE ANOMALOUS BEHAVIOUR OF PRECISION IN THE SWETS MODEL, AND ITS RESOLUTIONJournal of Documentation, 1974
- The Relative Operating Characteristic in PsychologyScience, 1973
- Effectiveness of information retrieval methodsAmerican Documentation, 1969
- THE MEASURES OF INFORMATION RETRIEVAL EFFECTIVENESS PROPOSED BY SWETSJournal of Documentation, 1968
- Information Retrieval SystemsScience, 1963