Are you reading what we are reading? The effect of who interprets medical images on estimates of diagnostic test accuracy in systematic reviews

Abstract

Observer variation and error in the interpretation of medical images is substantial and has been described as Radiology's Achilles' heel. The enormous development in imaging technologies has brought with it an increase in the complexity and volume of images produced. There is also increased diversity as to who interprets medical images. Whilst the influence of the observer on diagnostic test performance is frequently ignored, there is evidence that this influences estimates of accuracy. Characteristics of observers that should be considered when designing systematic reviews of diagnostic test accuracy are: allocation of images to be read by observers; number, experience and training of observers; profession of observers; and assessment of observer variability and examination of its effect on test accuracy. This information could be used to inform study appraisal, data synthesis, and the investigation of sources of heterogeneity. Establishing the effect of the role of the observer on estimates of accuracy and explaining heterogeneity is important for informing the delivery of these potentially expensive and resource-intensive imaging technologies and the continuing debate about who should read the images.