Clinical Prediction of Coronavirus Disease-2019: How Accurate Can One Be?

Abstract

Some models based on clinical information have been reported to predict which patients have Coronavirus Disease-2019 (COVID-19) pneumonia but have failed so far to yield reliable results. We aimed to determine if physicians were able to accurately predict which patients, as described in clinical vignettes, had, or did not have this infection using their clinical acumen and epidemiological data. Of 1177 patients under investigation for COVID-19 admitted, we selected 20 and presented them in a vignette form. We surveyed physicians from different levels of training (<5, and five or more years after graduation from medical school) and included non-medical participants as a control group. We asked all participants to predict the result of the PCR test for COVID-19. We measured the accuracy of responses as a whole, and at three stages of the pandemic associated with a growing incidence of COVID-19 in the community. We calculated the inter-rater reliability, sensitivity, and specificity of the clinical prediction as a whole and by pandemic stage. Between June 8 and August 28, 2020, 82 doctors and 20 non-medical participants completed the survey. The accuracy was 58% (59% for doctors and 52% for non-medical, p=0.002). The lowest accuracy was noted for cases in the pandemic middle stage; years of post-graduate training represented no difference. Of the 2040 total answers, 1176 were accurate and 864 inaccurate (349 false positives and 515 false negatives). The influence of symptomatic positivity, confirmation bias, and rapid expertise acquisition on accuracy is discussed, as the disease is new, time after graduation made no difference in the response accuracy. The limited clinical diagnostic capacity emphasizes the need for a reliable diagnostic test.