On Estimating Diagnostic Accuracy From Studies With Multiple Raters and Partial Gold Standard Evaluation
- 1 March 2008
- journal article
- Published by Taylor & Francis Ltd in Journal of the American Statistical Association
- Vol. 103 (481), 61-73
- https://doi.org/10.1198/016214507000000329
Abstract
We are often interested in estimating sensitivity and specificity of a group of raters or a set of new diagnostic tests in situations in which gold standard evaluation is expensive or invasive. Numerous authors have proposed latent modeling approaches for estimating diagnostic error without a gold standard. Albert and Dodd showed that, when modeling without a gold standard, estimates of diagnostic error can be biased when the dependence structure between tests is misspecified. In addition, they showed that choosing between different models for this dependence structure is difficult in most practical situations. While these results caution against using these latent class models, the difficulties of obtaining gold standard verification remain a practical reality. We extend two classes of models to provide a compromise that collects gold standard information on a subset of subjects but incorporates information from both the verified and nonverified subjects during estimation. We examine the robustness of diagnostic error estimation with this approach and show that choosing between competing models is easier in this context. In our analytic work and simulations, we consider situations in which verification is completely at random as well as settings in which the probability of verification depends on the actual test results. We apply our methodological work to a study designed to estimate the diagnostic error of digital radiography for gastric cancer.Keywords
This publication has 23 references indexed in Scilit:
- Performance of Commercially Available Enzyme Immunoassays for Detection of Antibodies against Herpes Simplex Virus Type 2 in African PopulationsJournal of Clinical Microbiology, 2004
- A Cautionary Note on the Robustness of Latent Class Models for Estimating Diagnostic Error without a Gold StandardBiometrics, 2004
- Misspecified maximum likelihood estimates and generalised linear mixed modelsBiometrika, 2001
- A two-stage validation study for determining sensitivity and specificity.Environmental Health Perspectives, 1994
- Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard ConditionsJournal of the American Statistical Association, 1987
- Longitudinal data analysis using generalized linear modelsBiometrika, 1986
- The Analysis of Multivariate Binary DataJournal of the Royal Statistical Society Series C: Applied Statistics, 1972