A comparison of the Dorfman–Berbaum–Metz and Obuchowski–Rockette methods for receiver operating characteristic (ROC) data

Abstract

There are several different statistical methods for analysing multireader ROC studies, with the Dorfman–Berbaum–Metz (DBM) method being the most frequently used. Another method is the corrected F method proposed by Obuchowski and Rockette (OR). The DBM and OR procedures at first appear quite different: DBM is a three‐way ANOVA analysis of pseudovalues while OR is a two‐way ANOVA analysis of accuracy estimates with correlated errors. We show that the original DBM and OR F statistics for testing the null hypothesis of equal treatments have the same form and will typically have similar values; however, differences in the denominator degrees of freedom will result in differences in p‐values even when the F statistics are identical. We show how the methods can be generalized to include variations in the accuracy measure, covariance method, and degrees of freedom. Identical results are obtained when the methods agree with respect to all three of these procedure parameters; hence for a particular choice of procedure parameters the choice of method appears to depend mainly on software preference and availability. The methods are compared using data from a factorial study with two modalities, five readers, and 114 patients. Copyright © 2005 John Wiley & Sons, Ltd.