McNemar χ2test revisited: comparing sensitivity and specificity of diagnostic examinations

Abstract

When evaluating a novel diagnostic examination for clinical use, it should be compared with a reference standard, defined as the best available examination, which may include clinical and laboratory criteria. The novel examination and reference standard's results are usually presented in the form of a 2×2 table, which allows calculation of sensitivity, specificity and accuracy. It has been recommended that the measures of statistical uncertainty should be reported, such as the 95% confidence interval, when evaluating the accuracy of diagnostic examinations. Comparing the difference in sensitivity or specificity of a novel examination with the reference standard is important when evaluating its usefulness. The McNemar χ² test, used to compare discordance of two dichotomous responses, can be applied for this purpose. However, applying the McNemar test to a 2×2 table for comparing the accuracy of examinations is not recommended, since this test is sensitive to the proportion of positive versus negative subjects. Moreover, if the novel examination has higher sensitivity than the one considered as the reference standard, constructing a classic 2×2 table would result in low specificity of the novel examination. Thus, in order to compare sensitivities and specificities between examinations, this table is inappropriate and an independent reference standard is necessary. In this article, we propose the use of the McNemar χ² test to compare sensitivities between examinations using a 2×2 table exclusively among diseased patients, defined by a set of criteria and follow‐up of patients. Likewise, specificities can be compared applying the McNemar test among healthy individuals.