Quantification of Agreement in Psychiatric Diagnosis

Abstract

THE LAST few years have seen a number of studies dealing with the problem of the reliability of psychiatric diagnoses. What makes it difficult to assimilate the various findings is the lack of uniform methods for quantifying the salient features of the data. Thus, one study will report an overall rate of perfect agreement of 54%,¹ while another will report an overall contingency coefficient of 0.714.² Still another will report that, given that one diagnostician has made a particular diagnosis, the probability that another diagnostician will make the same diagnosis is 0.57.³ Furthermore, as generally used, all of these methods suffer from one or more deficiencies which are illustrated using the hypothetical data of Table 1. (1) Chance agreement is not taken into account, or equivalently, the base rates at which the various diagnoses are made are not used to qualify the agreement measure.