Validation of a computerized assessment (PROQSY) of minor psychological morbidity by Relative Operating Characteristic analysis using a single GP's assessments as criterion measures

Abstract

Synopsis: This is a study of the criterion validity of a newly developed computerized assessment (PROQSY) of minor psychological morbidity based on the Clinical Interview Schedule, employing a single GP's assessments as criterion measures of such morbidity and Relative Operating Characteristic (ROC) analysis. Two-stage screening was carried out with a consecutive sample of 256 patients aged 15 69 consulting an experienced GP in a semi-rural setting over a six-week period. The GP made a standardized assessment of the reason for consultation and the degree of psychological disorder present. A 40% random sub-sample was then assessed using PROQSY. The area under the ROC curve using a GP criterion of minor psychological disorder (‘degree of psychological problem - mild or greater’) was 081 (s.D. = 006). However, the kappa coefficient obtained between this criterion assessment and the PROQSY assessment of minor psychological problem at the observed optimal cut-off score was only 0·28 (S.E. = 0–1; z = 2.83). It is concluded that PROQSY may be recommended for further development as a research tool providing a valid measure of minor psychological morbidity as assessed by GPs in primary care settings.