Can mental health diagnoses in administrative data be used for research? A systematic review of the accuracy of routinely collected diagnoses
Open Access
- 26 July 2016
- journal article
- review article
- Published by Springer Science and Business Media LLC in BMC Psychiatry
- Vol. 16 (1), 1-11
- https://doi.org/10.1186/s12888-016-0963-x
Abstract
There is increasing availability of data derived from diagnoses made routinely in mental health care, and interest in using these for research. Such data will be subject to both diagnostic (clinical) error and administrative error, and so it is necessary to evaluate its accuracy against a reference-standard. Our aim was to review studies where this had been done to guide the use of other available data. We searched PubMed and EMBASE for studies comparing routinely collected mental health diagnosis data to a reference standard. We produced diagnostic category-specific positive predictive values (PPV) and Cohen’s kappa for each study. We found 39 eligible studies. Studies were heterogeneous in design, with a wide range of outcomes. Administrative error was small compared to diagnostic error. PPV was related to base rate of the respective condition, with overall median of 76 %. Kappa results on average showed a moderate agreement between source data and reference standard for most diagnostic categories (median kappa = 0.45–0.55); anxiety disorders and schizoaffective disorder showed poorer agreement. There was no significant benefit in accuracy for diagnoses made in inpatients. The current evidence partly answered our questions. There was wide variation in the quality of source data, with a risk of publication bias. For some diagnoses, especially psychotic categories, administrative data were generally predictive of true diagnosis. For others, such as anxiety disorders, the data were less satisfactory. We discuss the implications of our findings, and the need for researchers to validate routine diagnostic data.Keywords
Funding Information
- UK Biobank
- Scottish Funding Council
- National Institute for Health Research Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London
This publication has 72 references indexed in Scilit:
- Money matters: does the reimbursement policy for second-generation antipsychotics influence the number of recorded schizophrenia patients and the burden of stigmatization?Social psychiatry. Sozialpsychiatrie. Psychiatrie sociale, 2013
- Studies using English administrative data (Hospital Episode Statistics) to assess health-care outcomes—systematic review and recommendations for reportingEuropean Journal of Public Health, 2012
- Classification of bipolar disorder in psychiatric hospital. a prospective cohort studyBMC Psychiatry, 2012
- Systematic review of discharge coding accuracyJournal of Public Health, 2012
- Validity of bipolar disorder hospital discharge diagnoses: file review and multiple register linkage in SwedenActa Psychiatrica Scandinavica, 2011
- Diagnostic ambivalence: psychiatric workarounds and the Diagnostic and Statistical Manual of Mental DisordersSociology of Health & Illness, 2010
- The South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLAM BRC) case register: development and descriptive dataBMC Psychiatry, 2009
- Validity of the diagnosis of a single depressive episode in a case registerClinical Practice and Epidemiology in Mental Health, 2009
- Prevalence, Severity, and Unmet Need for Treatment of Mental Disorders in the World Health Organization World Mental Health SurveysPublished by American Medical Association (AMA) ,2004
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960