Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up
- 1 October 2015
- journal article
- Published by Royal College of Psychiatrists in The British Journal of Psychiatry
- Vol. 207 (4), 357-362
- https://doi.org/10.1192/bjp.bp.113.143701
Abstract
Background Contradictions and initial overestimates are not unusual among highly cited studies. However, this issue has not been researched in psychiatry. Aims To assess how highly cited studies in psychiatry are replicated by subsequent studies. Method We selected highly cited studies claiming effective psychiatric treatments in the years 2000 through 2002. For each of these studies we searched for subsequent studies with a better-controlled design, or with a similar design but a larger sample. Results Among 83 articles recommending effective interventions, 40 had not been subject to any attempt at replication, 16 were contradicted, 11 were found to have substantially smaller effects and only 16 were replicated. The standardised mean differences of the initial studies were overestimated by 132%. Studies with a total sample size of 100 or more tended to produce replicable results. Conclusions Caution is needed when a study with a small sample size reports a large effect.This publication has 22 references indexed in Scilit:
- Extended results of the Alzheimer's disease anti‐inflammatory prevention trialAlzheimer's & Dementia, 2011
- A 3‐week, randomized, placebo‐controlled trial of asenapine in the treatment of acute mania in bipolar mania and mixed statesBipolar Disorders, 2009
- Evaluating the EvidenceCirculation, 2008
- Why Most Discovered True Associations Are InflatedEpidemiology, 2008
- Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological studyBMJ, 2008
- Why Most Published Research Findings Are FalsePLoS Medicine, 2005
- Olanzapine versus placebo in the treatment of psychosis with or without associated behavioral disturbances in patients with Alzheimer's diseaseInternational Journal of Geriatric Psychiatry, 2004
- A simple method for converting an odds ratio to effect size for use in meta-analysisStatistics in Medicine, 2000
- Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophreniaThe British Journal of Psychiatry, 2000
- A RATING SCALE FOR DEPRESSIONJournal of Neurology, Neurosurgery & Psychiatry, 1960