Empirical Comparison of Publication Bias Tests in Meta-Analysis
Top Cited Papers
- 1 August 2018
- journal article
- research article
- Published by Springer Science and Business Media LLC in Journal of General Internal Medicine
- Vol. 33 (8), 1260-1267
- https://doi.org/10.1007/s11606-018-4425-7
Abstract
Decision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data. This study compares seven commonly used publication bias tests (i.e., Begg's rank test, trim-and-fill, Egger's, Tang's, Macaskill's, Deeks', and Peters' regression tests) based on 28,655 meta-analyses available in the Cochrane Library. Egger's regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg's rank test and the trim-and-fill method. The agreement among Tang's, Macaskill's, Deeks', and Peters' regression tests for binary outcomes was moderately strong (most kappa's were around 0.6). Tang's and Deeks' tests had fairly similar performance (kappa > 0.9). The agreement among Begg's rank test, the trim-and-fill method, and Egger's regression test was weak or moderate (kappa < 0.5). Given the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.Keywords
This publication has 45 references indexed in Scilit:
- Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trialsBMJ, 2010
- Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation studyBMC Medical Research Methodology, 2009
- Reasons or excuses for avoiding meta-analysis in forest plotsBMJ, 2008
- Performance of the trim and fill method in the presence of publication bias and between‐study heterogeneityStatistics in Medicine, 2007
- Comparison of Two Methods to Detect Publication Bias in Meta-analysisJama-Journal Of The American Medical Association, 2006
- A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpointsStatistics in Medicine, 2005
- What to add to nothing? Use and avoidance of continuity corrections in meta‐analysis of sparse dataStatistics in Medicine, 2004
- Measuring inconsistency in meta-analysesBMJ, 2003
- Quantifying heterogeneity in a meta-analysisStatistics in Medicine, 2002
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960