Quantifying the risk of error when interpreting funnel plots
Open Access
- 11 March 2015
- journal article
- research article
- Published by Springer Science and Business Media LLC in Systematic Reviews
- Vol. 4 (1), 1-7
- https://doi.org/10.1186/s13643-015-0004-8
Abstract
Funnel plots are widely used to investigate possible publication bias in meta-analyses. There has, however, been little formal assessment of whether a visual inspection of a funnel plot is sufficient to identify publication bias. Visual assessment of bias in a funnel plot is quantified using two new statistics: the Imbalance and the Asymmetry Distance, both intended to replicate how a funnel plot is typically assessed. A simulation study was performed to assess the performance of these two statistics for identifying publication bias. The two statistics both have high type I error and low statistical power, unless the number of studies in the meta-analysis is very large. These results suggest that visual inspection of a funnel plot is unlikely to lead to a valid assessment of publication bias. In most systematic reviews, visual inspection of a funnel plot may give a misleading impression of the presence or absence of publication bias. Formal statistical tests for bias should generally be preferred.Keywords
This publication has 10 references indexed in Scilit:
- The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviewsBMJ, 2010
- The case of the misleading funnel plotBMJ, 2006
- In an empirical evaluation of the funnel plot, researchers could not visually identify publication biasJournal of Clinical Epidemiology, 2005
- Meta-analysis, funnel plots and sensitivity analysisBiostatistics, 2000
- Misleading funnel plot for detection of bias in meta-analysisJournal of Clinical Epidemiology, 2000
- Bias in meta-analysis detected by a simple, graphical testBMJ, 1997
- The existence of publication bias and risk factors for its occurrence.1990
- The Existence of Publication Bias and Risk Factors for Its OccurrenceJAMA, 1990
- Summing UpPublished by Harvard University Press ,1984
- Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: A synthesis of findings from 18 experiments.Journal of Educational Psychology, 1984