Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation
Top Cited Papers
Open Access
- 18 April 2008
- journal article
- research article
- Published by Oxford University Press (OUP) in International Journal of Epidemiology
- Vol. 37 (5), 1148-1157
- https://doi.org/10.1093/ije/dyn065
Abstract
Background Several approaches are available for evaluating heterogeneity in meta-analysis. Sensitivity analyses are often used, but these are often implemented in various non-standardized ways. Methods We developed and implemented sequential and combinatorial algorithms that evaluate the change in between-study heterogeneity as one or more studies are excluded from the calculations. The algorithms exclude studies aiming to achieve either the maximum or the minimum final I2 below a desired pre-set threshold. We applied these algorithms in databases of meta-analyses of binary outcome and ≥4 studies from Cochrane Database of Systematic Reviews (Issue 4, 2005, n = 1011) and meta-analyses of genetic associations (n = 50). Two I2 thresholds were used (50% and 25%). Results Both algorithms have succeeded in achieving the pre-specified final I2 thresholds. Differences in the number of excluded studies varied from 0% to 6% depending on the database and the heterogeneity threshold, while it was common to exclude different specific studies. Among meta-analyses with initial I2 > 50%, in the large majority [19 (90.5%) and 208 (85.9%) in genetic and Cochrane meta-analyses, respectively] exclusion of one or two studies sufficed to decrease I2 < 50%. Similarly, among meta-analyses with initial I2 > 25%, in most cases [16 (57.1%) and 382 (81.3%), respectively) exclusion of one or two studies sufficed to decrease heterogeneity even I2 (correlation coefficients 0.52–0.68 depending on algorithm used). Conclusions The proposed algorithms can be routinely applied in meta-analyses as standardized sensitivity analyses for heterogeneity. Caution is needed evaluating post hoc which specific studies are responsible for the heterogeneity.Keywords
This publication has 27 references indexed in Scilit:
- Implications of Small Effect Sizes of Individual Genetic Variants on the Design and Interpretation of Genetic Association Studies of Complex DiseasesAmerican Journal of Epidemiology, 2006
- Measuring inconsistency in meta-analysesBMJ, 2003
- A graphical method for exploring heterogeneity in meta‐analyses: application to a meta‐analysis of 65 trialsStatistics in Medicine, 2002
- Investigating causes of heterogeneity in systematic reviewsStatistics in Medicine, 2002
- Quantifying heterogeneity in a meta‐analysisStatistics in Medicine, 2002
- Systematic Reviews in Health CarePublished by Wiley ,2001
- Summing up evidence: one answer is not always enoughThe Lancet, 1998
- Systematic Review: Why sources of heterogeneity in meta-analysis should be investigatedBMJ, 1994
- A note on graphical presentation of estimated odds ratios from several clinical trialsStatistics in Medicine, 1988
- The Combination of Estimates from Different ExperimentsPublished by JSTOR ,1954