Exploring consequences of simulation design for apparent performance of methods of meta-analysis
Open Access
- 10 June 2021
- journal article
- research article
- Published by SAGE Publications in Statistical Methods in Medical Research
- Vol. 30 (7), 1667-1690
- https://doi.org/10.1177/09622802211013065
Abstract
Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of random-effects meta-analysis of log-odds-ratios, we investigate how choices in generating data affect such conclusions. The choices we study include the overall log-odds-ratio, the distribution of probabilities in the control arm, and the distribution of study-level sample sizes. We retain the customary normal distribution of study-level effects. To examine the impact of the components of simulations, we assess the performance of the best available inverse–variance–weighted two-stage method, a two-stage method with constant sample-size-based weights, and two generalized linear mixed models. The results show no important differences between fixed and random sample sizes. In contrast, we found differences among data-generation models in estimation of heterogeneity variance and overall log-odds-ratio. This sensitivity to design poses challenges for use of simulation in choosing methods of meta-analysis.Funding Information
- Economic and Social Research Council (ES/L011859/1)
This publication has 30 references indexed in Scilit:
- A PSEUDO-LIKELIHOOD APPROACH FOR THE META-ANALYSIS OF HOMOGENEOUS TREATMENT EFFECTS: EXPLOITING THE INFORMATION CONTAINED IN SINGLE-ARM AND DOUBLE-ZERO STUDIESJournal of Statistics: Advances in Theory and Applications, 2019
- Meta-analysis of binary outcomes via generalized linear mixed models: a simulation studyBMC Medical Research Methodology, 2018
- A comparison of seven random‐effects models for meta‐analyses that estimate the summary odds ratioStatistics in Medicine, 2018
- A Re-Evaluation of Fixed Effect(s) Meta-AnalysisJournal of the Royal Statistical Society Series A: Statistics in Society, 2017
- Statistical methods for meta‐analyses including information from studies without any events—add nothing to nothing and succeed neverthelessStatistics in Medicine, 2014
- Evidence Synthesis for Decision Making 2Medical Decision Making, 2012
- Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse dataStatistics in Medicine, 2010
- A simple confidence interval for meta‐analysisStatistics in Medicine, 2002
- A multilevel model framework for meta-analysis of clinical trials with binary outcomesStatistics in Medicine, 2000
- Generalized linear mixed models for meta-analysisStatistics in Medicine, 1999