Comparison of commonly used methods in random effects meta-analysis: application to preclinical data in drug discovery research
Open Access
- 25 February 2021
- journal article
- research article
- Published by Portico in BMJ Open Science
- Vol. 5 (1), e100074
- https://doi.org/10.1136/bmjos-2020-100074
Abstract
Background Meta-analysis of preclinical data is used to evaluate the consistency of findings and to inform the design and conduct of future studies. Unlike clinical meta-analysis, preclinical data often involve many heterogeneous studies reporting outcomes from a small number of animals. Here, we review the methodological challenges in preclinical meta-analysis in estimating and explaining heterogeneity in treatment effects. Methods Assuming aggregate-level data, we focus on two topics: (1) estimation of heterogeneity using commonly used methods in preclinical meta-analysis: method of moments (DerSimonian and Laird; DL), maximum likelihood (restricted maximum likelihood; REML) and Bayesian approach; (2) comparison of univariate versus multivariable meta-regression for adjusting estimated treatment effects for heterogeneity. Using data from a systematic review on the efficacy of interleukin-1 receptor antagonist in animals with stroke, we compare these methods, and explore the impact of multiple covariates on the treatment effects. Results We observed that the three methods for estimating heterogeneity yielded similar estimates for the overall effect, but different estimates for between-study variability. The proportion of heterogeneity explained by a covariate is estimated larger using REML and the Bayesian method as compared with DL. Multivariable meta-regression explains more heterogeneity than univariate meta-regression. Conclusions Our findings highlight the importance of careful selection of the estimation method and the use of multivariable meta-regression to explain heterogeneity. There was no difference between REML and the Bayesian method and both methods are recommended over DL. Multiple meta-regression is worthwhile to explain heterogeneity by more than one variable, reducing more variability than any univariate models and increasing the explained proportion of heterogeneity.Keywords
Funding Information
- Innovative Medicines Initiative 2 Joint Undertaking (777364)
This publication has 39 references indexed in Scilit:
- Systematic Review and Meta-Analysis of the Efficacy of Interleukin-1 Receptor Antagonist in Animal Models of Stroke: an UpdateTranslational Stroke Research, 2016
- Meta-analysis of data from animal studies: A practical guideJournal of Neuroscience Methods, 2014
- A Re-Evaluation of Random-Effects Meta-AnalysisJournal of the Royal Statistical Society Series A: Statistics in Society, 2008
- Hypothesis tests for population heterogeneity in meta‐analysisBritish Journal of Mathematical and Statistical Psychology, 2007
- Applied Mixed Models in MedicinePublished by Wiley ,2006
- Meta‐Analysis Of Controlled Clinical TrialsPublished by Wiley ,2002
- A comparison of statistical methods for meta‐analysisStatistics in Medicine, 2001
- Bayesian approaches to random‐effects meta‐analysis: A comparative studyStatistics in Medicine, 1995
- Systematic Review: Why sources of heterogeneity in meta-analysis should be investigatedBMJ, 1994
- A general parametric approach to the meta‐analysis of randomized clinical trialsStatistics in Medicine, 1991