Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study
Open Access
- 30 January 2020
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 15 (1), e0227580
- https://doi.org/10.1371/journal.pone.0227580
Abstract
Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007–2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed.Keywords
Funding Information
- National Institute for Health Research (15/71/06)
This publication has 35 references indexed in Scilit:
- Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated ReviewPLOS ONE, 2013
- Clarifying differences between review designs and methodsSystematic Reviews, 2012
- The Cochrane Collaboration's tool for assessing risk of bias in randomised trialsBMJ, 2011
- Building the Field of Health Policy and Systems Research: An Agenda for ActionPLoS Medicine, 2011
- Building the Field of Health Policy and Systems Research: Framing the QuestionsPLoS Medicine, 2011
- Interventions to improve teamwork and communications among healthcare staffBritish Journal of Surgery, 2011
- Shared decision making interventions for people with mental health conditionsEmergencias, 2010
- External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)PLOS ONE, 2007
- Current Priorities in Health Research Funding and Lack of Impact on the Number of Child Deaths per YearAmerican Journal of Public Health, 2007