Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews
Top Cited Papers
Open Access
- 19 April 2017
- journal article
- review article
- Published by Springer Science and Business Media LLC in BMC Medical Research Methodology
- Vol. 17 (1), 1-11
- https://doi.org/10.1186/s12874-017-0347-z
Abstract
Systematic reviews (SRs) are an important source of information about healthcare interventions. A key component of a well-conducted SR is a comprehensive literature search. There is limited evidence on the contribution of non-English reports, unpublished studies, and dissertations and their impact on results of meta-analyses. Our sample included SRs from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of reviews that searched for and included each study type; 2) proportion of relevant studies represented by each study type; and 3) impact on results and conclusions of the primary meta-analysis for each study type. Most SRs searched for non-English studies; however, these were included in only 12% of reviews and represented less than 5% of included studies. There was a change in results in only four reviews (total sample = 129); in two cases the change did not have an impact on the statistical or clinical significance of results. Most SRs searched for unpublished studies but the majority did not include these (only 6%) and they represented 2% of included studies. In most cases the impact of including unpublished studies was small; a substantial impact was observed in one case that relied solely on unpublished data. Few reviews in ARI (9%) and ID (3%) searched for dissertations compared to 65% in DPLP. Overall, dissertations were included in only nine SRs and represented less than 2% of included studies. In the majority of cases the change in results was negligible or small; in the case where a large change was noted, the estimate was more conservative without dissertations. The majority of SRs searched for non-English and unpublished studies; however, these represented a small proportion of included studies and rarely impacted the results and conclusions of the review. Inclusion of these study types may have an impact in situations where there are few relevant studies, or where there are questionable vested interests in the published literature. We found substantial variation in whether SRs searched for dissertations; in most reviews that included dissertations, these had little impact on results.Keywords
Funding Information
- Canadian Institutes of Health Research
This publication has 20 references indexed in Scilit:
- A taxonomy of rapid reviews links report types and methods to specific decision-making contextsJournal of Clinical Epidemiology, 2015
- Commentary on Cochrane review of neuraminidase inhibitors for preventing and treating influenza in healthy adults and childrenClinical Microbiology & Infection, 2015
- RAPID REVIEW: AN EMERGING APPROACH TO EVIDENCE SYNTHESIS IN HEALTH TECHNOLOGY ASSESSMENTInternational Journal of Technology Assessment in Health Care, 2014
- Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration DatabasePLoS Medicine, 2012
- Neuraminidase inhibitors for preventing and treating influenza in healthy adults and childrenPublished by Wiley ,2011
- Expediting systematic reviews: methods and implications of rapid reviewsImplementation Science, 2010
- Sources of evidence for systematic reviews of interventions in diabetesDiabetic Medicine, 2005
- Language bias in neuroscience—is the Tower of Babel located in Germany?European Psychiatry, 2004
- Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished dataThe Lancet, 2004
- Language bias in randomised controlled trials published in English and GermanThe Lancet, 1997