Low statistical power in biomedical science: a review of three human research domains
Open Access
- 1 February 2017
- journal article
- review article
- Published by The Royal Society in Royal Society Open Science
- Vol. 4 (2), 160254
- https://doi.org/10.1098/rsos.160254
Abstract
Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.Keywords
Funding Information
- Medical Research Council (MC_UU_12013/6)
This publication has 21 references indexed in Scilit:
- Small TelescopesPsychological Science, 2015
- Beyond Power CalculationsPerspectives on Psychological Science, 2014
- Article Commentary: On the Persistence of Low Power in Psychological ScienceThe Quarterly Journal of Experimental Psychology, 2014
- Increasing value and reducing waste in research design, conduct, and analysisThe Lancet, 2014
- Empirical evidence for low reproducibility indicates low pre-study odds.Nature Reviews Neuroscience, 2013
- Clarifications on the application and interpretation of the test for excess significance and its extensionsJournal of Mathematical Psychology, 2013
- Confidence and precision increase with high statistical powerNature Reviews Neuroscience, 2013
- Power failure: why small sample size undermines the reliability of neuroscienceNature Reviews Neuroscience, 2013
- A peculiar prevalence of p values just below .05The Quarterly Journal of Experimental Psychology, 2012
- Why Most Published Research Findings Are FalsePLoS Medicine, 2005