Revised standards for statistical evidence
Open Access
- 11 November 2013
- journal article
- research article
- Published by Proceedings of the National Academy of Sciences in Proceedings of the National Academy of Sciences of the United States of America
- Vol. 110 (48), 19313-19317
- https://doi.org/10.1073/pnas.1313476110
Abstract
Recent advances in Bayesian hypothesis testing have led to the development of uniformly most powerful Bayesian tests, which represent an objective, default class of Bayesian hypothesis tests that have the same rejection regions as classical significance tests. Based on the correspondence between these two classes of tests, it is possible to equate the size of classical hypothesis tests with evidence thresholds in Bayesian tests, and to equate P values with Bayes factors. An examination of these connections suggest that recent concerns over the lack of reproducibility of scientific studies can be attributed largely to the conduct of significance tests at unjustifiably high levels of significance. To correct this problem, evidence thresholds required for the declaration of a significant finding should be increased to 25–50:1, and to 100–200:1 for the declaration of a highly significant finding. In terms of classical hypothesis tests, these evidence standards mandate the conduct of tests at the 0.005 or 0.001 level of significance. Significance The lack of reproducibility of scientific research undermines public confidence in science and leads to the misuse of resources when researchers attempt to replicate and extend fallacious research findings. Using recent developments in Bayesian hypothesis testing, a root cause of nonreproducibility is traced to the conduct of significance tests at inappropriately high levels of significance. Modifications of common standards of evidence are proposed to reduce the rate of nonreproducibility of scientific research by a factor of 5 or greater.Keywords
This publication has 19 references indexed in Scilit:
- Evidence that publication bias contaminated studies relating social class and unethical behaviorProceedings of the National Academy of Sciences of the United States of America, 2012
- Statistical Evidence in Experimental PsychologyPerspectives on Psychological Science, 2011
- What is the probability of replicating a statistically significant effect?Psychonomic Bulletin & Review, 2009
- A Bayesian Perspective on Hypothesis TestingPsychological Science, 2006
- Contradicted and Initially Stronger Effects in Highly Cited Clinical ResearchJama-Journal Of The American Medical Association, 2005
- Bayes FactorsJournal of the American Statistical Association, 1995
- Bayes FactorsJournal of the American Statistical Association, 1995
- Testing a Point Null Hypothesis: The Irreconcilability of P Values and EvidenceJournal of the American Statistical Association, 1987
- Bayesian statistical inference for psychological research.Psychological Review, 1963