Too good to be true: Publication bias in two prominent studies from experimental psychology
Top Cited Papers
- 15 February 2012
- journal article
- Published by Springer Science and Business Media LLC in Psychonomic Bulletin & Review
- Vol. 19 (2), 151-156
- https://doi.org/10.3758/s13423-012-0227-9
Abstract
Empirical replication has long been considered the final arbiter of phenomena in science, but replication is undermined when there is evidence for publication bias. Evidence for publication bias in a set of experiments can be found when the observed number of rejections of the null hypothesis exceeds the expected number of rejections. Application of this test reveals evidence of publication bias in two prominent investigations from experimental psychology that have purported to reveal evidence of extrasensory perception and to indicate severe limitations of the scientific method. The presence of publication bias suggests that those investigations cannot be taken as proper scientific studies of such phenomena, because critical data are not available to the field. Publication bias could partly be avoided if experimental psychologists started using Bayesian data analysis techniques.This publication has 23 references indexed in Scilit:
- Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical ResultsPLOS ONE, 2011
- Must psychologists change the way they analyze their data?Journal of Personality and Social Psychology, 2011
- A Bayes factor meta-analysis of Bem’s ESP claimPsychonomic Bulletin & Review, 2011
- Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect.Journal of Personality and Social Psychology, 2011
- How reliable are scientific studies?The British Journal of Psychiatry, 2010
- Bayesian data analysisWIREs Cognitive Science, 2010
- Comments on `An exploratory test for an excess of significant findings' by JPA loannidis and TA TrikalinosClinical Trials, 2007
- On the Post Hoc Power in Testing Mean DifferencesJournal of Educational and Behavioral Statistics, 2005
- A meta‐analysis of the verbal overshadowing effect in face identificationApplied Cognitive Psychology, 2001
- Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice VersaThe American Statistician, 1995