False-Positive Psychology
Top Cited Papers
- 17 October 2011
- journal article
- Published by SAGE Publications in Psychological Science
- Vol. 22 (11), 1359-1366
- https://doi.org/10.1177/0956797611417632
Abstract
In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.Keywords
This publication has 8 references indexed in Scilit:
- Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).Journal of Personality and Social Psychology, 2011
- Motivated Reasoning and Performance on the was on Selection TaskPersonality and Social Psychology Bulletin, 2002
- Explaining Bargaining Impasse: The Role of Self-Serving BiasesJournal of Economic Perspectives, 1997
- The case for motivated reasoning.Psychological Bulletin, 1990
- Biased evaluation and persistence in gambling.Journal of Personality and Social Psychology, 1983
- Attribution of success and failure revisited, or: The motivational bias is alive and well in attribution theoryJournal of Personality, 1979
- Group sequential methods in the design and analysis of clinical trialsBiometrika, 1977
- They saw a game; a case study.The Journal of Abnormal and Social Psychology, 1954