Research Practices That Can Prevent an Inflation of False-Positive Rates
Open Access
- 21 August 2013
- journal article
- research article
- Published by SAGE Publications in Personality and Social Psychology Review
- Vol. 18 (2), 107-118
- https://doi.org/10.1177/1088868313496330
Abstract
Recent studies have indicated that research practices in psychology may be susceptible to factors that increase false-positive rates, raising concerns about the possible prevalence of false-positive findings. The present article discusses several practices that may run counter to the inflation of false-positive rates. Taking these practices into account would lead to a more balanced view on the false-positive issue. Specifically, we argue that an inflation of false-positive rates would diminish, sometimes to a substantial degree, when researchers (a) have explicit a priori theoretical hypotheses, (b) include multiple replication studies in a single paper, and (c) collect additional data based on observed results. We report findings from simulation studies and statistical evidence that support these arguments. Being aware of these preventive factors allows researchers not to overestimate the pervasiveness of false-positives in psychology and to gauge the susceptibility of a paper to possible false-positives in practical and fair ways.Keywords
This publication has 55 references indexed in Scilit:
- Evidence that publication bias contaminated studies relating social class and unethical behaviorProceedings of the National Academy of Sciences of the United States of America, 2012
- Measuring the Prevalence of Questionable Research Practices With Incentives for Truth TellingPsychological Science, 2012
- Too good to be true: Publication bias in two prominent studies from experimental psychologyPsychonomic Bulletin & Review, 2012
- Adaptive Trial DesignsAnnual Review of Pharmacology and Toxicology, 2012
- Statistical Conclusion Validity: Some Common Threats and Simple RemediesFrontiers in Psychology, 2012
- The Same Old New Look: Publication Bias in a Study of Wishful SeeingI-Perception, 2012
- Type I and Type II error concerns in fMRI research: re-balancing the scaleSocial Cognitive and Affective Neuroscience, 2009
- Why Current Publication Practices May Distort SciencePLoS Medicine, 2008
- Why Most Published Research Findings Are FalsePLoS Medicine, 2005
- A Multiple Comparison Procedure for Comparing Several Treatments with a ControlJournal of the American Statistical Association, 1955