Addressing the theory crisis in psychology
Top Cited Papers
- 12 September 2019
- journal article
- review article
- Published by Springer Science and Business Media LLC in Psychonomic Bulletin & Review
- Vol. 26 (5), 1596-1618
- https://doi.org/10.3758/s13423-019-01645-2
Abstract
A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans.Keywords
This publication has 69 references indexed in Scilit:
- A tutorial on sensitivity analyses in clinical trials: the what, why, when and howBMC Medical Research Methodology, 2013
- Variability in encoding precision accounts for visual short-term memory limitationsProceedings of the National Academy of Sciences of the United States of America, 2012
- Turning the Hands of TimeSocial Psychological and Personality Science, 2011
- What is the probability of replicating a statistically significant effect?Psychonomic Bulletin & Review, 2009
- Experiencing Physical Warmth Promotes Interpersonal WarmthScience, 2008
- A context-based theory of recency and contiguity in free recall.Psychological Review, 2008
- Dynamic Shifts of Limited Working Memory Resources in Human VisionScience, 2008
- Dual-process theory and signal-detection theory of recognition memory.Psychological Review, 2007
- Why Most Published Research Findings Are FalsePLoS Medicine, 2005
- Attentional limits on the perception and memory of visual information.Journal of Experimental Psychology: Human Perception and Performance, 1990