Increasing Transparency Through a Multiverse Analysis
Top Cited Papers
Open Access
- 29 September 2016
- journal article
- research article
- Published by SAGE Publications in Perspectives on Psychological Science
- Vol. 11 (5), 702-712
- https://doi.org/10.1177/1745691616658637
Abstract
Empirical research inevitably includes constructing a data set by processing raw data into a form ready for statistical analysis. Data processing often involves choices among several reasonable options for excluding, transforming, and coding data. We suggest that instead of performing only one analysis, researchers could perform a multiverse analysis, which involves performing all analyses across the whole set of alternatively processed data sets corresponding to a large set of reasonable scenarios. Using an example focusing on the effect of fertility on religiosity and political attitudes, we show that analyzing a single data set can be misleading and propose a multiverse analysis as an alternative practice. A multiverse analysis offers an idea of how much the conclusions change because of arbitrary choices in data construction and gives pointers as to which choices are most consequential in the fragility of the result.Keywords
This publication has 35 references indexed in Scilit:
- How valid are assessments of conception probability in ovulatory cycle research? Evaluations, recommendations, and theoretical implicationsEvolution and Human Behavior, 2015
- Playing the Field: The Effect of Fertility on Women’s Desire for VarietyJournal of Consumer Research, 2015
- Fertility can have different effects on single and nonsingle women: reply to Harris and Mickes (2014).Psychological Science, 2014
- Money, Status, and the Ovulatory CycleJournal of Marketing Research, 2014
- The Fluctuating Female VotePsychological Science, 2013
- Recommendations for Increasing Replicability in PsychologyEuropean Journal of Personality, 2013
- Registered Reports: A new publishing initiative at CortexCortex, 2012
- The Rules of the Game Called Psychological SciencePerspectives on Psychological Science, 2012
- Ovulation leads women to perceive sexy cads as good dads.Journal of Personality and Social Psychology, 2012
- Ovulation, Female Competition, and Product Choice: Hormonal Influences on Consumer BehaviorJournal of Consumer Research, 2011