Behavioural science is unlikely to change the world without a heterogeneity revolution
- 22 July 2021
- journal article
- research article
- Published by Springer Science and Business Media LLC in Nature Human Behaviour
- Vol. 5 (8), 980-989
- https://doi.org/10.1038/s41562-021-01143-3
Abstract
In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.Keywords
This publication has 97 references indexed in Scilit:
- Modeling Heterogeneous Treatment Effects in Survey Experiments with Bayesian Additive Regression TreesPublic Opinion Quarterly, 2012
- The Role of Application Assistance and Information in College Decisions: Results from the H&R Block Fafsa Experiment*The Quarterly Journal of Economics, 2012
- Measuring the Prevalence of Questionable Research Practices With Incentives for Truth TellingPsychological Science, 2012
- Motivating voter turnout by invoking the selfProceedings of the National Academy of Sciences of the United States of America, 2011
- Social-Psychological Interventions in EducationReview of Educational Research, 2011
- Yes, but what’s the mechanism? (don’t expect an easy answer).Journal of Personality and Social Psychology, 2010
- Ambient belonging: How stereotypical cues impact gender participation in computer science.Journal of Personality and Social Psychology, 2009
- Richard H. Thaler, Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happinessConstitutional Political Economy, 2008
- A threat in the air: How stereotypes shape intellectual identity and performance.American Psychologist, 1997
- Statistics and Causal InferenceJournal of the American Statistical Association, 1986