Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail
Open Access
- 31 October 2018
- journal article
- research article
- Published by Springer Science and Business Media LLC in Journal of Computational Neuroscience
- Vol. 45 (3), 163-172
- https://doi.org/10.1007/s10827-018-0702-z
Abstract
Replicability and reproducibility of computational models has been somewhat understudied by “the replication movement.” In this paper, we draw on methodological studies into the replicability of psychological experiments and on the mechanistic account of explanation to analyze the functions of model replications and model reproductions in computational neuroscience. We contend that model replicability, or independent researchers' ability to obtain the same output using original code and data, and model reproducibility, or independent researchers' ability to recreate a model without original code, serve different functions and fail for different reasons. This means that measures designed to improve model replicability may not enhance (and, in some cases, may actually damage) model reproducibility. We claim that although both are undesirable, low model reproducibility poses more of a threat to long-term scientific progress than low model replicability. In our opinion, low model reproducibility stems mostly from authors' omitting to provide crucial information in scientific papers and we stress that sharing all computer code and data is not a solution. Reports of computational studies should remain selective and include all and only relevant bits of code.Keywords
Funding Information
- National Science Centre, Poland (2014/14/E/HS1/00803, 2015/19/B/HS1/03310)
- Foundation For Polish Science (START Scholarship)
This publication has 51 references indexed in Scilit:
- Ten Simple Rules for Reproducible Computational ResearchPLoS Computational Biology, 2013
- Reproducible Research in Computational ScienceScience, 2011
- Modeling Signal Transduction Leading to Synaptic Plasticity: Evaluation and Comparison of Five ModelsEURASIP Journal on Bioinformatics and Systems Biology, 2011
- NeuroML: A Language for Describing Data Driven Models of Neurons and Networks with a High Degree of Biological DetailPLoS Computational Biology, 2010
- Towards Reproducible Descriptions of Neuronal Network ModelsPLoS Computational Biology, 2009
- Glutamate regulation of calcium and IP3 oscillating and pulsating dynamics in astrocytesJournal of Biological Physics, 2009
- Why Are Computational Neuroscience and Systems Biology So Separate?PLoS Computational Biology, 2008
- A mathematical model of spontaneous calcium(II) oscillations in astrocytesJournal of Theoretical Biology, 2008
- PyNN: a common interface for neuronal network simulatorsFrontiers in Neuroscience, 2008
- Thinking about MechanismsPhilosophy of Science, 2000