No substantial change in the balance between model-free and model-based control via training on the two-step task
Open Access
- 14 November 2019
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLoS Computational Biology
- Vol. 15 (11), e1007443
- https://doi.org/10.1371/journal.pcbi.1007443
Abstract
Human decisions can be habitual or goal-directed, also known as model-free (MF) or model-based (MB) control. Previous work suggests that the balance between the two decision systems is impaired in psychiatric disorders such as compulsion and addiction, via overreliance on MF control. However, little is known whether the balance can be altered through task training. Here, 20 healthy participants performed a well-established two-step task that differentiates MB from MF control, across five training sessions. We used computational modelling and functional near-infrared spectroscopy to assess changes in decision-making and brain hemodynamic over time. Mixed-effects modelling revealed overall no substantial changes in MF and MB behavior across training. Although our behavioral and brain findings show task-induced changes in learning rates, these parameters have no direct relation to either MF or MB control or the balance between the two systems, and thus do not support the assumption of training effects on MF or MB strategies. Our findings indicate that training on the two-step paradigm in its current form does not support a shift in the balance between MF and MB control. We discuss these results with respect to implications for restoring the balance between MF and MB control in psychiatric conditions. Psychiatric conditions such as compulsion or addiction are associated with an overreliance on habitual, or model-free, decision-making. Goal-directed, or model-based, decision-making may protect against such overreliance. We therefore asked whether model-free control could be reduced, and model-based control strengthened, via task training. We used the well-characterized two-step task that differentiates model-based from model-free actions. Our results suggest that training on the current form of the two-step task does not support a shift in the balance between model-free and model-based strategies. Factors such as devaluation, demotivation or automatization during training may play a role in the missing emergence of a training effect. Future studies could adapt the two-step task so as to separate such factors from decision-making strategies.This publication has 68 references indexed in Scilit:
- Habits, action sequences and reinforcement learningEuropean Journal of Neuroscience, 2012
- Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision TreesPLoS Computational Biology, 2012
- Quantification of the cortical contribution to the NIRS signal over the motor cortex using concurrent NIRS-fMRI measurementsNeuroImage, 2012
- Speed/Accuracy Trade-Off between the Habitual and the Goal-Directed ProcessesPLoS Computational Biology, 2011
- Model-Based Influences on Humans' Choices and Striatal Prediction ErrorsNeuron, 2011
- The latent structure of social anxiety disorder: Consequences of shifting to a dimensional diagnosis.Journal of Abnormal Psychology, 2010
- States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement LearningNeuron, 2010
- Neuronal Correlates of Instrumental Learning in the Dorsal StriatumJournal of Neurophysiology, 2009
- A specific role for posterior dorsolateral striatum in human habit learningEuropean Journal of Neuroscience, 2009
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978