Dopamine, prediction error and associative learning: A model-based account
- 1 January 2006
- journal article
- research article
- Published by Informa UK Limited in Network: Computation in Neural Systems
- Vol. 17 (1), 61-84
- https://doi.org/10.1080/09548980500361624
Abstract
The notion of prediction error has established itself at the heart of formal models of animal learning and current hypotheses of dopamine function. Several interpretations of prediction error have been offered, including the model-free reinforcement learning method known as temporal difference learning (TD), and the important Rescorla–Wagner (RW) learning rule. Here, we present a model-based adaptation of these ideas that provides a good account of empirical data pertaining to dopamine neuron firing patterns and associative learning paradigms such as latent inhibition, Kamin blocking and overshadowing. Our departure from model-free reinforcement learning also offers: 1) a parsimonious distinction between tonic and phasic dopamine functions; 2) a potential generalization of the role of phasic dopamine from valence-dependent “reward” processing to valence-independent “salience” processing; 3) an explanation for the selectivity of certain dopamine manipulations on motivation for distal rewards; and 4) a plausible link between formal notions of prediction error and accounts of disturbances of thought in schizophrenia (in which dopamine dysfunction is strongly implicated). The model distinguishes itself from existing accounts by offering novel predictions pertaining to the firing of dopamine neurons in various untested behavioral scenarios.Keywords
This publication has 88 references indexed in Scilit:
- Computational roles for dopamine in behavioural controlNature, 2004
- Dopamine, learning and motivationNature Reviews Neuroscience, 2004
- The interpretation of the measurement of nucleus accumbens dopamine by in vivo dialysis: the kick, the craving or the cognition?Neuroscience & Biobehavioral Reviews, 2003
- Discrete Coding of Reward Probability and Uncertainty by Dopamine NeuronsScience, 2003
- Gating of information flow within the limbic system and the pathophysiology of schizophreniaBrain Research Reviews, 2000
- Reduced latent inhibition in people with schizophrenia: an effect of psychosis or of its treatmentThe British Journal of Psychiatry, 1998
- A Neural Substrate of Prediction and RewardScience, 1997
- Dissociation in prefrontal cortex of affective and attentional shiftsNature, 1996
- A theory of attention: Variations in the associability of stimuli with reinforcement.Psychological Review, 1975
- Latent inhibition: The effect of nonreinforced pre-exposure to the conditional stimulus.Journal of Comparative and Physiological Psychology, 1959