Vigor in the Face of Fluctuating Rates of Reward: An Experimental Examination
- 1 December 2011
- journal article
- research article
- Published by MIT Press in Journal of Cognitive Neuroscience
- Vol. 23 (12), 3933-3938
- https://doi.org/10.1162/jocn_a_00090
Abstract
Two fundamental questions underlie the expression of behavior, namely what to do and how vigorously to do it. The former is the topic of an overwhelming wealth of theoretical and empirical work particularly in the fields of reinforcement learning and decision-making, with various forms of affective prediction error playing key roles. Although vigor concerns motivation, and so is the subject of many empirical studies in diverse fields, it has suffered a dearth of computational models. Recently, Niv et al. [Niv, Y., Daw, N. D., Joel, D., & Dayan, P. Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berlin), 191, 507–520, 2007] suggested that vigor should be controlled by the opportunity cost of time, which is itself determined by the average rate of reward. This coupling of reward rate and vigor can be shown to be optimal under the theory of average return reinforcement learning for a particular class of tasks but may also be a more general, perhaps hard-wired, characteristic of the architecture of control. We, therefore, tested the hypothesis that healthy human participants would adjust their RTs on the basis of the average rate of reward. We measured RTs in an odd-ball discrimination task for rewards whose magnitudes varied slowly but systematically. Linear regression on the subjects' individual RTs using the time varying average rate of reward as the regressor of interest, and including nuisance regressors such as the immediate reward in a round and in the preceding round, showed that a significant fraction of the variance in subjects' RTs could indeed be explained by the rate of experienced reward. This validates one of the key proposals associated with the model, illuminating an apparently mandatory form of coupling that may involve tonic levels of dopamine.Keywords
This publication has 39 references indexed in Scilit:
- Opponency Revisited: Competition and Cooperation Between Dopamine and SerotoninNeuropsychopharmacology, 2010
- Serotonin and Dopamine: Unifying Affective, Activational, and Decision FunctionsNeuropsychopharmacology, 2010
- Serotonin in Affective ControlAnnual Review of Neuroscience, 2009
- Reward-Motivated Learning: Mesolimbic Activation Precedes Memory FormationNeuron, 2006
- The computational neurobiology of learning and rewardCurrent Opinion in Neurobiology, 2006
- Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error SignalNeuron, 2005
- Tryptophan Depletion Disrupts the Motivational Guidance of Goal-Directed Behavior as a Function of Trait ImpulsivityNeuropsychopharmacology, 2005
- Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmissionNature Neuroscience, 2003
- What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?Brain Research Reviews, 1998
- 5-HT and mechanisms of defenceJournal of Psychopharmacology, 1991