Alterations in choice behavior by manipulations of world model
- 30 August 2010
- journal article
- Published by Proceedings of the National Academy of Sciences in Proceedings of the National Academy of Sciences of the United States of America
- Vol. 107 (37), 16401-16406
- https://doi.org/10.1073/pnas.1001709107
Abstract
How to compute initially unknown reward values makes up one of the key problems in reinforcement learning theory, with two basic approaches being used. Model-free algorithms rely on the accumulation of substantial amounts of experience to compute the value of actions, whereas in model-based learning, the agent seeks to learn the generative process for outcomes from which the value of actions can be predicted. Here we show that (i) "probability matching"-a consistent example of suboptimal choice behavior seen in humans-occurs in an optimal Bayesian model-based learner using a max decision rule that is initialized with ecologically plausible, but incorrect beliefs about the generative process for outcomes and (ii) human behavior can be strongly and predictably altered by the presence of cues suggestive of various generative processes, despite statistically identical outcome generation. These results suggest human decision making is rational and model based and not consistent with model-free learning.Keywords
This publication has 27 references indexed in Scilit:
- Striatal Activity Underlies Novelty-Based Choice in HumansNeuron, 2008
- Noise in the nervous systemNature Reviews Neuroscience, 2008
- Probability matching involves rule-generating ability: A neuropsychological mechanism dealing with probabilities.Neuropsychology, 2007
- Cortical substrates for exploratory decisions in humansNature, 2006
- Searching for Patterns in Random Sequences.Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 2004
- Is probability matching smart? Associations between probabilistic choices and cognitive abilityMemory & Cognition, 2003
- A re‐examination of probability matching and rational choiceJournal of Behavioral Decision Making, 2002
- Probability matching: Encouraging optimal responding in humans.Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 2002
- An Economist’s Perspective on Probability MatchingJournal of Economic Surveys, 2000
- Perception of the statistical structure of a random series of binary symbols.Journal of Experimental Psychology, 1953