Meta-learning within Projective Simulation
Open Access
- 25 April 2016
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 4, 2110-2122
- https://doi.org/10.1109/access.2016.2556579
Abstract
Learning models of artificial intelligence can nowadays perform very well on a large variety of tasks. However, in practice, different task environments are best handled by different learning models, rather than a single universal approach. Most non-trivial models thus require the adjustment of several to many learning parameters, which is often done on a case-by-case basis by an external party. Meta-learning refers to the ability of an agent to autonomously and dynamically adjust its own learning parameters or meta-parameters. In this paper, we show how projective simulation, a recently developed model of artificial intelligence, can naturally be extended to account for meta-learning in reinforcement learning settings. The projective simulation approach is based on a random walk process over a network of clips. The suggested meta-learning scheme builds upon the same design and employs clip networks to monitor the agent's performance and to adjust its meta-parameters on the fly. We distinguish between reflex-type adaptation and adaptation through learning, and show the utility of both approaches. In addition, a trade-off between flexibility and learning-time is addressed. The extended model is examined on three different kinds of reinforcement learning tasks, in which the agent has different optimal values of the meta-parameters, and is shown to perform well, reaching near-optimal to optimal success rates in all of them, without ever needing to manually adjust any meta-parameter.Funding Information
- Austrian Science Fund through the FoQuS Project (F4012)
- Templeton World Charity Foundation (TWCF0078/AB46)
This publication has 33 references indexed in Scilit:
- Projective simulation for artificial intelligenceScientific Reports, 2012
- MetalearningScholarpedia, 2010
- Model selection for the LS-SVM. Application to handwriting recognitionPattern Recognition, 2009
- Tuning continual exploration in reinforcement learning: An optimality property of the Boltzmann strategyNeurocomputing, 2008
- Discrete Quantum Walks Hit Exponentially FasterProbability Theory and Related Fields, 2005
- Introduction to the Special Issue on Meta-LearningMachine Learning, 2004
- Control of exploitation–exploration meta-parameter in reinforcement learningNeural Networks, 2002
- No free lunch theorems for optimizationIEEE Transactions on Evolutionary Computation, 1997
- Classifier Fitness Based on AccuracyEvolutionary Computation, 1995
- Quantum random walksPhysical Review A, 1993