How Inhibitory Oscillations Can Train Neural Networks and Punish Competitors
- 1 July 2006
- journal article
- Published by MIT Press in Neural Computation
- Vol. 18 (7), 1577-1610
- https://doi.org/10.1162/neco.2006.18.7.1577
Abstract
We present a new learning algorithm that leverages oscillations in the strength of neural inhibition to train neural networks. Raising inhibition can be used to identify weak parts of target memories, which are then strengthened. Conversely, lowering inhibition can be used to identify competitors, which are then weakened. To update weights, we apply the Contrastive Hebbian Learning equation to successive time steps of the network. The sign of the weight change equation varies as a function of the phase of the inhibitory oscillation. We show that the learning algorithm can memorize large numbers of correlated input patterns without collapsing and that it shows good generalization to test patterns that do not exactly match studied patterns.Keywords
This publication has 34 references indexed in Scilit:
- Rethinking interference theory: Executive control and the mechanisms of forgettingJournal of Memory and Language, 2003
- Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach.Psychological Review, 2003
- Hippocampal Theta Oscillations and Classical ConditioningNeurobiology of Learning and Memory, 2001
- Generative models for discovering sparse distributed representationsPhilosophical Transactions Of The Royal Society B-Biological Sciences, 1997
- Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory.Psychological Review, 1995
- On the status of inhibitory mechanisms in cognition: Memory retrieval as a model case.Psychological Review, 1995
- Setting the Activity Level in Sparse Random NetworksNeural Computation, 1994
- Hippocampal synaptic enhancement and information storage within a distributed memory systemTrends in Neurosciences, 1987
- A learning algorithm for boltzmann machinesCognitive Science, 1985
- Simple memory: a theory for archicortexPhilosophical Transactions of the Royal Society of London. B, Biological Sciences, 1971