Neural Systems as Nonlinear Filters
- 1 August 2000
- journal article
- Published by MIT Press in Neural Computation
- Vol. 12 (8), 1743-1772
- https://doi.org/10.1162/089976600300015123
Abstract
Experimental data show that biological synapses behave quite differently from the symbolic synapses in all common artificial neural network models. Biological synapses are dynamic; their “weight” changes on a short timescale by several hundred percent in dependence of the past input to the synapse. In this article we address the question how this inherent synaptic dynamics (which should not be confused with long term learning) affects the computational power of a neural network. In particular, we analyze computations on temporal and spatiotemporal patterns, and we give a complete mathematical characterization of all filters that can be approximated by feedforward neural networks with dynamic synapses. It turns out that even with just a single hidden layer, such networks can approximate a very rich class of nonlinear filters: all filters that can be characterized by Volterra series. This result is robust with regard to various changes in the model for synaptic dynamics. Our characterization result provides for all nonlinear filters that are approximable by Volterra series a new complexity hierarchy related to the cost of implementing such filters in neural systems.Keywords
This publication has 23 references indexed in Scilit:
- Synaptic Depression and Cortical Gain ControlScience, 1997
- Sample complexity for learning recurrent perceptron mappingsIEEE Transactions on Information Theory, 1996
- Decoding neuronal firing and modelling neural networksQuarterly Reviews of Biophysics, 1994
- Cortical dynamics of feature binding and reset: Control of visual persistenceVision Research, 1994
- FIR and IIR Synapses, a New Neural Network Architecture for Time Series ModelingNeural Computation, 1991
- Neuronal Population Coding of Movement DirectionScience, 1986
- Fading memory and the problem of approximating nonlinear operators with Volterra seriesIEEE Transactions on Circuits and Systems, 1985
- Representations of nonlinear systems via the stone-weierstrass theoremAutomatica, 1976
- A neural theory of punishment and avoidance, II: quantitative theoryMathematical Biosciences, 1972
- On the production and release of chemical transmitters and related topics in cellular controlJournal of Theoretical Biology, 1969