Neural Networks and the Bias/Variance Dilemma
- 1 January 1992
- journal article
- Published by MIT Press in Neural Computation
- Vol. 4 (1), 1-58
- https://doi.org/10.1162/neco.1992.4.1.1
Abstract
Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.Keywords
This publication has 49 references indexed in Scilit:
- Structural Image Restoration Through Deformable TemplatesJournal of the American Statistical Association, 1991
- Links between Markov models and multilayer perceptronsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1990
- The Perceptron Algorithm is Fast for Nonmalicious DistributionsNeural Computation, 1990
- Neural networks and principal component analysis: Learning from examples without local minimaNeural Networks, 1989
- Multiresolution elastic matchingComputer Vision, Graphics, and Image Processing, 1989
- What Size Net Gives Valid Generalization?Neural Computation, 1989
- Auto-association by multilayer perceptrons and singular value decompositionBiological Cybernetics, 1988
- Estimating Optimal Transformations for Multiple Regression and CorrelationJournal of the American Statistical Association, 1985
- A learning algorithm for boltzmann machinesCognitive Science, 1985
- Distinctive features, categorical perception, and probability learning: Some applications of a neural model.Psychological Review, 1977