Local linear perceptrons for classification
- 1 May 1996
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 7 (3), 788-794
- https://doi.org/10.1109/72.501737
Abstract
A structure composed of local linear perceptrons for approximating global class discriminants is investigated. Such local linear models may be combined in a cooperative or competitive way. In the cooperative model, a weighted sum of the outputs of the local perceptrons is computed where the weight is a function of the distance between the input and the position of the local perceptron. In the competitive model, the cost function dictates a mixture model where only one of the local perceptrons give output. Learning of the local models' positions and the linear mappings they implement are coupled and both supervised. We show that this is preferable to the uncoupled case where the positions are trained in an unsupervised manner before the separate, supervised training of mappings. We use goodness criteria based on the cross-entropy and give learning equations for both the cooperative and competitive cases. The coupled and uncoupled versions of cooperative and competitive approaches are compared among themselves and with multilayer perceptrons of sigmoidal hidden units and radial basis functions (RBFs) of Gaussian units on the application of recognition of handwritten digits. The criteria of comparison are the generalization accuracy, learning time, and the number of free parameters. We conclude that even on such a high-dimensional problem, such local models are promising. They generalize much better than RBF's and use much less memory. When compared with multilayer perceptrons, we note that local models learn much faster and generalize as well and sometimes better with comparable number of parameters.Keywords
This publication has 8 references indexed in Scilit:
- Hierarchical Mixtures of Experts and the EM AlgorithmNeural Computation, 1994
- 'Neural-gas' network for vector quantization and its application to time-series predictionIEEE Transactions on Neural Networks, 1993
- Local Learning AlgorithmsNeural Computation, 1992
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991
- Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern RecognitionPublished by Springer Science and Business Media LLC ,1990
- Topology-conserving maps for learning visuo-motor-coordinationNeural Networks, 1989
- Fast Learning in Networks of Locally-Tuned Processing UnitsNeural Computation, 1989
- Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation NetworksNeural Computation, 1989