Stacked Extreme Learning Machines
- 28 October 2014
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 45 (9), 2013-2025
- https://doi.org/10.1109/tcyb.2014.2363492
Abstract
Extreme learning machine (ELM) has recently attracted many researchers' interest due to its very fast learning speed, good generalization ability, and ease of implementation. It provides a unified solution that can be used directly to solve regression, binary, and multiclass classification problems. In this paper, we propose a stacked ELMs (S-ELMs) that is specially designed for solving large and complex data problems. The S-ELMs divides a single large ELM network into multiple stacked small ELMs which are serially connected. The S-ELMs can approximate a very large ELM network with small memory requirement. To further improve the testing accuracy on big data problems, the ELM autoencoder can be implemented during each iteration of the S-ELMs algorithm. The simulation results show that the S-ELMs even with random hidden nodes can achieve similar testing accuracy to support vector machine (SVM) while having low memory requirements. With the help of ELM autoencoder, the S-ELMs can achieve much better testing accuracy than SVM and slightly better accuracy than deep belief network (DBN) with much faster training speed.Keywords
Funding Information
- Singapore Academic Research Fund (AcRF) Tier 1 (RG 22/08 (M52040128))
- Singapore’s National Research Foundation (NRF2011NRF-CRP001-090)
This publication has 20 references indexed in Scilit:
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Enhanced random search based incremental extreme learning machineNeurocomputing, 2008
- Convex incremental extreme learning machineNeurocomputing, 2007
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden NodesIEEE Transactions on Neural Networks, 2006
- Nonlinear Autoassociation Is Not Equivalent to PCANeural Computation, 2000
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- A database for handwritten text recognition researchIEEE Transactions on Pattern Analysis and Machine Intelligence, 1994
- A Resource-Allocating Network for Function InterpolationNeural Computation, 1991