Incremental Hierarchical Discriminant Regression
- 5 March 2007
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 18 (2), 397-415
- https://doi.org/10.1109/tnn.2006.889942
Abstract
This paper presents incremental hierarchical discriminant regression (IHDR) which incrementally builds a decision tree or regression tree for very high-dimensional regression or decision spaces by an online, real-time learning system. Biologically motivated, it is an approximate computational model for automatic development of associative cortex, with both bottom-up sensory inputs and top-down motor projections. At each internal node of the IHDR tree, information in the output space is used to automatically derive the local subspace spanned by the most discriminating features. Embedded in the tree is a hierarchical probability distribution model used to prune very unlikely cases during the search. The number of parameters in the coarse-to-fine approximation is dynamic and data-driven, enabling the IHDR tree to automatically fit data with unknown distribution shapes (thus, it is difficult to select the number of parameters up front). The IHDR tree dynamically assigns long-term memory to avoid the loss-of-memory problem typical with a global-fitting learning algorithm for neural networks. A major challenge for an incrementally built tree is that the number of samples varies arbitrarily during the construction process. An incrementally updated probability model, called sample-size-dependent negative-log-likelihood (SDNLL) metric is used to deal with large sample-size cases, small sample-size cases, and unbalanced sample-size cases, measured among different internal nodes of the IHDR tree. We report experimental results for four types of data: synthetic data to visualize the behavior of the algorithms, large face image data, continuous video stream from robot navigation, and publicly available data sets that use human defined featuresKeywords
This publication has 23 references indexed in Scilit:
- Fast Modular Network Implementation for Support Vector MachinesIEEE Transactions on Neural Networks, 2005
- Classifiability-Based Omnivariate Decision TreesIEEE Transactions on Neural Networks, 2005
- Omnivariate decision treesIEEE Transactions on Neural Networks, 2001
- Autonomous Mental Development by Robots and AnimalsScience, 2001
- Hierarchical discriminant regressionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Hierarchical discriminant analysis for image retrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
- Using discriminant eigenfeatures for image retrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 1996
- Eigenfaces for RecognitionJournal of Cognitive Neuroscience, 1991
- Application of the Karhunen-Loeve procedure for the characterization of human facesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1990
- Association areas of the cerebral cortexTrends in Neurosciences, 1982