Hierarchical discriminant regression
Open Access
- 1 January 2000
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Ieee Transactions On Pattern Analysis and Machine Intelligence
- Vol. 22 (11), 1277-1293
- https://doi.org/10.1109/34.888712
Abstract
The main motivation of this paper is to propose a new classification and regression method for challenging high-dimensional data. The proposed new technique casts classification problems (class labels as output) and regression problems (numeric values as output) into a unified regression problem. This unified view enables classification problems to use numeric information in the output space that is available for regression problems but are traditionally not readily available for classification problems驴distance metric among clustered class labels for coarse and fine classifications. A doubly clustered subspace-based hierarchical discriminating regression (HDR) method is proposed in this work. The major characteristics include: 1) Clustering is performed in both output space and input space at each internal node, termed 驴doubly clustered.驴 Clustering in the output space provides virtual labels for computing clusters in the input space. 2) Discriminants in the input space are automatically derived from the clusters in the input space. These discriminants span the discriminating subspace at each internal node of the tree. 3) A hierarchical probability distribution model is applied to the resulting discriminating subspace at each internal node. This realizes a coarse-to-fine approximation of probability distribution of the input samples, in the hierarchical discriminating subspaces. No global distribution models are assumed. 4) To relax the per class sample requirement of traditional discriminant analysis techniques, a sample-size dependent negative-log-likelihood (NLL) is introduced. This new technique is designed for automatically dealing with small-sample applications, large-sample applications, and unbalanced-sample applications. 5) The execution of HDR method is fast, due to the empirical logarithmic time complexity of the HDR algorithm. Although the method is applicable to any data, we report the experimental results for three types of data: synthetic data for examining the near-optimal performance, large raw face-image data bases, and traditional databases with manually selected features along with a comparison with some major existing methods, such as CART, C5.0, and OC1.Keywords
This publication has 13 references indexed in Scilit:
- Geometric Hashing: A General And Efficient Model-based Recognition SchemePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Automatic Construction of Decision Trees from Data: A Multi-Disciplinary SurveyData Mining and Knowledge Discovery, 1998
- The FERET September 1996 database and evaluation procedurePublished by Springer Science and Business Media LLC ,1997
- The Nature of Statistical Learning TheoryPublished by Springer Science and Business Media LLC ,1995
- Visual learning and recognition of 3-d objects from appearanceInternational Journal of Computer Vision, 1995
- View-based and modular eigenspaces for face recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1994
- Eigenfaces for RecognitionJournal of Cognitive Neuroscience, 1991
- A survey of decision tree classifier methodologyIEEE Transactions on Systems, Man, and Cybernetics, 1991
- On recognizing and positioning curved 3-D objects from image contoursIeee Transactions On Pattern Analysis and Machine Intelligence, 1990
- Automatic generation of object recognition programsProceedings of the IEEE, 1988