Modified Principal Component Analysis: An Integration of Multiple Similarity Subspace Models
- 6 January 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 25 (8), 1538-1552
- https://doi.org/10.1109/tnnls.2013.2294492
Abstract
We modify the conventional principal component analysis (PCA) and propose a novel subspace learning framework, modified PCA (MPCA), using multiple similarity measurements. MPCA computes three similarity matrices exploiting the similarity measurements: 1) mutual information; 2) angle information; and 3) Gaussian kernel similarity. We employ the eigenvectors of similarity matrices to produce new subspaces, referred to as similarity subspaces. A new integrated similarity subspace is then generated using a novel feature selection approach. This approach needs to construct a kind of vector set, termed weak machine cell (WMC), which contains an appropriate number of the eigenvectors spanning the similarity subspaces. Combining the wrapper method and the forward selection scheme, MPCA selects a WMC at a time that has a powerful discriminative capability to classify samples. MPCA is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality. MPCA outperforms the other state-of-the-art PCA-based methods in terms of both classification accuracy and clustering result. In addition, MPCA can be applied to face image reconstruction. MPCA can use other types of similarity measurements. Extensive experiments on many popular real-world data sets, such as face databases, show that MPCA achieves desirable classification results, as well as has a powerful capability to represent data.Keywords
Funding Information
- 973 Program (2014CB347600)
- National Science Fund for Distinguished Young Scholars (61125305, 61233011)
- National Science Foundation of China (61263032, 61071179, 71262011, 61020106004, 60902099, 61362031, 61332011, 61203376)
- Shenzhen Municipal Science and Technology Innovation Council (JC 201005260122A, JCYJ20120613153352732)
- Jiangxi Provincial Science and Technology Foundation of China (KJLD12067)
This publication has 55 references indexed in Scilit:
- Linear Subspace Learning-Based Dimensionality ReductionIEEE Signal Processing Magazine, 2011
- Graph Regularized Nonnegative Matrix Factorization for Data RepresentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- PRINCIPAL MANIFOLDS AND GRAPHS IN PRACTICE: FROM MOLECULAR BIOLOGY TO DYNAMICAL SYSTEMSInternational Journal of Neural Systems, 2010
- Boosting random subspace methodNeural Networks, 2008
- Maximization of Mutual Information for Supervised Linear Feature ExtractionIEEE Transactions on Neural Networks, 2007
- Survey of Clustering AlgorithmsIEEE Transactions on Neural Networks, 2005
- Robust linear dimensionality reductionIEEE Transactions on Visualization and Computer Graphics, 2004
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997
- LIII. On lines and planes of closest fit to systems of points in spaceThe London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 1901