Dimensionality reduction by unsupervised regression
- 1 June 2008
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2008 IEEE Conference on Computer Vision and Pattern Recognition
Abstract
We consider the problem of dimensionality reduction, where given high-dimensional data we want to estimate two mappings: from high to low dimension (dimensionality reduction) and from low to high dimension (reconstruction). We adopt an unsupervised regression point of view by introducing the unknown low-dimensional coordinates of the data as parameters, and formulate a regularised objective functional of the mappings and low-dimensional coordinates. Alternating minimisation of this functional is straightforward: for fixed low-dimensional coordinates, the mappings have a unique solution; and for fixed mappings, the coordinates can be obtained by finite-dimensional non-linear minimisation. Besides, the coordinates can be initialised to the output of a spectral method such as Laplacian eigenmaps. The model generalises PCA and several recent methods that learn one of the two mappings but not both; and, unlike spectral methods, our model provides out-of-sample mappings by construction. Experiments with toy and real-world problems show that the model is able to learn mappings for convoluted manifolds, avoiding bad local optima that plague other methods.Keywords
This publication has 10 references indexed in Scilit:
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Local distance preservation in the GP-LVM through back constraintsPublished by Association for Computing Machinery (ACM) ,2006
- Learning Appearance Manifolds from VideoPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Principal surfaces from unsupervised kernel regressionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Laplacian Eigenmaps for Dimensionality Reduction and Data RepresentationNeural Computation, 2003
- A Global Geometric Framework for Nonlinear Dimensionality ReductionScience, 2000
- 10.1162/15324430152748227Applied Physics Letters, 2000
- Numerical OptimizationPublished by Springer Science and Business Media LLC ,1999
- GTM: The Generative Topographic MappingNeural Computation, 1998
- A mathematical analysis of the motion coherence theoryInternational Journal of Computer Vision, 1989