On the Performance of Manhattan Nonnegative Matrix Factorization
- 12 August 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 27 (9), 1851-1863
- https://doi.org/10.1109/tnnls.2015.2458986
Abstract
Extracting low-rank and sparse structures from matrices has been extensively studied in machine learning, compressed sensing, and conventional signal processing, and has been widely applied to recommendation systems, image reconstruction, visual analytics, and brain signal processing. Manhattan nonnegative matrix factorization (MahNMF) is an extension of the conventional NMF, which models the heavy-tailed Laplacian noise by minimizing the Manhattan distance between a nonnegative matrix X and the product of two nonnegative low-rank factor matrices. Fast algorithms have been developed to restore the low-rank and sparse structures of X in the MahNMF. In this paper, we study the statistical performance of the MahNMF in the frame of the statistical learning theory. We decompose the expected reconstruction error of the MahNMF into the estimation error and the approximation error. The estimation error is bounded by the generalization error bounds of the MahNMF, while the approximation error is analyzed using the asymptotic results of the minimum distortion of vector quantization. The generalization error bound is valuable for determining the size of the training sample needed to guarantee a desirable upper bound for the defect between the expected and empirical reconstruction errors. Statistical performance analysis shows how the reduced dimensionality affects the estimation and approximation errors. Our framework can also be used for analyzing the performance of the NMF.Keywords
Funding Information
- Australian Research Council (DP-140102164, FT-130101457)
This publication has 47 references indexed in Scilit:
- Clustering XML documents by patternsKnowledge and Information Systems, 2015
- Classification in the Presence of Label Noise: a SurveyIEEE Transactions on Neural Networks and Learning Systems, 2014
- NeNMF: An Optimal Gradient Method for Nonnegative Matrix FactorizationIEEE Transactions on Signal Processing, 2012
- A novel computational framework for simultaneous integration of multiple types of genomic data to identify microRNA-gene regulatory modulesBioinformatics, 2011
- $K$-Dimensional Coding Schemes in Hilbert SpacesIEEE Transactions on Information Theory, 2010
- How do your data grow?Nature, 2008
- Topology Preserving Non-negative Matrix Factorization for Face RecognitionIEEE Transactions on Image Processing, 2008
- The Nature of Statistical Learning TheoryPublished by Springer Science and Business Media LLC ,2000
- On nonnegative factorization of matricesLinear Algebra and its Applications, 1987
- Rank Factorization of Nonnegative Matrices (A. Berman)Siam Review, 1974