A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing
Top Cited Papers
- 25 December 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Magazine
- Vol. 33 (1), 57-77
- https://doi.org/10.1109/msp.2015.2481563
Abstract
This article presents a powerful algorithmic framework for big data optimization, called the block successive upper-bound minimization (BSUM). The BSUM includes as special cases many well-known methods for analyzing massive data sets, such as the block coordinate descent (BCD) method, the convex-concave procedure (CCCP) method, the block coordinate proximal gradient (BCPG) method, the nonnegative matrix factorization (NMF) method, the expectation maximization (EM) method, etc. In this article, various features and properties of the BSUM are discussed from the viewpoint of design flexibility, computational efficiency, parallel/distributed implementation, and the required communication overhead. Illustrative examples from networking, signal processing, and machine learning are presented to demonstrate the practical performance of the BSUM framework.Keywords
This publication has 69 references indexed in Scilit:
- Decomposition by Successive Convex Approximation: A Unifying Approach for Linear Transceiver Design in Heterogeneous NetworksIEEE Transactions on Wireless Communications, 2015
- Coordinate descent algorithmsMathematical Programming, 2015
- On the complexity analysis of randomized block-coordinate descent methodsMathematical Programming, 2014
- Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent frameworkJournal of Global Optimization, 2013
- A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth OptimizationSIAM Journal on Optimization, 2013
- Tensor Decompositions and ApplicationsSIAM Review, 2009
- Monotonic convergence of distributed interference pricing in wireless networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- The Concave-Convex ProcedureNeural Computation, 2003
- Convergence of a block coordinate descent method for nondifferentiable minimizationJournal of Optimization Theory and Applications, 2001
- On search directions for minimization algorithmsMathematical Programming, 1973