Convolutional Dictionary Learning: Acceleration and Convergence
- 9 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 27 (4), 1697-1712
- https://doi.org/10.1109/tip.2017.2761545
Abstract
Convolutional dictionary learning (CDL or sparsifying CDL) has many applications in image processing and computer vision. There has been growing interest in developing efficient algorithms for CDL, mostly relying on the augmented Lagrangian (AL) method or the variant alternating direction method of multipliers (ADMM). When their parameters are properly tuned, AL methods have shown fast convergence in CDL. However, the parameter tuning process is not trivial due to its data dependence and, in practice, the convergence of AL methods depends on the AL parameters for nonconvex CDL problems. To moderate these problems, this paper proposes a new practically feasible and convergent Block Proximal Gradient method using a Majorizer (BPG-M) for CDL. The BPG-Mbased CDL is investigated with different block updating schemes and majorization matrix designs, and further accelerated by incorporating some momentum coefficient formulas and restarting techniques. All of the methods investigated incorporate a boundary artifacts removal (or, more generally, sampling) operator in the learning model. Numerical experiments show that, without needing any parameter tuning process, the proposed BPG-M approach converges more stably to desirable solutions of lower objective values than the existing state-of-the-art ADMM algorithm and its memory-efficient variant do. Compared to the ADMM approaches, the BPG-M method using a multi-block updating scheme is particularly useful in single-threaded CDL algorithm handling large datasets, due to its lower memory requirement and no polynomial computational complexity. Image denoising experiments show that, for relatively strong additive white Gaussian noise, the filters learned by BPG-M-based CDL outperform those trained by the ADMM approach.Keywords
Funding Information
- Keck Foundation
- UM-SJTU seed grant
This publication has 39 references indexed in Scilit:
- Iteration complexity of randomized block-coordinate descent methods for minimizing a composite functionMathematical Programming, 2012
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends® in Machine Learning, 2010
- From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and ImagesSiam Review, 2009
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Realistic CT simulation using the 4D XCAT phantomMedical Physics, 2008
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse RepresentationIEEE Transactions on Signal Processing, 2006
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997
- Emergence of simple-cell receptive field properties by learning a sparse code for natural imagesNature, 1996
- On complete-data spaces for PET reconstruction algorithmsIEEE Transactions on Nuclear Science, 1993