Representation Learning via Cauchy Convolutional Sparse Coding
Open Access
- 12 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 100447-100459
- https://doi.org/10.1109/access.2021.3096643
Abstract
In representation learning, Convolutional Sparse Coding (CSC) enables unsupervised learning of features by jointly optimising both an $\ell _{2}$ -norm fidelity term and a sparsity enforcing penalty. This work investigates using a regularisation term derived from an assumed Cauchy prior for the coefficients of the feature maps of a CSC generative model. The sparsity penalty term resulting from this prior is solved via its proximal operator, which is then applied iteratively, element-wise, on the coefficients of the feature maps to optimise the CSC cost function. The performance of the proposed Iterative Cauchy Thresholding (ICT) algorithm in reconstructing natural images is compared against algorithms based on minimising standard penalty functions via soft and hard thresholding as well as against the Iterative Log-Thresholding (ILT) method. ICT outperforms the Iterative Hard Thresholding (IHT), Iterative Soft Thresholding (IST), and ILT algorithms in most of our reconstruction experiments across various datasets, with an average Peak Signal to Noise Ratio (PSNR) of up to 11.30 dB, 7.04 dB, and 7.74 dB over IST, IHT, and ILT respectively. The source code for the implementation of the proposed approach is publicly available at https://github.com/p-mayo/cauchycscFunding Information
- Consejo Nacional de Ciencia y Tecnología (CONACyT) PhD studentship (461322 (to Mayo))
- Engineering and Physical Sciences Research Council (EP/R009260/1 (AssenSAR))
- Leverhulme Trust Research Fellowship
This publication has 39 references indexed in Scilit:
- Accelerated iterative hard thresholdingSignal Processing, 2012
- Segmentation of noisy colour images using Cauchy distribution in the complex wavelet domainIET Image Processing, 2011
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Compression of facial images using the K-SVD algorithmJournal of Visual Communication and Image Representation, 2008
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse RepresentationIEEE Transactions on Signal Processing, 2006
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraintCommunications on Pure and Applied Mathematics, 2004
- Novel Bayesian multiscale method for speckle removal in medical ultrasound imagesIEEE Transactions on Medical Imaging, 2001
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997
- An Information-Maximization Approach to Blind Separation and Blind DeconvolutionNeural Computation, 1995
- Principal component analysisChemometrics and Intelligent Laboratory Systems, 1987