On ${l}_{q}$ Optimization and Sparse Inverse Covariance Selection
- 29 January 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 62 (7), 1644-1654
- https://doi.org/10.1109/tsp.2014.2303429
Abstract
Graphical models are well established in providing meaningful conditional probability descriptions of complex multivariable interactions. In the Gaussian case, the conditional independencies between different variables correspond to zero entries in the precision (inverse covariance) matrix. Hence, there has been much recent interest in sparse precision matrix estimation in areas such as statistics, machine learning, computer vision, pattern recognition, and signal processing. A popular estimation method involves optimizing a penalized log-likelihood problem. The penalty is responsible for inducing sparsity and a common choice is the convex l1 norm. Even though the l0 penalty is the natural choice guaranteeing maximum sparsity, it has been avoided due to lack of convexity. As a result, in this paper we bridge the gap between these two penalties and propose the non-concave lq penalized log-likelihood problem for sparse precision matrix estimation where 0 ≤ q <; 1. A novel algorithm is developed for the optimization and we provide some of its theoretic properties that are useful in sparse linear regression. We illustrate on synthetic and real data, showing reconstruction quality comparisons of sparsity inducing penalties:l0, lq with 0 <; q <; 1, l1, and SCAD.Keywords
This publication has 38 references indexed in Scilit:
- On $l_q$ Optimization and Matrix CompletionIEEE Transactions on Signal Processing, 2012
- Sparsistency and rates of convergence in large covariance matrix estimationThe Annals of Statistics, 2009
- Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC ProgrammingIEEE Transactions on Signal Processing, 2009
- Network exploration via the adaptive LASSO and SCAD penaltiesThe Annals of Applied Statistics, 2009
- Exact Reconstruction of Sparse Signals via Nonconvex MinimizationIEEE Signal Processing Letters, 2007
- Covariance matrix selection and estimation via penalised normal likelihoodBiometrika, 2006
- Analysis of call centre arrival data using singular value decompositionApplied Stochastic Models in Business and Industry, 2005
- Statistical Analysis of a Telephone Call CenterJournal of the American Statistical Association, 2005
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001
- Atomic Decomposition by Basis PursuitSIAM Journal on Scientific Computing, 1998