Penalized regressions: The bridge versus the lasso

Abstract

Bridge regression, a special family of penalized regressions of a penalty function Sigma \beta(j)\(gamma) with gamma greater than or equal to 1, is considered. A general approach to solve for the bridge estimator is developed. A new algorithm for the lasso (gamma = 1) is obtained by studying the structure of the bridge estimators. The shrinkage parameter gamma and the tuning parameter lambda are selected via generalized cross-validation (GCV). Comparison between the bridge model (gamma greater than or equal to 1) and several other shrinkage models, namely the ordinary least squares regression (lambda = 0), the lasso (gamma = 1) and ridge regression (gamma = 2), is made through a simulation study. It is shown that the bridge regression performs well compared to the lasso and ridge regression. These methods are demonstrated through an analysis of a prostate cancer data. Some computational advantages and limitations are discussed.