Regularization of Wavelet Approximations
- 1 September 2001
- journal article
- Published by Informa UK Limited in Journal of the American Statistical Association
- Vol. 96 (455), 939-967
- https://doi.org/10.1198/016214501753208942
Abstract
In this paper, we introduce nonlinear regularized wavelet estimators for estimating nonparametric regression functions when sampling points are not uniformly spaced. The approach can apply readily to many other statistical contexts. Various new penalty functions are proposed. The hard-thresholding and soft-thresholding estimators of Donoho and Johnstone are specific members of nonlinear regularized wavelet estimators. They correspond to the lower and upper envelopes of a class of the penalized least squares estimators. Necessary conditions for penalty functions are given for regularized estimators to possess thresholding properties. Oracle inequalities and universal thresholding parameters are obtained for a large class of penalty functions. The sampling properties of nonlinear regularized wavelet estimators are established and are shown to be adaptively minimax. To efficiently solve penalized least squares problems, nonlinear regularized Sobolev interpolators (NRSI) are proposed as initial estimators, which are shown to have good sampling properties. The NRSI is further ameliorated by regularized one-step estimators, which are the one-step estimators of the penalized least squares problems using the NRSI as initial estimators. The graduated nonconvexity algorithm is also introduced to handle penalized least squares problems. The newly introduced approaches are illustrated by a few numerical examples.Keywords
This publication has 52 references indexed in Scilit:
- Wavelet domain image restoration with adaptive edge-preserving regularizationIEEE Transactions on Image Processing, 2000
- Multiple shrinkage and subset selection in waveletsBiometrika, 1998
- Wavelet-based statistical signal processing using hidden Markov modelsIEEE Transactions on Signal Processing, 1998
- Wavelet decomposition approaches to statistical inverse problemsBiometrika, 1998
- Random design wavelet curve smoothingStatistics & Probability Letters, 1997
- On discontinuity-adaptive smoothness priors in computer visionIeee Transactions On Pattern Analysis and Machine Intelligence, 1995
- Formulae for Mean Integrated Squared Error of Nonlinear Wavelet-Based Density EstimatorsThe Annals of Statistics, 1995
- An algorithm for the minimization of mixed l/sub 1/ and l/sub 2/ norms with application to Bayesian estimationIEEE Transactions on Signal Processing, 1994
- A generalized Gaussian image model for edge-preserving MAP estimationIEEE Transactions on Image Processing, 1993
- Comparison of the efficiency of deterministic and stochastic algorithms for visual reconstructionIeee Transactions On Pattern Analysis and Machine Intelligence, 1989