Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions
- 26 May 2015
- journal article
- research article
- Published by Wiley in Statistics in Medicine
- Vol. 34 (23), 3133-3143
- https://doi.org/10.1002/sim.6537
Abstract
Penalization is a very general method of stabilizing or regularizing estimates, which has both frequentist and Bayesian rationales. We consider some questions that arise when considering alternative penalties for logistic regression and related models. The most widely programmed penalty appears to be the Firth small-sample bias-reduction method (albeit with small differences among implementations and the results they provide), which corresponds to using the log density of the Jeffreys invariant prior distribution as a penalty function. The latter representation raises some serious contextual objections to the Firth reduction, which also apply to alternative penalties based on t-distributions (including Cauchy priors). Taking simplicity of implementation and interpretation as our chief criteria, we propose that the log-F(1,1) prior provides a better default penalty than other proposals. Penalization based on more general log-F priors is trivial to implement and facilitates mean-squared error reduction and sensitivity analyses of penalty strength by varying the number of prior degrees of freedom. We caution however against penalization of intercepts, which are unduly sensitive to covariate coding and design idiosyncrasies. Copyright © 2015 John Wiley & Sons, Ltd.Keywords
This publication has 31 references indexed in Scilit:
- Sensitivity Analyses for Sparse-Data Problems—Using Weakly Informative Bayesian PriorsEpidemiology, 2013
- Bayesian regression in SAS softwareInternational Journal of Epidemiology, 2012
- Simpson’s Paradox From Adding Constants in Contingency Tables as an Example of Bayesian NoncollapsibilityThe American Statistician, 2010
- Bias‐reduced and separation‐proof conditional logistic regression with small or sparse data setsStatistics in Medicine, 2010
- The formal definition of reference priorsThe Annals of Statistics, 2009
- Properties and Implementation of Jeffreys’s Prior in Binomial Regression ModelsJournal of the American Statistical Association, 2008
- Prior data for non‐normal priorsStatistics in Medicine, 2007
- Model-based Estimation of Relative Risks and Other Epidemiologic Measures in Studies of Common Outcomes and in Case-Control StudiesAmerican Journal of Epidemiology, 2004
- The Bayesian Analysis of Contingency TablesThe Annals of Mathematical Statistics, 1964