LOTUS: An Algorithm for Building Accurate and Comprehensible Logistic Regression Trees
- 1 December 2004
- journal article
- research article
- Published by Taylor & Francis Ltd in Journal of Computational and Graphical Statistics
- Vol. 13 (4), 826-852
- https://doi.org/10.1198/106186004x13064
Abstract
Logistic regression is a powerful technique for fitting models to data with a binary response variable, but the models are difficult to interpret if collinearity, nonlinearity, or interactions are present. Besides, it is hard to judge model adequacy because there are few diagnostics for choosing variable transformations and no true goodness-of-fit test. To overcome these problems, this article proposes to fit a piecewise (multiple or simple) linear logistic regression model by recursively partitioning the data and fitting a different logistic regression in each partition. This allows nonlinear features of the data to be modeled without requiring variable transformations. The binary tree that results from the partitioning process is pruned to minimize a cross-validation estimate of the predicted deviance. This obviates the need for a formal goodness-of-fit test. The resulting model is especially easy to interpret if a simple linear logistic regression is fitted to each partition, because the tree structure and the set of graphs of the fitted functions in the partitions comprise a complete visual description of the model. Trend-adjusted chi-square tests are used to control bias in variable selection at the intermediate nodes. This protects the integrity of inferences drawn from the tree structure. The method is compared with standard stepwise logistic regression on 30 real datasets, with several containing tens to hundreds of thousands of observations. Averaged across the datasets, the results show that the method reduces predicted mean deviance by 9% to 16%.We use an example from the Dutch insurance industry to demonstrate how the method can identify and produce an intelligible profile of prospective customers.Keywords
This publication has 9 references indexed in Scilit:
- SmcHD1, containing a structural-maintenance-of-chromosomes hinge domain, has a critical role in X inactivationNature Genetics, 2008
- 10.1162/153244304322972694Applied Physics Letters, 2000
- A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification AlgorithmsMachine Learning, 2000
- Ethnic wage differences in Malaysia: parametric and semiparametric estimation of the Chinese-Malay wage gapJournal of Applied Econometrics, 1998
- Rule-based Machine Learning Methods for Functional PredictionJournal of Artificial Intelligence Research, 1995
- AI '92Published by World Scientific Pub Co Pte Ltd ,1992
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974
- Tests for Linear Trends in Proportions and FrequenciesBiometrics, 1955
- Some Methods for Strengthening the Common χ 2 TestsBiometrics, 1954