Accurate intelligible models with pairwise interactions
Top Cited Papers
- 11 August 2013
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 623-631
- https://doi.org/10.1145/2487575.2487579
Abstract
Standard generalized additive models (GAMs) usually model the dependent variable as a sum of univariate models. Although previous studies have shown that standard GAMs can be interpreted by users, their accuracy is significantly less than more complex models that permit interactions. In this paper, we suggest adding selected terms of interacting pairs of features to standard GAMs. The resulting models, which we call GA2{M}$-models, for Generalized Additive Models plus Interactions, consist of univariate terms and a small number of pairwise interaction terms. Since these models only include one- and two-dimensional components, the components of GA2M-models can be visualized and interpreted by users. To explore the huge (quadratic) number of pairs of features, we develop a novel, computationally efficient method called FAST for ranking all possible pairs of features as candidates for inclusion into the model. In a large-scale empirical study, we show the effectiveness of FAST in ranking candidate pairs of features. In addition, we show the surprising result that GA2M-models have almost the same performance as the best full-complexity models on a number of real datasets. Thus this paper postulates that for many problems, GA2M-models can yield models that are both intelligible and accurate.Keywords
This publication has 9 references indexed in Scilit:
- Intelligible models for classification and regressionPublished by Association for Computing Machinery (ACM) ,2012
- Predictive learning via rule ensemblesThe Annals of Applied Statistics, 2008
- Detecting statistical interactions with additive groves of treesPublished by Association for Computing Machinery (ACM) ,2008
- Additive Groves of Regression TreesLecture Notes in Computer Science, 2007
- Generalized Functional ANOVA Diagnostics for High-Dimensional Functions of Dependent VariablesJournal of Computational and Graphical Statistics, 2007
- Discovering additive structure in black box functionsPublished by Association for Computing Machinery (ACM) ,2004
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001
- An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and VariantsMachine Learning, 1999
- Sparse spatial autoregressionsStatistics & Probability Letters, 1997