"Why Should I Trust You?"
Top Cited Papers
- 13 August 2016
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM) in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Abstract
Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally varound the prediction. We also propose a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem. We demonstrate the flexibility of these methods by explaining different models for text (e.g. random forests) and image classification (e.g. neural networks). We show the utility of explanations via novel experiments, both simulated and with human subjects, on various scenarios that require trust: deciding if one should trust a prediction, choosing between models, improving an untrustworthy classifier, and identifying why a classifier should not be trusted.Keywords
Funding Information
- Defense Sciences Office, DARPA
This publication has 18 references indexed in Scilit:
- Intelligible Models for HealthCarePublished by Association for Computing Machinery (ACM) ,2015
- Deep visual-semantic alignments for generating image descriptionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Principles of Explanatory Debugging to Personalize Interactive Machine LearningPublished by Association for Computing Machinery (ACM) ,2015
- Towards Transparent Systems: Semantic Characterization of Failure ModesLecture Notes in Computer Science, 2014
- You Are the Only Possible Oracle: Effective Test Selection for End Users of Interactive Machine Learning SystemsIEEE Transactions on Software Engineering, 2013
- Leakage in data miningPublished by Association for Computing Machinery (ACM) ,2011
- Least angle regressionThe Annals of Statistics, 2004
- The role of trust in automation relianceInternational Journal of Human-Computer Studies, 2003
- Explaining collaborative filtering recommendationsPublished by Association for Computing Machinery (ACM) ,2000
- A threshold of ln n for approximating set coverJournal of the ACM, 1998