Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems
Top Cited Papers
- 18 August 2016
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 598-617
- https://doi.org/10.1109/sp.2016.42
Abstract
Algorithmic systems that employ machine learning play an increasing role in making substantive decisions in modern society, ranging from online personalization to insurance and credit decisions to predictive policing. But their decision-making processes are often opaque-it is difficult to explain why a certain decision was made. We develop a formal foundation to improve the transparency of such decision-making systems. Specifically, we introduce a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. These measures provide a foundation for the design of transparency reports that accompany system decisions (e.g., explaining a specific credit decision) and for testing tools useful for internal and external oversight (e.g., to detect algorithmic discrimination). Distinctively, our causal QII measures carefully account for correlated inputs while measuring influence. They support a general class of transparency queries and can, in particular, explain decisions about individuals (e.g., a loan decision) and groups (e.g., disparate impact based on gender). Finally, since single inputs may not always have high influence, the QII measures also quantify the joint influence of a set of inputs (e.g., age and income) on outcomes (e.g. loan decisions) and the marginal influence of individual inputs within such a set (e.g., income). Since a single input may be part of multiple influential sets, the average marginal influence of the input is computed using principled aggregation measures, such as the Shapley value, previously applied to measure influence in voting. Further, since transparency reports could compromise privacy, we explore the transparency-privacy tradeoff and prove that a number of useful transparency reports can be made differentially private with very little addition of noise. Our empirical validation with standard machine learning algorithms demonstrates that QII measures are a useful transparency mechanism when black box access to the learning system is available. In particular, they provide better explanations than standard associative measures for a host of scenarios that we consider. Further, we show that in the situations we consider, QII is efficiently approximable and can be made differentially private while preserving accuracy.Keywords
This publication has 36 references indexed in Scilit:
- Centrality in directed social networks. A game theoretic approachSocial Networks, 2011
- Regression Shrinkage and Selection via The Lasso: A RetrospectiveJournal of the Royal Statistical Society Series B: Statistical Methodology, 2011
- Approximating power indices: theoretical and empirical analysisAutonomous Agents and Multi-Agent Systems, 2009
- Causes and Explanations: A Structural-Model Approach. Part I: CausesThe British Journal for the Philosophy of Science, 2005
- Fair Attribution of Functional Contribution in Artificial and Biological NetworksNeural Computation, 2004
- Protein interaction networks from yeast to humanCurrent Opinion in Structural Biology, 2004
- Monotonic solutions of cooperative gamesInternational Journal of Game Theory, 1985
- A new index of power for simplen-person gamesInternational Journal of Game Theory, 1978
- Probability Inequalities for Sums of Bounded Random VariablesJournal of the American Statistical Association, 1963
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951