Use (and abuse) of expert elicitation in support of decision making for public policy
Open Access
- 12 May 2014
- journal article
- Published by Proceedings of the National Academy of Sciences in Proceedings of the National Academy of Sciences of the United States of America
- Vol. 111 (20), 7176-7184
- https://doi.org/10.1073/pnas.1319946111
Abstract
The elicitation of scientific and technical judgments from experts, in the form of subjective probability distributions, can be a valuable addition to other forms of evidence in support of public policy decision making. This paper explores when it is sensible to perform such elicitation and how that can best be done. A number of key issues are discussed, including topics on which there are, and are not, experts who have knowledge that provides a basis for making informed predictive judgments; the inadequacy of only using qualitative uncertainty language; the role of cognitive heuristics and of overconfidence; the choice of experts; the development, refinement, and iterative testing of elicitation protocols that are designed to help experts to consider systematically all relevant knowledge when they make their judgments; the treatment of uncertainty about model functional form; diversity of expert opinion; and when it does or does not make sense to combine judgments from different experts. Although it may be tempting to view expert elicitation as a low-cost, low-effort alternative to conducting serious research and analysis, it is neither. Rather, expert elicitation should build on and use the best available research and analysis and be undertaken only when, given those, the state of knowledge will remain insufficient to support timely informed assessment and decision making.Keywords
This publication has 60 references indexed in Scilit:
- The future costs of nuclear power using multiple expert elicitations: effects of RD&D and elicitation designEnvironmental Research Letters, 2013
- Expert judgments about transient climate response to alternative future trajectories of radiative forcingProceedings of the National Academy of Sciences of the United States of America, 2010
- The use of expert elicitation in environmental health impact assessment: a seven step procedureEnvironmental Health, 2010
- Expert elicitation on ultrafine particles: likelihood of health effects and causal pathwaysParticle and Fibre Toxicology, 2009
- Expert judgements on the response of the Atlantic meridional overturning circulation to climate changeClimatic Change, 2007
- Evaluation of potential cost reductions from improved amine-based CO2 capture systemsEnergy Policy, 2006
- Elicitation of Expert Judgments of Aerosol ForcingClimatic Change, 2006
- A model framework for integrated studies of the climate problemEnergy Policy, 1993
- Sulfur Control In Coal Fired Power Plants: A Probabilistic Approach to Policy AnalysisJournal of the Air Pollution Control Association, 1978
- Judgment under Uncertainty: Heuristics and BiasesScience, 1974