Kullback–Leibler Divergence as a Forecast Skill Score with Classic Reliability–Resolution–Uncertainty Decomposition
- 1 September 2010
- journal article
- Published by American Meteorological Society in Monthly Weather Review
- Vol. 138 (9), 3387-3399
- https://doi.org/10.1175/2010mwr3229.1
Abstract
This paper presents a score that can be used for evaluating probabilistic forecasts of multicategory events. The score is a reinterpretation of the logarithmic score or ignorance score, now formulated as the relative entropy or Kullback–Leibler divergence of the forecast distribution from the observation distribution. Using the information–theoretical concepts of entropy and relative entropy, a decomposition into three components is presented, analogous to the classic decomposition of the Brier score. The information–theoretical twins of the components uncertainty, resolution, and reliability provide diagnostic information about the quality of forecasts. The overall score measures the information conveyed by the forecast. As was shown recently, information theory provides a sound framework for forecast verification. The new decomposition, which has proven to be very useful for the Brier score and is widely used, can help acceptance of the logarithmic score in meteorology.Keywords
This publication has 17 references indexed in Scilit:
- Scoring Rules for Forecast VerificationMonthly Weather Review, 2010
- Proper Scores for Probability Forecasts Can Never Be EquitableMonthly Weather Review, 2008
- Information-Based Skill Scores for Probabilistic ForecastsMonthly Weather Review, 2008
- Verification tools for probabilistic forecasts of continuous hydrological variablesHydrology and Earth System Sciences, 2007
- Measuring Dynamical Prediction Utility Using Relative EntropyJournal of the Atmospheric Sciences, 2002
- Information Theory and Climate PredictionJournal of Climate, 1990
- A Scoring System for Probability Forecasts of Ranked CategoriesJournal of Applied Meteorology, 1969
- A new interpretation of information rateIEEE Transactions on Information Theory, 1956
- Rational DecisionsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1952
- VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITYMonthly Weather Review, 1950