Calibration of a Confidence Interval for a Classification Accuracy
Open Access
- 1 January 2021
- journal article
- research article
- Published by Scientific Research Publishing, Inc. in Open Journal of Forestry
- Vol. 11 (01), 14-36
- https://doi.org/10.4236/ojf.2021.111002
Abstract
Coverage of nominal 95% confidence intervals of a proportion estimated from a sample obtained under a complex survey design, or a proportion estimated from a ratio of two random variables, can depart significantly from its target. Effective calibration methods exist for intervals for a proportion derived from a single binary study variable, but not for estimates of thematic classification accuracy. To promote a calibration of confidence intervals within the context of land-cover mapping, this study first illustrates a common problem of under and over-coverage with standard confidence intervals, and then proposes a simple and fast calibration that more often than not will improve coverage. The demonstration is with simulated sampling from a classified map with four classes, and a reference class known for every unit in a population of 160,000 units arranged in a square array. The simulations include four common probability sampling designs for accuracy assessment, and three sample sizes. Statistically significant over- and under-coverage was present in estimates of user’s (UA) and producer’s accuracy (PA) as well as in estimates of class area proportion. A calibration with Bayes intervals for UA and PA was most efficient with smaller sample sizes and two cluster sampling designs.Keywords
This publication has 51 references indexed in Scilit:
- Pixels, blocks of pixels, and polygons: Choosing a spatial unit for thematic accuracy assessmentRemote Sensing of Environment, 2011
- Determining sample size in national forest inventories by cost-plus-loss analysis: an exploratory case studyEuropean Journal of Forest Research, 2011
- A stochastic neighborhood conditional autoregressive model for spatial dataComputational Statistics & Data Analysis, 2009
- Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result from Adding Two Successes and Two FailuresThe American Statistician, 2000
- Basic probability sampling designs for thematic map accuracy assessmentInternational Journal of Remote Sensing, 1999
- Technical note Assessing the accuracy of Landsat Thematic Mapper classification using double samplingInternational Journal of Remote Sensing, 1998
- Invited Discussion Paper Resampling Methods in Sample SurveysStatistics, 1996
- A Suggestion for Using Powerful and Informative Tests of NormalityThe American Statistician, 1990
- Resampling Inference with Complex Survey DataJournal of the American Statistical Association, 1988
- Note on the Multivariate and the Generalized Multivariate Beta DistributionsJournal of the American Statistical Association, 1969