Strategies for Evaluating Quality Assurance Procedures
Open Access
- 1 June 2008
- journal article
- Published by American Meteorological Society in Journal of Applied Meteorology and Climatology
- Vol. 47 (6), 1785-1791
- https://doi.org/10.1175/2007jamc1706.1
Abstract
The evaluation strategies outlined in this paper constitute a set of tools beneficial to the development and documentation of robust automated quality assurance (QA) procedures. Traditionally, thresholds for the QA of climate data have been based on target flag rates or statistical confidence limits. However, these approaches do not necessarily quantify a procedure’s effectiveness at detecting true errors in the data. Rather, as illustrated by way of an “extremes check” for daily precipitation totals, information on the performance of a QA test is best obtained through a systematic manual inspection of samples of flagged values combined with a careful analysis of geographical and seasonal patterns of flagged observations. Such an evaluation process not only helps to document the effectiveness of each individual test, but, when applied repeatedly throughout the development process, it also aids in choosing the optimal combination of QA procedures and associated thresholds. In addition, the approach described here constitutes a mechanism for reassessing system performance whenever revisions are made following initial development.Keywords
This publication has 15 references indexed in Scilit:
- Overview of the Integrated Global Radiosonde ArchiveJournal of Climate, 2006
- Quality Control of Pre-1948 Cooperative Observer Network DataJournal of Atmospheric and Oceanic Technology, 2005
- Performance of Quality Assurance Procedures for an Applied Climate Information SystemJournal of Atmospheric and Oceanic Technology, 2005
- Improved Quality Assurance for Historical Hourly Temperature and Humidity: Development and Application to Environmental AnalysisJournal of Applied Meteorology and Climatology, 2004
- Complex Quality Assurance of Historical Hourly Surface Airways Meteorological DataJournal of Atmospheric and Oceanic Technology, 2004
- The Impact of Unique Meteorological Phenomena Detected by the Oklahoma Mesonet and ARS Micronet on Automated Quality ControlBulletin of the American Meteorological Society, 2001
- The Operational Complex Quality Control of Radiosonde Heights and Temperatures at the National Centers for Environmental Prediction. Part I: Description of the MethodJournal of Applied Meteorology and Climatology, 2001
- An Expanded Digital Daily Database for Climatic Resources Applications in the Midwestern United StatesBulletin of the American Meteorological Society, 1998
- In Situ Meteorological Sounding Archives for Arctic StudiesBulletin of the American Meteorological Society, 1992
- Measuring the Performance of Data ValidatorsBulletin of the American Meteorological Society, 1988