Fixed Rank Filtering for Spatio-Temporal Data
- 1 January 2010
- journal article
- Published by Taylor & Francis Ltd in Journal of Computational and Graphical Statistics
- Vol. 19 (3), 724-745
- https://doi.org/10.1198/jcgs.2010.09051
Abstract
Datasets from remote-sensing platforms and sensor networks are often spatial, temporal, and very large. Processing massive amounts of data to provide current estimates of the (hidden) state from current and past data is challenging, even for the Kalman filter. A large number of spatial locations observed through time can quickly lead to an overwhelmingly high-dimensional statistical model. Dimension reduction without sacrificing complexity is our goal in this article. We demonstrate how a Spatio-Temporal Random Effects (STRE) component of a statistical model reduces the problem to one of fixed dimension with a very fast statistical solution, a methodology we call Fixed Rank Filtering (FRF). This is compared in a simulation experiment to successive, spatial-only predictions based on an analogous Spatial Random Effects (SRE) model, and the value of incorporating temporal dependence is quantified. A remote-sensing dataset of aerosol optical depth (AOD), from the Multi-angle Imaging SpectroRadiometer (MISR) instrument on the Terra satellite, is used to compare spatio-temporal FRF with spatial-only prediction. FRF achieves rapid production of optimally filtered AOD predictions, along with their prediction standard errors. In our case, over 100,000 spatio-temporal data were processed: Parameter estimation took 64.4 seconds and optimal predictions and their standard errors took 77.3 seconds to compute. Supplemental materials giving complete details on the design and analysis of a simulation experiment, the simulation code, and the MISR data used are available on-line.Keywords
This publication has 21 references indexed in Scilit:
- Spatio-Temporal Analysis of Total Nitrate Concentrations Using Dynamic Statistical ModelsJournal of the American Statistical Association, 2010
- A Spatio-Temporal Downscaler for Output From Numerical ModelsJournal of Agricultural, Biological and Environmental Statistics, 2010
- Gaussian Predictive Process Models for Large Spatial Data SetsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Fixed Rank Kriging for Very Large Spatial Data SetsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Dynamic multi-resolution spatial modelsEnvironmental and Ecological Statistics, 2007
- Fast, Resolution-Consistent Spatial Prediction of Global Processes From Satellite DataJournal of Computational and Graphical Statistics, 2002
- New Directions in Earth Observing: Scientific Applications ofMultiangle Remote SensingBulletin of the American Meteorological Society, 1999
- Spatio-temporal prediction of snow water equivalent using the Kalman filterComputational Statistics & Data Analysis, 1996
- On Deriving the Inverse of a Sum of MatricesSIAM Review, 1981
- A New Approach to Linear Filtering and Prediction ProblemsJournal of Basic Engineering, 1960