Effects of data cleansing on load prediction algorithms
- 1 April 2013
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
The rollout of advanced metering infrastructure that is planned in many countries worldwide will lead to a massive inflow of data from moderately reliable sensory equipment. In principle, this will make intelligent and automated planning and operation possible at an increasingly finer scale in the electric grid. However, errors can creep into the meter data, either from faulty sensors or during transmission from the meters to the database. This work studies the role of data cleansing as a preprocessing step for short-term (24-hour) power load prediction. We focus on cleansing and prediction at several levels of granularity, from the transmission level via distribution substations down to single households. We believe that preprocessing filters such as cleansing should lead to more robustness and/or precision in the subsequent processing step. However, load cleansing frameworks tend to make the popular assumption of normally and independently distributed noise in the time series. We show that this is incorrect at the diurnal level, due to the characteristic pattern of power consumption, with two peak loads during daytime and a nighttime trough. Moreover, we present empirical evidence that a preprocessing step based on this assumption fails to contribute positively to the performance of the subsequent prediction step. To rectify this problem, we suggest to subtract the average power load consumption in a given period before cleansing. We present empirical evidence that this improves the robustness and efficiency of load cleansing as a preprocessing step. Data cleansing and load prediction is performed by a system that searches out parameters using an evolutionary approach.Keywords
This publication has 17 references indexed in Scilit:
- Robust Nonparametric Regression via Sparsity Control With Application to Load Curve Data CleansingIEEE Transactions on Signal Processing, 2011
- Short Term Load Forecasting using Echo State NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Construction of Optimal Prediction Intervals for Load Forecasting ProblemsIEEE Transactions on Power Systems, 2010
- Results of monitoring of AMR systems in Norway: analysis of metered data and definition of the performance parametersPublished by Institution of Engineering and Technology (IET) ,2009
- An hourly periodic state space model for modelling French national electricity loadInternational Journal of Forecasting, 2008
- Optimization and applications of echo state networks with leaky- integrator neuronsNeural Networks, 2007
- Automatic speech recognition using a predictive echo state network classifierNeural Networks, 2007
- Wavelet-based nonlinear multiscale decomposition model for electricity load forecastingNeurocomputing, 2006
- Functional Data AnalysisPublished by Wiley ,2005
- Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless CommunicationScience, 2004