Optimal Demand Response Using Device-Based Reinforcement Learning
- 16 February 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 6 (5), 2312-2324
- https://doi.org/10.1109/tsg.2015.2396993
Abstract
Demand response (DR) for residential and small commercial buildings is estimated to account for as much as 65% of the total energy savings potential of DR, and previous work shows that a fully automated energy management system (EMS) is a necessary prerequisite to DR in these areas. In this paper, we propose a novel EMS formulation for DR problems in these sectors. Specifically, we formulate a fully automated EMS's rescheduling problem as a reinforcement learning (RL) problem, and argue that this RL problem can be approximately solved by decomposing it over device clusters. Compared with existing formulations, our new formulation does not require explicitly modeling the user's dissatisfaction on job rescheduling, enables the EMS to self-initiate jobs, allows the user to initiate more flexible requests, and has a computational complexity linear in the number of device clusters. We also demonstrate the simulation results of applying Q-learning, one of the most popular and classical RL algorithms, to a representative example.Keywords
Other Versions
This publication has 14 references indexed in Scilit:
- Demand response of a heterogeneous cluster of electric water heaters using batch reinforcement learningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Approximate Dynamic Programming for Optimizing Oil ProductionPublished by Wiley ,2012
- Using smart devices for system-level management and control in the smart grid: A reinforcement learning frameworkPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Smart finite state devices: A modeling framework for demand response technologiesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Use of Approximate Dynamic Programming for Production OptimizationPublished by Society of Petroleum Engineers (SPE) ,2011
- An Integrated Architecture for Demand Response Communications and ControlPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- Residential implementation of critical-peak pricing of electricityEnergy Policy, 2007
- An exploratory analysis of California residential customer response to critical peak pricing of electricityEnergy, 2006
- Quantifying Customer Response to Dynamic PricingThe Electricity Journal, 2005
- Industrial power demand response analysis for one-part real-time pricingIEEE Transactions on Power Systems, 1998