Reinforcement-Learning Approach Guidelines for Energy Management
- 1 September 2019
- journal article
- research article
- Published by American Scientific Publishers in Journal of Low Power Electronics
- Vol. 15 (3), 283-293
- https://doi.org/10.1166/jolpe.2019.1618
Abstract
IoT and autonomous systems are in charge of an increasing number of sensing, processing and communications tasks. These systems may be equipped with energy harvesting devices. Nevertheless, the energy harvested is uncertain and variable, which makes it difficult to manage the energy in these systems. Reinforcement learning algorithms can handle such uncertainties, however selecting the adapted algorithm is a difficult problem. Many algorithms are available and each has its own advantages and drawbacks. In this paper, we try to provide an overview of different approaches to help designer to determine the most appropriate algorithm according to its application and system. We focus on Q-learning, a popular reinforcement learning algorithm and several of these variants. The approach of Q-learning is based on the use of look up table, however some algorithms use a neural network approach. We compare different variants of Q-learning for the energy management of a sensor node. We show that depending on the desired performance and the constraints inherent in the application of the node, the appropriate approach changes.Keywords
This publication has 9 references indexed in Scilit:
- DQELR: An Adaptive Deep Q-Network-Based Energy- and Latency-Aware Routing Protocol Design for Underwater Acoustic Sensor NetworksIEEE Access, 2019
- Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement LearningACM Transactions on Embedded Computing Systems, 2017
- A New Energy Prediction Algorithm for Energy-Harvesting Wireless Sensor Networks With Q-LearningIEEE Access, 2016
- Analysis of PKF: A Communication Cost Reduction Scheme for Wireless Sensor NetworksIEEE Transactions on Wireless Communications, 2015
- A detailed modeling of photovoltaic module using MATLABNRIAG Journal of Astronomy and Geophysics, 2014
- Performance and degradation analysis for long term reliability of solar photovoltaic systems: A reviewRenewable and Sustainable Energy Reviews, 2013
- Q-Learning Based Energy Management Policies for a Single Sensor Node with Finite BufferIEEE Wireless Communications Letters, 2012
- Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio NetworksIEEE Transactions on Vehicular Technology, 2010
- Calcul multi-échelle du rayonnement solaire pour la modélisation prédictive de la végétationAnnals of Forest Science, 2007