Deep multi-agent Reinforcement Learning for cost-efficient distributed load frequency control
Open Access
- 1 September 2021
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Energy Systems Integration
- Vol. 3 (3), 327-343
- https://doi.org/10.1049/esi2.12030
Abstract
The rise of microgrid-based architectures is modifying significantly the energy control landscape in distribution systems, making distributed control mechanisms necessary to ensure reliable power system operations. In this article, the use of Reinforcement Learning techniques is proposed to implement load frequency control (LFC) without requiring a central authority. To this end, a detailed model of power system dynamic behaviour is formulated by representing individual generator dynamics, generator rate and network constraints, renewable-based generation, and realistic load realisations. The LFC problem is recast as a Markov Decision Process, and the Multi-Agent Deep Deterministic Policy Gradient algorithm is used to approximate the optimal solution of all LFC layers, that is, primary, secondary and tertiary. The proposed LFC framework operates through centralised learning and distributed implementation. In particular, there is no information interchange between generating units during operation. Thus, no communication infrastructure is necessary and information privacy between them is respected. The proposed framework is validated through numerical results and it is shown that it can be used to implement LFC in a distributed and cost-efficient manner.This publication has 38 references indexed in Scilit:
- Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed AveragingIEEE Transactions on Industrial Electronics, 2015
- Distributed Tertiary Control of DC Microgrid ClustersIEEE Transactions on Power Electronics, 2015
- A Review of Power Electronics Based MicrogridsJournal of Power Electronics, 2012
- Load–frequency control: a GA-based multi-agent reinforcement learningIET Generation, Transmission & Distribution, 2010
- A reinforcement learning approach to automatic generation controlElectric Power Systems Research, 2002
- Long Short-Term MemoryNeural Computation, 1997
- Software agents: an overviewThe Knowledge Engineering Review, 1996
- Markov games as a framework for multi-agent reinforcement learningPublished by Elsevier BV ,1994
- Q-learningMachine Learning, 1992
- Automatic generation controlInternational Journal of Electrical Power & Energy Systems, 1980