Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Top Cited Papers
- 6 December 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 29 (6), 2042-2062
- https://doi.org/10.1109/tnnls.2017.2773458
Abstract
This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal H₂ and H∞$ control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.Keywords
Funding Information
- U.S. NSF (ECCS-1405173)
- ONR (N00014-17-1-2239)
- China NSFC (61633007)
- NATO through the Virginia Tech Startup Fund (SPS G5176)
This publication has 95 references indexed in Scilit:
- A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systemsAutomatica, 2013
- Autonomous Helicopter Aerobatics through Apprenticeship LearningThe International Journal of Robotics Research, 2010
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009
- Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity controlAutomatica, 2007
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Imitation with ALICE: learning to imitate corresponding actions across dissimilar embodimentsIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2002
- L/sub 2/-gain analysis of nonlinear systems and nonlinear state-feedback H/sub infinity / controlIEEE Transactions on Automatic Control, 1992
- State-space solutions to standard H/sub 2/ and H/sub infinity / control problemsIEEE Transactions on Automatic Control, 1989
- Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inversesIEEE Transactions on Automatic Control, 1981