Optimized Adaptive Nonlinear Tracking Control Using Actor-Critic Reinforcement Learning Strategy
- 31 August 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 15 (9), 4969-4977
- https://doi.org/10.1109/TII.2019.2894282
Abstract
This paper proposes an optimized tracking control approach using neural network (NN) based reinforcement learning (RL) for a class of nonlinear dynamic systems, which requires both tracking and optimizing to be performed simultaneously. Generally, for obtaining optimal control solution, Hamilton-Jacobi-Bellman equation is expected to be solvable, but, owing to strong nonlinearity, the equation is solved difficultly or even impossibly by analytical methods. Therefore, adaptive NN approximation based RL is usually considered. In the optimized control design, for driving output state following to the desired trajectory, an error term is split from optimal performance index function, and then both actor and critic NNs are built to perform RL algorithm. Actor NN aims to execute control behaviors, and critic NN aims to appraise control performance and make feedback to actor. The proof of stability concludes that the desired control performances are obtained. A numerical simulation is designed and implemented, and the desired results are shown.Keywords
Funding Information
- Natural Science Foundation of Shandong Province (ZR2018MF015)
- National Natural Science Foundation of China (61751202, 61572540)
- Binzhou University (2016Y14)
- Shandong University of Science and Technology
This publication has 30 references indexed in Scilit:
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systemsAutomatica, 2014
- Neural‐network‐based online optimal control for uncertain non‐linear continuous‐time systems with control constraintsIET Control Theory & Applications, 2013
- Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraintsInternational Journal of Control, 2013
- Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2013
- Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programmingInternational Journal of Control, 2013
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Global asymptotic stability analysis for integro-differential systems modeling neural networks with delaysZeitschrift für angewandte Mathematik und Physik, 2010
- Adaptive neural network control of nonlinear systems by state and output feedbackIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equationAutomatica, 1997