Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems
Top Cited Papers
- 8 October 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 26 (1), 140-151
- https://doi.org/10.1109/tnnls.2014.2358227
Abstract
This paper presents a partially model-free adaptive optimal control solution to the deterministic nonlinear discrete-time (DT) tracking control problem in the presence of input constraints. The tracking error dynamics and reference trajectory dynamics are first combined to form an augmented system. Then, a new discounted performance function based on the augmented system is presented for the optimal nonlinear tracking problem. In contrast to the standard solution, which finds the feedforward and feedback terms of the control input separately, the minimization of the proposed discounted performance function gives both feedback and feedforward parts of the control input simultaneously. This enables us to encode the input constraints into the optimization problem using a nonquadratic performance function. The DT tracking Bellman equation and tracking Hamilton-Jacobi-Bellman (HJB) are derived. An actor-critic-based reinforcement learning algorithm is used to learn the solution to the tracking HJB equation online without requiring knowledge of the system drift dynamics. That is, two neural networks (NNs), namely, actor NN and critic NN, are tuned online and simultaneously to generate the optimal bounded control policy. A simulation example is given to show the effectiveness of the proposed method.Keywords
Funding Information
- National Science Foundation (ECCS-1405173, IIS-1208623)
- U.S. Office of Naval Research, Arlington, VA, USA (N00014-13-1-0562)
- Air Force Office of Scientific Research, Arlington, VA, USA, through the European Office of Aerospace Research and Development Project (13-3055)
- National Natural Science Foundation of China (61120106011)
- 111 Project, Ministry of Education, China (B08015)
This publication has 35 references indexed in Scilit:
- Reinforcement -learning for optimal tracking control of linear discrete-time systems with unknown dynamicsAutomatica, 2014
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systemsAutomatica, 2014
- Optimal tracking control for linear discrete-time systems using reinforcement learningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal GasificationIEEE Transactions on Automation Science and Engineering, 2013
- Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2013
- Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithmNeurocomputing, 2013
- Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming MethodIEEE Transactions on Neural Networks, 2011
- Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamicsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration AlgorithmIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- Nonlinear Control SystemsPublished by Springer Science and Business Media LLC ,1995