Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems
- 28 July 2014
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 44 (12), 2706-2718
- https://doi.org/10.1109/tcyb.2014.2313915
Abstract
The problem of H ∞ state feedback control of affine nonlinear discrete-time systems with unknown dynamics is investigated in this paper. An online adaptive policy learning algorithm (APLA) based on adaptive dynamic programming (ADP) is proposed for learning in real-time the solution to the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in the H ∞ control problem. In the proposed algorithm, three neural networks (NNs) are utilized to find suitable approximations of the optimal value function and the saddle point feedback control and disturbance policies. Novel weight updating laws are given to tune the critic, actor, and disturbance NNs simultaneously by using data generated in real-time along the system trajectories. Considering NN approximation errors, we provide the stability analysis of the proposed algorithm with Lyapunov approach. Moreover, the need of the system input dynamics for the proposed algorithm is relaxed by using a NN identification scheme. Finally, simulation examples show the effectiveness of the proposed algorithm.Keywords
Funding Information
- National Natural Science Foundation of China (61034005, 61104010)
- National High Technology Research and Development Program of China (2012AA040104)
This publication has 47 references indexed in Scilit:
- Stochastic Optimal Controller Design for Uncertain Nonlinear Networked Control System via Neuro Dynamic ProgrammingIEEE Transactions on Neural Networks and Learning Systems, 2013
- A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systemsAutomatica, 2012
- Neural Network Based Online Simultaneous Policy Update Algorithm for Solving the HJI Equation in Nonlinear $H_{\infty}$ ControlIEEE Transactions on Neural Networks and Learning Systems, 2012
- An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential gamesAutomatica, 2010
- Optimal control of affine nonlinear continuous-time systems using an online Hamilton-Jacobi-Isaacs formulationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2008
- Neurodynamic Programming and Zero-Sum Games for Constrained Control SystemsIEEE Transactions on Neural Networks, 2008
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002