Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems
Top Cited Papers
- 2 November 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 46 (3), 840-853
- https://doi.org/10.1109/tcyb.2015.2492242
Abstract
In this paper, a value iteration adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon undiscounted optimal control problems for discrete-time nonlinear systems. The present value iteration ADP algorithm permits an arbitrary positive semi-definite function to initialize the algorithm. A novel convergence analysis is developed to guarantee that the iterative value function converges to the optimal performance index function. Initialized by different initial functions, it is proven that the iterative value function will be monotonically nonincreasing, monotonically nondecreasing, or nonmonotonic and will converge to the optimum. In this paper, for the first time, the admissibility properties of the iterative control laws are developed for value iteration algorithms. It is emphasized that new termination criteria are established to guarantee the effectiveness of the iterative control laws. Neural networks are used to approximate the iterative value function and compute the iterative control law, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.Keywords
Funding Information
- National Natural Science Foundation of China (61533017, 61273140, 61374105, 61233001)
This publication has 62 references indexed in Scilit:
- A Novel Iterative $\theta $-Adaptive Dynamic Programming for Discrete-Time Nonlinear SystemsIEEE Transactions on Automation Science and Engineering, 2013
- Numerical adaptive learning control scheme for discrete‐time non‐linear systemsIET Control Theory & Applications, 2013
- A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systemsAutomatica, 2013
- Dual iterative adaptive dynamic programming for a class of discrete-time nonlinear systems with time-delaysNeural Computing & Applications, 2012
- An iterative -optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial stateNeural Networks, 2012
- An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential gamesAutomatica, 2011
- Stability of Dynamical SystemsPublished by Elsevier BV ,2007
- Approximate dynamic programming-based approaches for input–output data-driven control of nonlinear processesAutomatica, 2005
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002
- Online learning control by association and reinforcementIEEE Transactions on Neural Networks, 2001