Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics
Top Cited Papers
- 10 December 2012
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 24 (1), 145-157
- https://doi.org/10.1109/tnnls.2012.2227339
Abstract
To synthesize fixed-final-time control-constrained optimal controllers for discrete-time nonlinear control-affine systems, a single neural network (NN)-based controller called the Finite-horizon Single Network Adaptive Critic is developed in this paper. Inputs to the NN are the current system states and the time-to-go, and the network outputs are the costates that are used to compute optimal feedback control. Control constraints are handled through a nonquadratic cost function. Convergence proofs of: 1) the reinforcement learning-based training method to the optimal solution; 2) the training error; and 3) the network weights are provided. The resulting controller is shown to solve the associated time-varying Hamilton-Jacobi-Bellman equation and provide the fixed-final-time optimal solution. Performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite-time attitude maneuver subject to control bounds. The new formulation has great potential for implementation since it consists of only one NN with single set of weights and it provides comprehensive feedback solutions online, though it is trained offline.Keywords
This publication has 27 references indexed in Scilit:
- Simple and Fast Calculation of the Second-Order Gradients for Globalized Dual Heuristic Dynamic Programming in Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2012
- Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy UpdateIEEE Transactions on Neural Networks and Learning Systems, 2012
- Approximate dynamic programming solutions with a single network adaptive critic for a class of nonlinear systemsControl Theory and Technology, 2011
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009
- A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systemsNeural Networks, 2006
- Online Adaptive Critic Flight ControlJournal of Guidance, Control, and Dynamics, 2004
- State-constrained agile missile control with adaptive-critic-based neural networksIEEE Transactions on Control Systems Technology, 2002
- Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogeneratorIEEE Transactions on Neural Networks, 2002
- Adaptive-critic-based neural networks for aircraft optimal controlJournal of Guidance, Control, and Dynamics, 1996