Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization
- 1 June 2019
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integral-derivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.Keywords
This publication has 28 references indexed in Scilit:
- Autonomous navigation of UAV by using real-time model-based reinforcement learningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Extended observer based on adaptive second order sliding mode control for a fixed wing UAVISA Transactions, 2017
- A Q-Learning Approach to Flocking With UAVs in a Stochastic EnvironmentIEEE Transactions on Cybernetics, 2016
- Non-linear Model Predictive Control for Longitudinal and Lateral Guidance of a Small Fixed-Wing UAV in Precision Deep Stall LandingPublished by American Institute of Aeronautics and Astronautics (AIAA) ,2016
- Human-level control through deep reinforcement learningNature, 2015
- Path Following for Small Unmanned Aerial Vehicles Using L1 Adaptive Augmentation of Commercial AutopilotsJournal of Guidance, Control, and Dynamics, 2010
- Experiments in Fixed-Wing UAV PerchingPublished by American Institute of Aeronautics and Astronautics (AIAA) ,2008
- Automatic Landing System Design using Feedback Linearization MethodPublished by American Institute of Aeronautics and Astronautics (AIAA) ,2007
- Autonomous helicopter control using reinforcement learning policy search methodsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Nonlinear Adaptive Flight Control Using Backstepping and Neural Networks ControllerJournal of Guidance, Control, and Dynamics, 2001