Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning
- 1 October 2021
- journal article
- research article
- Published by American Institute of Aeronautics and Astronautics (AIAA) in Journal of Aerospace Information Systems
- Vol. 18 (10), 739-750
- https://doi.org/10.2514/1.i010961
Abstract
In this study, reinforcement learning (RL)-based centralized path planning is performed for an unmanned combat aerial vehicle (UCAV) fleet in a human-made hostile environment. The proposed method provides a novel approach in which closing speed and approximate time-to-go terms are used in the reward function to obtain cooperative motion while ensuring no-fly-zones (NFZs) and time-of-arrival constraints. Proximal policy optimization (PPO) algorithm is used in the training phase of the RL agent. System performance is evaluated in two different cases. In case 1, the warfare environment contains only the target area, and simultaneous arrival is desired to obtain the saturated attack effect. In case 2, the warfare environment contains NFZs in addition to the target area and the standard saturated attack and collision avoidance requirements. Particle swarm optimization (PSO)-based cooperative path planning algorithm is implemented as the baseline method, and it is compared with the proposed algorithm in terms of execution time and developed performance metrics. Monte Carlo simulation studies are performed to evaluate the system performance. According to the simulation results, the proposed system is able to generate feasible flight paths in real-time while considering the physical and operational constraints such as acceleration limits, NFZ restrictions, simultaneous arrival, and collision avoidance requirements. In that respect, the approach provides a novel and computationally efficient method for solving the large-scale cooperative path planning for UCAV fleets.Keywords
This publication has 13 references indexed in Scilit:
- Four-Dimensional Trajectory Generation for UAVs Based on Multi-Agent Q LearningJournal of Navigation, 2020
- Learn to Navigate: Cooperative Path Planning for Unmanned Surface Vehicles Using Deep Reinforcement LearningIEEE Access, 2019
- Distributed Wildfire Surveillance with Autonomous Aircraft Using Deep Reinforcement LearningJournal of Guidance, Control, and Dynamics, 2019
- Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2019
- Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning ApproachIEEE Transactions on Wireless Communications, 2019
- A Virtual Motion Camouflage Approach for Cooperative Trajectory Planning of Multiple UCAVsMathematical Problems in Engineering, 2014
- Small Unmanned AircraftPublished by Walter de Gruyter GmbH ,2012
- Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for Real-Time UAV Path PlanningIEEE Transactions on Industrial Informatics, 2012
- Path Planning in Two DimensionsPublished by Wiley ,2010
- Cooperative Control of Multiple UCAVs for Suppression of Enemy Air DefensePublished by American Institute of Aeronautics and Astronautics (AIAA) ,2004