Monte Carlo Tree Search Methods for the Earth-Observing Satellite Scheduling Problem
- 1 January 2022
- journal article
- research article
- Published by American Institute of Aeronautics and Astronautics (AIAA) in Journal of Aerospace Information Systems
- Vol. 19 (1), 70-82
- https://doi.org/10.2514/1.i010992
Abstract
This work explores on-board planning for the single spacecraft, multiple ground station Earth-observing satellite scheduling problem through artificial neural network function approximation of state–action value estimates generated by Monte Carlo tree search (MCTS). An extensive hyperparameter search is conducted for MCTS on the basis of performance, safety, and downlink opportunity utilization to determine the best hyperparameter combination for data generation. A hyperparameter search is also conducted on neural network architectures. The learned behavior of each network is explored, and each network architecture’s robustness to orbits and epochs outside of the training distributions is investigated. Furthermore, each algorithm is compared with a genetic algorithm, which serves to provide a baseline for optimality. MCTS is shown to compute near-optimal solutions in comparison to the genetic algorithm. The state–action value networks are shown to match or exceed the performance of MCTS in six orders of magnitude less execution time, showing promise for execution on board spacecraft.Keywords
Funding Information
- National Aeronautics and Space Administration (80NSSC20 K1162)
This publication has 16 references indexed in Scilit:
- Using Taskable Remote Sensing in a Sensor Web for Thailand Flood MonitoringJournal of Aerospace Information Systems, 2019
- Optimization-Based Scheduling Method for Agile Earth-Observing Satellite ConstellationJournal of Aerospace Information Systems, 2018
- Scheduling algorithms for rapid imaging using agile Cubesat constellationsAdvances in Space Research, 2018
- Mastering the game of Go without human knowledgeNature, 2017
- Onboard Autonomy on the Intelligent Payload EXperiment CubeSat MissionJournal of Aerospace Information Systems, 2017
- Optimization-based scheduling for the single-satellite, multi-ground station communication problemComputers & Operations Research, 2015
- Sequential ProblemsPublished by MIT Press ,2015
- Using Autonomy Flight Software to Improve Science Return on Earth Observing OneJournal of Aerospace Information Systems, 2005
- Casper: space exploration through continuous planningIEEE Intelligent Systems, 2001
- A Markovian Decision ProcessIndiana University Mathematics Journal, 1957