StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning
Top Cited Papers
- 27 April 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 3 (1), 73-84
- https://doi.org/10.1109/tetci.2018.2823329
Abstract
Real-time strategy games have been an important field of game artificial intelligence in recent years. This paper presents a reinforcement learning and curriculum transfer learning method to control multiple units in StarCraft micromanagement. We define an efficient state representation, which breaks down the complexity caused by the large state space in the game environment. Then, a parameter sharing multi-agent gradient-descent Sarsa($\lambda$) algorithm is proposed to train the units. The learning policy is shared among our units to encourage cooperative behaviors. We use a neural network as a function approximator to estimate the action–value function, and propose a reward function to help units balance their move and attack. In addition, a transfer learning method is used to extend our model to more difficult scenarios, which accelerates the training process and improves the learning performance. In small-scale scenarios, our units successfully learn to combat and defeat the built-in AI with 100% win rates. In large-scale scenarios, the curriculum transfer learning method is used to progressively train a group of units, and it shows superior performance over some baseline methods in target scenarios. With reinforcement learning and curriculum transfer learning, our units are able to learn appropriate strategies in StarCraft micromanagement scenarios.Keywords
Funding Information
- National Natural Science Foundation of China (61573353, 61603382, 61533017)
This publication has 31 references indexed in Scilit:
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Deep learningNature, 2015
- Hybrid Pathfinding in StarCraftIEEE Transactions on Computational Intelligence and AI in Games, 2015
- Human-level control through deep reinforcement learningNature, 2015
- A review of computational intelligence in RTS gamesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Self-teaching adaptive dynamic programming for GomokuNeurocomputing, 2012
- Connectionist reinforcement learning for intelligent unit micro management in StarCraftPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Reinforcement learning with replacing eligibility tracesMachine Learning, 1996
- Markov games as a framework for multi-agent reinforcement learningPublished by Elsevier BV ,1994