Connectionist reinforcement learning for intelligent unit micro management in StarCraft
- 1 July 2011
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1794-1801
- https://doi.org/10.1109/ijcnn.2011.6033442
Abstract
Real Time Strategy Games are one of the most popular game schemes in PC markets and offer a dynamic environment that involves several interacting agents. The core strategies that need to be developed in these games are unit micro management, building order, resource management, and the game main tactic. Unfortunately, current games only use scripted and fixed behaviors for their artificial intelligence (AI), and the player can easily learn the counter measures to defeat the AI. In this paper, we describe a system based on neural networks that controls a set of units of the same type in the popular game StarCraft. Using the neural networks, the units will either choose a unit to attack or evade from the battlefield. The system uses reinforcement learning combined with neural networks using online Sarsa and neural-fitted Sarsa, both with a short term memory reward function. We also present an incremental learning method for training the units for larger scenarios involving more units using trained neural networks on smaller scenarios. Additionally, we developed a novel sensing system to feed the environment data to the neural networks using separate vision grids. The simulation results show superior performance against the human-made AI scripts in StarCraft.Keywords
This publication has 7 references indexed in Scilit:
- Opponent modelling for case-based adaptive game AIEntertainment Computing, 2009
- Using NEAT for continuous adaptation and teamwork formation in PacmanPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- Implementing Parametric Reinforcement Learning in Robocup Rescue SimulationLecture Notes in Computer Science, 2008
- Ambulance Decision Support Using Evolutionary Reinforcement Learning in Robocup Rescue Simulation LeagueLecture Notes in Computer Science, 2007
- Using Dynamic Neural Network to Model Team Performance for Coordination Algorithm Configuration and Reconfiguration of Large Multi-Agent TeamsPublished by ASME International ,2006
- Reinforcement Learning: A SurveyJournal of Artificial Intelligence Research, 1996
- Q-learningMachine Learning, 1992