Character-Based Value Factorization For MADRL
- 19 September 2022
- journal article
- research article
- Published by Oxford University Press (OUP) in The Computer Journal
- Vol. 66 (11), 2782-2793
- https://doi.org/10.1093/comjnl/bxac121
Abstract
Value factorization is a popular method for cooperative multi-agent deep reinforcement learning. In this method, agents generally have the same ability and rely only on individual value function to select actions, which is calculated from total environment reward. It ignores the impact of individual characteristics of heterogeneous agents on actions selection, which leads to the lack of pertinence during training and the increase of difficulty in learning effective policies. In order to stimulate individual awareness of heterogeneous agents and improve their learning efficiency and stability, we propose a novel value factorization method based on Personality Characteristics, PCQMIX, which assigns personality characteristics to each agent and takes them as internal rewards to train agents. As a result, PCQMIX can generate heterogeneous agents with specific personality characteristics suitable for specific scenarios. Experiments show that PCQMIX generates agents with stable personality characteristics and outperforms all baselines in multiple scenarios of the StarCraft II micromanagement task.Keywords
This publication has 10 references indexed in Scilit:
- Tactical UAV path optimization under radar threat using deep reinforcement learningNeural Computing & Applications, 2022
- AVD-Net: Attention Value Decomposition Network For Deep Multi-Agent Reinforcement LearningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2021
- Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and ApplicationsIEEE Transactions on Cybernetics, 2020
- Human-level performance in 3D multiplayer games with population-based reinforcement learningScience, 2019
- Connected Vehicle as a Mobile Sensor for Real Time Queue Length at Signalized IntersectionsSensors, 2019
- A Novel Heterogeneous Swarm Reinforcement Learning Method for Sequential Decision Making ProblemsMachine Learning and Knowledge Extraction, 2019
- Multiagent cooperation and competition with deep reinforcement learningPLOS ONE, 2017
- Design and Analysis of Self-Adapted Task Scheduling Strategies in Wireless Sensor NetworksSensors, 2011
- Optimal and Approximate Q-value Functions for Decentralized POMDPsJournal of Artificial Intelligence Research, 2008
- CharacterPublished by Oxford University Press (OUP) ,1995