Deep Hierarchical Reinforcement Learning for Autonomous Driving with Distinct Behaviors
- 1 June 2018
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1239-1244
- https://doi.org/10.1109/ivs.2018.8500368
Abstract
Deep reinforcement learning has achieved great progress recently in domains such as learning to play Atari games from raw pixel input. The model-free characteristics of reinforcement learning free us from hand-encoding complex policies. However, for real world tasks such as autonomous driving, there are some complex sequential decision making processes that contain distinct behaviors. Due to the delayed rewards and the averaged gradient, it is pretty difficult for a flat deep reinforcement learning algorithm to learn a good policy. In this paper, we design a hierarchical neural network policy and propose a hierarchical policy gradient method to train the network with the semi markov decision process (SMDP) temporal abstraction formulation. We apply this method to a traffic light passing scenario in autonomous driving, where the vehicle has two distinct behaviors (e.g., pass and stop) and its primitive actions (e.g., acceleration) should follow the corresponding behavior. We show via simulation that our method is able to select correct decision and acts appropriately when the traffic light turns yellow. On the contrary, the flat reinforcement learning algorithm is not able to achieve a good performance and exhibits a large variance. Furthermore, the trained neural network modules are reusable in the future to cover more scenarios.This publication has 11 references indexed in Scilit:
- Safe and feasible motion generation for autonomous driving via constrained policy netPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Empagliflozin for Type 2 Diabetes Mellitus: An Overview of Phase 3 Clinical TrialsCurrent Diabetes Reviews, 2017
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Deep Reinforcement Learning framework for Autonomous DrivingElectronic Imaging, 2017
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Human-level control through deep reinforcement learningNature, 2015
- Recent Advances in Hierarchical Reinforcement LearningDiscrete Event Dynamic Systems, 2003
- Hierarchical Reinforcement Learning with the MAXQ Value Function DecompositionJournal of Artificial Intelligence Research, 2000
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999
- Reinforcement Learning: A SurveyJournal of Artificial Intelligence Research, 1996