Pre-training with non-expert human demonstration for deep reinforcement learning
- 26 July 2019
- journal article
- review article
- Published by Cambridge University Press (CUP) in The Knowledge Engineering Review
Abstract
Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from slow learning speeds and often requires a prohibitively large amount of training time and data to reach reasonable performance, making it inapplicable to real-world settings where data are expensive. In this work, we improve data efficiency in deep RL by addressing one of the two learning goals, feature learning. We leverage supervised learning to pre-train on a small set of non-expert human demonstrations and empirically evaluate our approach using the asynchronous advantage actor-critic algorithms in the Atari domain. Our results show significant improvements in learning speed, even when the provided demonstration is noisy and of low quality.Keywords
This publication has 14 references indexed in Scilit:
- Deep learning for healthcare: review, opportunities and challengesBriefings in Bioinformatics, 2017
- Towards Knowledge Transfer in Deep Reinforcement LearningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Deep Direct Reinforcement Learning for Financial Signal Representation and TradingIEEE Transactions on Neural Networks and Learning Systems, 2016
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Association of eating speed and energy intake of main meals with overweight in Chinese pre-school childrenPublic Health Nutrition, 2013
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- Q-learningMachine Learning, 1992