Machine Teaching for Human Inverse Reinforcement Learning
Open Access
- 30 June 2021
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Robotics and AI
Abstract
As robots continue to acquire useful skills, their ability to teach their expertise will provide humans the two-fold benefit of learning from robots and collaborating fluently with them. For example, robot tutors could teach handwriting to individual students and delivery robots could convey their navigation conventions to better coordinate with nearby human workers. Because humans naturally communicate their behaviors through selective demonstrations, and comprehend others’ through reasoning that resembles inverse reinforcement learning (IRL), we propose a method of teaching humans based on demonstrations that are informative for IRL. But unlike prior work that optimizes solely for IRL, this paper incorporates various human teaching strategies (e.g. scaffolding, simplicity, pattern discovery, and testing) to better accommodate human learners. We assess our method with user studies and find that our measure of test difficulty corresponds well with human performance and confidence, and also find that favoring simplicity and pattern discovery increases human performance on difficult tests. However, we did not find a strong effect for our method of scaffolding, revealing shortcomings that indicate clear directions for future work.Funding Information
- Office of Naval Research (N00014-18-1-2503)
- Defense Advanced Research Projects Agency (W911NF-20-1-0006)
This publication has 33 references indexed in Scilit:
- The effectiveness of adaptive difficulty adjustments on students' motivation and learning in an educational computer gameComputers & Education, 2013
- Mapping value based planning and extensively trained choice in the human brainNature Neuroscience, 2012
- A Comparative Study of Redundant Constraints Identification Methods in Linear Programming ProblemsMathematical Problems in Engineering, 2010
- Action understanding as inverse planningCognition, 2009
- Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral controlNature Neuroscience, 2005
- How Many Variables Can Humans Process?Psychological Science, 2005
- Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student WorkJournal of the Learning Sciences, 2004
- THE ROLE OF TUTORING IN PROBLEM SOLVING*Journal of Child Psychology and Psychiatry, 1976