Robostar: An interaction game with humanoid robots for learning sign language

Abstract

This paper presents results from interactive robotic games based on a multimodal user interface. This game is intended to be used as part of children's education and therapy, within a project based on human-robot and humancomputer interaction games featuring non-verbal gestures and imitation for the purpose of teaching sign language. The humanoid robot acts as a social peer and assistant in the games to motivate the child, teach non verbal signs, evaluate the child's effort, and give appropriate feedback to improve the learning and recognition rate of children. In the games, videos of the robot are used, and the performance of the children with hearing impairment are assessed in terms of subjective and objective evaluation criteria. We verified that the results of the tests using the robot's video-based games are in accordance with the physical robots performing same word sets and test setups.