Assistive Navigation Using Deep Reinforcement Learning Guiding Robot With UWB/Voice Beacons and Semantic Feedbacks for Blind and Visually Impaired People
Open Access
- 22 June 2021
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Robotics and AI
Abstract
Facilitating navigation in pedestrian environments is critical for enabling people who are blind and visually impaired (BVI) to achieve independent mobility. A deep reinforcement learning (DRL)–based assistive guiding robot with ultrawide-bandwidth (UWB) beacons that can navigate through routes with designated waypoints was designed in this study. Typically, a simultaneous localization and mapping (SLAM) framework is used to estimate the robot pose and navigational goal; however, SLAM frameworks are vulnerable in certain dynamic environments. The proposed navigation method is a learning approach based on state-of-the-art DRL and can effectively avoid obstacles. When used with UWB beacons, the proposed strategy is suitable for environments with dynamic pedestrians. We also designed a handle device with an audio interface that enables BVI users to interact with the guiding robot through intuitive feedback. The UWB beacons were installed with an audio interface to obtain environmental information. The on-handle and on-beacon verbal feedback provides points of interests and turn-by-turn information to BVI users. BVI users were recruited in this study to conduct navigation tasks in different scenarios. A route was designed in a simulated ward to represent daily activities. In real-world situations, SLAM-based state estimation might be affected by dynamic obstacles, and the visual-based trail may suffer from occlusions from pedestrians or other obstacles. The proposed system successfully navigated through environments with dynamic pedestrians, in which systems based on existing SLAM algorithms have failed.Funding Information
- Ministry of Science and Technology, Taiwan
- Qualcomm
This publication has 30 references indexed in Scilit:
- Location Fingerprinting With Bluetooth Low Energy BeaconsIEEE Journal on Selected Areas in Communications, 2015
- Including human factors for planning comfortable pathsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Fusing ultra-wideband range measurements with accelerometers and rate gyroscopes for quadrocopter state estimationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Overcoming Barriers and Increasing Independence – Service Robots for Elderly and Disabled PeopleInternational Journal of Advanced Robotic Systems, 2015
- Visibility analysis for autonomous vehicle comfortable navigationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- The ballbot: An omnidirectional balancing mobile robotThe International Journal of Robotics Research, 2013
- Robot-assisted shopping for the blind: issues in spatial cognition and product selectionIntelligent Service Robotics, 2008
- Robot-assisted wayfinding for the visually impaired in structured indoor environmentsAutonomous Robots, 2006
- The GuideCane-applying mobile robot technologies to assist the visually impairedIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2001
- A Formal Basis for the Heuristic Determination of Minimum Cost PathsIEEE Transactions on Systems Science and Cybernetics, 1968