A novel tracking system for human following robots with fusion of MMW radar and monocular vision
- 16 September 2021
- journal article
- research article
- Published by Emerald in Industrial Robot: the international journal of robotics research and application
- Vol. 49 (1), 120-131
- https://doi.org/10.1108/ir-02-2021-0030
Abstract
This paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system. Limited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance. A series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person. This paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.Keywords
This publication has 19 references indexed in Scilit:
- A Novel Vision-Based Tracking Algorithm for a Human-Following Mobile RobotIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2016
- Identification of a specific person using color, height, and gait features for a person following robotRobotics and Autonomous Systems, 2016
- On-Road Vehicle Detection and Tracking Using MMW Radar and Monovision FusionIEEE Transactions on Intelligent Transportation Systems, 2016
- A General Approach to Spatiotemporal Calibration in Multisensor SystemsIEEE Transactions on Robotics, 2016
- Development of Human Following Mobile Robot System Using Laser Range ScannerProcedia Computer Science, 2015
- Data fusion of radar and image measurements for multi-object tracking via Kalman filteringInformation Sciences, 2014
- Development of a Laser-Range-Finder-Based Human Tracking and Control Algorithm for a Marathoner Service RobotIEEE/ASME Transactions on Mechatronics, 2013
- Millimeter-Wave Technology for Automotive Radar Sensors in the 77 GHz Frequency BandIEEE Transactions on Microwave Theory and Techniques, 2012
- Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection ApplicationsSensors, 2011
- Multisensor-Based Human Detection and Tracking for Mobile Service RobotsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008