4D Association Graph for Realtime Multi-Person Motion Capture Using Multiple Video Cameras
- 1 June 2020
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- p. 1321-1330
- https://doi.org/10.1109/cvpr42600.2020.00140
Abstract
his paper contributes a novel realtime multi-person motion capture algorithm using multiview video inputs. Due to the heavy occlusions and closely interacting motions in each view, joint optimization on the multiview images and multiple temporal frames is indispensable, which brings up the essential challenge of realtime efficiency. To this end, for the first time, we unify per-view parsing, cross-view matching, and temporal tracking into a single optimization framework, i.e., a 4D association graph that each dimension (image space, viewpoint and time) can be treated equally and simultaneously. To solve the 4D association graph efficiently, we further contribute the idea of 4D limb bundle parsing based on heuristic searching, followed with limb bundle assembling by proposing a bundle Kruskal's algorithm. Our method enables a realtime motion capture system running at 30fps using 5 cameras on a 5-person scene. Benefiting from the unified parsing, matching and tracking constraints, our method is robust to noisy detection due to severe occlusions and close interacting motions, and achieves high-quality online pose reconstruction quality. The proposed method outperforms state-of-the-art methods quantitatively without using high-level appearance information.Keywords
This publication has 30 references indexed in Scilit:
- Multiple human 3D pose estimation from multiview imagesMultimedia Tools and Applications, 2017
- ArtTrack: Articulated Multi-Person Tracking in the WildPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- DeepCut: Joint Subset Partition and Labeling for Multi Person Pose EstimationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Convolutional Pose MachinesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- 3D Pictorial Structures Revisited: Multiple Human Pose EstimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Multiple Human Pose Estimation with Temporally Consistent 3D Pictorial StructuresLecture Notes in Computer Science, 2015
- 3D Pictorial Structures for Multiple Human Pose EstimationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Multi-view Body Part Recognition with Random ForestsPublished by British Machine Vision Association and Society for Pattern Recognition ,2013
- Markerless motion capture of interacting characters using multi-view image segmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Techniques for nonlinear least squares and robust regressionCommunications in Statistics - Simulation and Computation, 1978