3D Finger CAPE: Clicking Action and Position Estimation under Self-Occlusions in Egocentric Viewpoint
- 19 January 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 21 (4), 501-510
- https://doi.org/10.1109/tvcg.2015.2391860
Abstract
In this paper we present a novel framework for simultaneous detection of click action and estimation of occluded fingertip positions from egocentric viewed single-depth image sequences. For the detection and estimation, a novel probabilistic inference based on knowledge priors of clicking motion and clicked position is presented. Based on the detection and estimation results, we were able to achieve a fine resolution level of a bare hand-based interaction with virtual objects in egocentric viewpoint. Our contributions include: (i) a rotation and translation invariant finger clicking action and position estimation using the combination of 2D image-based fingertip detection with 3D hand posture estimation in egocentric viewpoint. (ii) a novel spatio-temporal random forest, which performs the detection and estimation efficiently in a single framework. We also present (iii) a selection process utilizing the proposed clicking action detection and position estimation in an arm reachable AR/VR space, which does not require any additional device. Experimental results show that the proposed method delivers promising performance under frequent self-occlusions in the process of selecting objects in AR/VR space whilst wearing an egocentric-depth camera-attached HMD.Keywords
Funding Information
- Global Frontier R&D Program on <Human-centered Interaction for Coexistence>
- National Research Foundation of Korea
- Korean governmen (2010–0029751)
This publication has 31 references indexed in Scilit:
- Latent Regression Forest: Structured Estimation of 3D Articulated Hand PosturePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Realtime and Robust Hand Tracking from DepthPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Real-Time Articulated Hand Pose Estimation Using Semi-supervised Transductive Regression ForestsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Real-time modeling and tracking manual workflows from first-person visionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Real-time human action search using random forest based hough votingPublished by Association for Computing Machinery (ACM) ,2011
- Unsupervised random forest indexing for fast action searchPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Vision-Based Hand Interaction in Augmented Reality EnvironmentInternational Journal of Human–Computer Interaction, 2011
- A Hough transform-based voting framework for action recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- A real-time multi-cue hand tracking algorithm based on computer visionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Spatiotemporal Relational Random ForestsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009