Deep Human Parsing with Active Template Regression
- 3 March 2015
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 37 (12), 2402-2414
- https://doi.org/10.1109/tpami.2015.2408360
Abstract
In this work, the human parsing task, namely decomposing a human image into semantic fashion/body regions, is formulated as an active template regression (ATR) problem, where the normalized mask of each fashion/body item is expressed as the linear combination of the learned mask templates, and then morphed to a more precise mask with the active shape parameters, including position, scale and visibility of each semantic region. The mask template coefficients and the active shape parameters together can generate the human parsing results, and are thus called the structure outputs for human parsing. The deep Convolutional Neural Network (CNN) is utilized to build the end-to-end relation between the input human image and the structure outputs for human parsing. More specifically, the structure outputs are predicted by two separate networks. The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters. For a new image, the structure outputs of the two networks are fused to generate the probability of each label for each pixel, and super-pixel smoothing is finally used to refine the human parsing result. Comprehensive evaluations on a large dataset well demonstrate the significant superiority of the ATR framework over other state-of-the-arts for human parsing. In particular, the F1-score reaches 64.38 percent by our ATR framework, significantly higher than 44.76 percent based on the state-of-the-art algorithm [28].Keywords
Funding Information
- National Natural Science Foundation of China (61328205)
- Microsoft Research Asia collaboration projects, Guangdong Natural Science Foundation (S2013050014548)
- Program of Guangzhou Zhujiang Star of Science and Technology (2013J2200067)
- Special Project on Integration of Industry, Education and Research of Guangdong (2012B091000101)
This publication has 21 references indexed in Scilit:
- Discriminatively Trained And-Or Graph Models for Object Shape DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Clothing Co-parsing by Joint Image Segmentation and LabelingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Rich Feature Hierarchies for Accurate Object Detection and Semantic SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- DeepPose: Human Pose Estimation via Deep Neural NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- A Deformable Mixture Parsing Model with ParseletsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing ItemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Human Pose Estimation Using Body Parts Dependent Joint RegressorsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Efficient Graph-Based Image SegmentationInternational Journal of Computer Vision, 2004
- Active appearance modelsIeee Transactions On Pattern Analysis and Machine Intelligence, 2001
- Active Shape Models-Their Training and ApplicationComputer Vision and Image Understanding, 1995