Hierarchical Generation of Human Pose With Part-Based Layer Representation
- 15 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149), 7856-7866
- https://doi.org/10.1109/tip.2021.3108023
Abstract
Human pose transfer has been becoming one of the emerging research topics in recent years. However, state-of-the-art results are still far from satisfactory. One main reason is that these end-to-end methods are often blindly trained without the semantic understanding of its content. In this paper, we propose a novel method for human pose transfer with consideration of the semantic part-based representation of a human. In particular, we propose to segment the human body into multiple parts, and each of them represents a semantic region of a human. With the proposed part-based layer generators, a high-quality result is guaranteed for each local semantic region. We design a three-stage hierarchical framework to fuse local representations into the final result in a coarse-to-fine manner, which provides adaptive attention for global consistency and local details, respectively. Via exploiting spatial guidance from 3D human model through the framework, our method can naturally handle the ambiguity of self-occlusions which always causes artifacts in previous methods. With semantic-aware and spatial-aware representations, our method outperforms previous approaches quantitatively and qualitatively in better handling self-occlusions, fine detail preservation/synthesis and a higher resolution result.Keywords
Funding Information
- National Natural Science Foundation of China (61521002)
This publication has 46 references indexed in Scilit:
- Globally and locally consistent image completionACM Transactions on Graphics, 2017
- Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial NetworksPublished by Springer Science and Business Media LLC ,2016
- Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single ImagePublished by Springer Science and Business Media LLC ,2016
- DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich AnnotationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Optimizing distributed actor systems for dynamic interactive servicesPublished by Association for Computing Machinery (ACM) ,2016
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- SMPLACM Transactions on Graphics, 2015
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Image Quality Assessment: From Error Visibility to Structural SimilarityIEEE Transactions on Image Processing, 2004
- Simplification and repair of polygonal models using volumetric techniquesIEEE Transactions on Visualization and Computer Graphics, 2003