Full body virtual try‐on with semi‐self‐supervised learning
Open Access
- 12 September 2021
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 57 (24), 915-917
- https://doi.org/10.1049/ell2.12307
Abstract
This paper proposes a full body virtual try-on which handles both top and bottom garments and generates realistic try-on images. For the full body virtual try-on, this paper addresses lack of suitable training data to align and fit top and bottom naturally. The proposed system consists of three modules: Clothing Guide Module (CGM), Geometric Matching Module (GMM), and Try-On Module (TOM). CGM is introduced to generate a clothing guide map (CGMap) which describes the shape of a garment on a model. Unlike the single-garment virtual try-on scheme, it is impractical to collect meaningful data at a large scale for the multi-garment system. To address this problem, two novel training strategies are proposed to leverage the existing training data. First, a pseudo triplet of model-top-bottom is generated from a pair of model-top or model-bottom which are already secured. Second, the CGM network is arranged to be exposed to both top and bottom garments during training. Then, the following GMM networks warp and align the top and bottom garments. Finally, TOM synthesizes a realistic try-on image with the aligned garment and the CGMap. Experimental results prove remarkable performance of the proposed method in the full body virtual try-on.Keywords
Funding Information
- Korea Evaluation Institute of Industrial Technology (20008625)
This publication has 10 references indexed in Scilit:
- Toward Accurate and Realistic Outfits Visualization with Attention to DetailsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2021
- Image Based Virtual Try-On Network From Unpaired DataPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2020
- Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-Based ApproachPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2020
- Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image ContentPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2020
- LA-VITON: A Network for Looking-Attractive Virtual Try-OnPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2019
- Semantic Image Synthesis With Spatially-Adaptive NormalizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2019
- CBAM: Convolutional Block Attention ModulePublished by Springer Science and Business Media LLC ,2018
- Instance-Level Human Parsing via Part Grouping NetworkPublished by Springer Science and Business Media LLC ,2018
- VITON: An Image-Based Virtual Try-on NetworkPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2018
- DensePose: Dense Human Pose Estimation in the WildPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2018