ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
Open Access
- 21 January 2022
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Neurorobotics
- Vol. 15, 785808
- https://doi.org/10.3389/fnbot.2021.785808
Abstract
With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute.This publication has 33 references indexed in Scilit:
- RSGANPublished by Association for Computing Machinery (ACM) ,2018
- On Face Segmentation, Face Swapping, and Face PerceptionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2018
- Arbitrary Style Transfer in Real-Time with Adaptive Instance NormalizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Realistic Dynamic Facial Textures from a Single Image Using GANsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Fast Face-Swap Using Convolutional Neural NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- A Novel Method to Detect Functional microRNA Regulatory Modules by Bicliques MergingIEEE/ACM Transactions on Computational Biology and Bioinformatics, 2015
- Visual Cryptography for Biometric PrivacyIEEE Transactions on Information Forensics and Security, 2010
- Face swappingACM Transactions on Graphics, 2008
- Scope of validity of PSNR in image/video quality assessmentElectronics Letters, 2008
- Image Quality Assessment: From Error Visibility to Structural SimilarityIEEE Transactions on Image Processing, 2004