Research on face specular removal and intrinsic decomposition based on polarization characteristics
- 22 September 2021
- journal article
- research article
- Published by Optica Publishing Group in Optics Express
- Vol. 29 (20), 32256-32270
- https://doi.org/10.1364/oe.440778
Abstract
It is well known that the specular component in the face image destroys the true informantion of the original image and is detrimental to the feature extraction and subsequent processing. However, in many face image processing tasks based on Deep Learning methods, the lack of effective datasets and methods has led researchers to routinely neglect the specular removal process. To solve this problem, we formed the first high-resolution Asian Face Specular-Diffuse-Image-Material (FaceSDIM) dataset based on polarization characterisitics, which consists of real human face specular images, diffuse images, and various corresponding material maps. Secondly, we proposed a joint specular removal and intrinsic decomposition multi-task GAN to generate a de-specular image, normal map, albedo map, residue map and visibility map from a single face image, and also further verified that the prediected de-specular images have a positive enhancement effect on face intrinsic decomposition. Compared with the SOTA algorithm, our method achieves optimal performance both in corrected linear images and in uncorrected wild images of faces.Funding Information
- the Key-Area Research and Development Program of Guangdong Province (No.2019B010149001)
- National Natural Science Foundation of China (No.62072036)
- 111 Project (B18005)
This publication has 28 references indexed in Scilit:
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- Shape and Reflectance Estimation in the WildIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Shape, Illumination, and Reflectance from ShadingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- From Shading to Local ShapeIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Multiview face capture using polarized spherical gradient illuminationACM Transactions on Graphics, 2011
- Practical modeling and acquisition of layered facial reflectanceACM Transactions on Graphics, 2008
- The Appearance of Human Skin: A SurveyFoundations and Trends® in Computer Graphics and Vision, 2007
- The measurement of highlights in color imagesInternational Journal of Computer Vision, 1988
- Using color to separate reflection componentsColor Research & Application, 1985