Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video
Open Access
- 25 February 2020
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 10 (5), 1562
- https://doi.org/10.3390/app10051562
Abstract
Depth image-based rendering (DIBR) plays an important role in 3D video and free viewpoint video synthesis. However, artifacts might occur in the synthesized view due to viewpoint changes and stereo depth estimation errors. Holes are usually out-of-field regions and disocclusions, and filling them appropriately becomes a challenge. In this paper, a virtual view synthesis approach based on asymmetric bidirectional DIBR is proposed. A depth image preprocessing method is applied to detect and correct unreliable depth values around the foreground edges. For the primary view, all pixels are warped to the virtual view by the modified DIBR method. For the auxiliary view, only the selected regions are warped, which contain the contents that are not visible in the primary view. This approach reduces the computational cost and prevents irrelevant foreground pixels from being warped to the holes. During the merging process, a color correction approach is introduced to make the result appear more natural. In addition, a depth-guided inpainting method is proposed to handle the remaining holes in the merged image. Experimental results show that, compared with bidirectional DIBR, the proposed rendering method can reduce about 37% rendering time and achieve 97% hole reduction. In terms of visual quality and objective evaluation, our approach performs better than the previous methods.This publication has 30 references indexed in Scilit:
- Virtual view synthesis using layered depth image generation and depth-based inpainting for filling disocclusions and translucent disocclusionsJournal of Visual Communication and Image Representation, 2016
- Overview of the Multiview and 3D Extensions of High Efficiency Video CodingIEEE Transactions on Circuits and Systems for Video Technology, 2015
- A Novel 2D-to-3D Video Conversion Method Using Time-Coherent Depth MapsSensors, 2015
- VSI: A Visual Saliency-Induced Index for Perceptual Image Quality AssessmentIEEE Transactions on Image Processing, 2014
- FTV: Free-viewpoint TelevisionSignal Processing: Image Communication, 2012
- Stereoscopic Image Generation Based on Depth Images for 3D TVIEEE Transactions on Broadcasting, 2005
- Smoothing depth maps for improved steroscopic image qualityPublished by SPIE-Intl Soc Optical Eng ,2004
- Region Filling and Object Removal by Exemplar-Based Image InpaintingIEEE Transactions on Image Processing, 2004
- ACM Transactions on Graphics, 2004
- Image Quality Assessment: From Error Visibility to Structural SimilarityIEEE Transactions on Image Processing, 2004