Boosting Single Image Super-Resolution Learnt From Implicit Multi-Image Prior
- 23 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149), 3240-3251
- https://doi.org/10.1109/tip.2021.3059507
Abstract
Learning-based single image super-resolution (SISR) aims to learn a versatile mapping from low resolution (LR) image to its high resolution (HR) version. The critical challenge is to bias the network training towards continuous and sharp edges. For the first time in this work, we propose an implicit boundary prior learnt from multi-view observations to significantly mitigate the challenge in SISR we outline. Specifically, the multi-image prior that encodes both disparity information and boundary structure of the scene supervise a SISR network for edge-preserving. For simplicity, in the training procedure of our framework, light field (LF) serves as an effective multi-image prior, and a hybrid loss function jointly considers the content, structure, variance as well as disparity information from 4D LF data. Consequently, for inference, such a general training scheme boosts the performance of various SISR networks, especially for the regions along edges. Extensive experiments on representative backbone SISR architectures constantly show the effectiveness of the proposed method, leading to around 0.6 dB gain without modifying the network architecture.Keywords
Funding Information
- Natural Science Foundation of China (61860206003)
- Shenzhen Science and Technology Research and Development Funds (JCYJ20180507183706645)
This publication has 57 references indexed in Scilit:
- A+: Adjusted Anchored Neighborhood Regression for Fast Super-ResolutionPublished by Springer Science and Business Media LLC ,2015
- Learning a Deep Convolutional Network for Image Super-ResolutionLecture Notes in Computer Science, 2014
- Variational Light Field Analysis for Disparity Estimation and Super-ResolutionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Single lens 3D-camera with extended depth-of-fieldPublished by SPIE-Intl Soc Optical Eng ,2012
- Super-resolution image reconstruction: a technical overviewIEEE Signal Processing Magazine, 2003
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statisticsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Limits on super-resolution and how to break themIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Restoration of a single superresolution image from several blurred, noisy, and undersampled measured imagesIEEE Transactions on Image Processing, 1997
- Improving resolution by image registrationCVGIP: Graphical Models and Image Processing, 1991
- Generalized sampling expansionIEEE Transactions on Circuits and Systems, 1977