Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes
- 20 August 2021
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 40 (5), 1-18
- https://doi.org/10.1145/3453720
Abstract
For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help deblur a target frame. However, the motion compensation methods adopted by previous deblurring methods are not blur-invariant, and consequently, their accuracy is limited for blurry frames with different blur amounts. To alleviate this problem, we propose two novel approaches to deblur videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors. We combine these two processes to propose an effective recurrent video deblurring network that fully exploits deblurred previous frames. Experiments show that our method achieves the state-of-the-art performance both quantitatively and qualitatively compared to recent methods that use deep learning.Keywords
Funding Information
- Ministry of Science and ICT, Korea (IITP-2015-0-00174, and IITP-2019-0-01906)
- NRF (NRF-2018R1A5A1060031, and NRF-2020R1C1C1014863)
This publication has 45 references indexed in Scilit:
- High-Resolution Stereo Datasets with Subpixel-Accurate Ground TruthLecture Notes in Computer Science, 2014
- TV-L1 Optical Flow EstimationImage Processing On Line, 2013
- Video deblurring for hand-held cameras using patch-based synthesisACM Transactions on Graphics, 2012
- Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World DatabaseLecture Notes in Computer Science, 2012
- A Naturalistic Open Source Movie for Optical Flow EvaluationLecture Notes in Computer Science, 2012
- Non-uniform Deblurring for Shaken ImagesInternational Journal of Computer Vision, 2011
- Two-Phase Kernel Estimation for Robust Motion DeblurringLecture Notes in Computer Science, 2010
- Fast motion deblurringACM Transactions on Graphics, 2009
- PatchMatchACM Transactions on Graphics, 2009
- Image Quality Assessment: From Error Visibility to Structural SimilarityIEEE Transactions on Image Processing, 2004