Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder
- 20 July 2017
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 36 (4), 1-12
- https://doi.org/10.1145/3072959.3073601
Abstract
We describe a machine learning technique for reconstructing image sequences rendered using Monte Carlo methods. Our primary focus is on reconstruction of global illumination with extremely low sampling budgets at interactive rates. Motivated by recent advances in image restoration with deep convolutional networks, we propose a variant of these networks better suited to the class of noise present in Monte Carlo rendering. We allow for much larger pixel neighborhoods to be taken into account, while also improving execution speed by an order of magnitude. Our primary contribution is the addition of recurrent connections to the network in order to drastically improve temporal stability for sequences of sparsely sampled input images. Our method also has the desirable property of automatically modeling relationships based on auxiliary per-pixel input channels, such as depth and normals. We show significantly higher quality results compared to existing methods that run at comparable speeds, and furthermore argue a clear path for making our method run at realtime rates in the near future.Keywords
This publication has 55 references indexed in Scilit:
- Path Space Regularization for Holistic and Robust Light TransportComputer Graphics Forum, 2013
- Guided Image Filtering for Interactive High‐quality Global IlluminationComputer Graphics Forum, 2011
- Frequency analysis and sheared filtering for shadow light fields of complex occludersACM Transactions on Graphics, 2011
- OptiXACM Transactions on Graphics, 2010
- Frequency analysis and sheared reconstruction for rendering motion blurACM Transactions on Graphics, 2009
- Anisotropic diffusion for Monte Carlo noise reductionACM Transactions on Graphics, 1999
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Generalization of backpropagation with application to a recurrent gas market modelNeural Networks, 1988
- The triangle processor and normal vector shaderACM SIGGRAPH Computer Graphics, 1988
- Estimation of the Mean of a Multivariate Normal DistributionThe Annals of Statistics, 1981