Video Salient Object Detection via Fully Convolutional Networks
Top Cited Papers
- 31 December 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 27 (1), 38-49
- https://doi.org/10.1109/TIP.2017.2754941
Abstract
This paper proposes a deep learning model to efficiently detect salient regions in videos. It addresses two important issues: 1) deep video saliency model training with the absence of sufficiently large and pixel-wise annotated video data and 2) fast video saliency training and detection. The proposed deep video saliency network consists of two modules, for capturing the spatial and temporal saliency information, respectively. The dynamic saliency model, explicitly incorporating saliency estimates from the static saliency model, directly produces spatiotemporal saliency inference without time-consuming optical flow computation. We further propose a novel data augmentation technique that simulates video training data from existing annotated image data sets, which enables our network to learn diverse saliency information and prevents overfitting with the limited number of training videos. Leveraging our synthetic video data (150K video sequences) and real videos, our deep video saliency model successfully learns both spatial and temporal saliency cues, thus producing accurate spatiotemporal saliency estimate. We advance the state-of-the-art on the densely annotated video segmentation data set (MAE of .06) and the Freiburg-Berkeley Motion Segmentation data set (MAE of .07), and do so with much improved speed (2 fps with all steps).Funding Information
- National Basic Research Program of China 973 Program (2013CB328805)
- National Natural Science Foundation of China (61272359)
- Fok Ying-Tong Education Foundation for Young Teachers
- Joint Building Program of Beijing Municipal Education Commission
This publication has 59 references indexed in Scilit:
- Scene Recognition by Manifold Regularized Deep Learning ArchitectureIEEE Transactions on Neural Networks and Learning Systems, 2015
- Global Contrast Based Salient Region DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Video Saliency Incorporating Spatiotemporal Cues and Uncertainty WeightingIEEE Transactions on Image Processing, 2014
- Lazy Random Walks for Superpixel SegmentationIEEE Transactions on Image Processing, 2014
- Do low-level visual features have a causal influence on gaze during dynamic scene viewing?Journal of Vision, 2013
- Attentional synchrony and the influence of viewing task on gaze behavior in static and dynamic scenesJournal of Vision, 2013
- Object Segmentation by Long Term Analysis of Point TrajectoriesLecture Notes in Computer Science, 2010
- Static and space-time visual saliency detection by self-resemblanceJournal of Vision, 2009
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- A feature-integration theory of attentionCognitive Psychology, 1980