Efficient hierarchical graph-based video segmentation
Top Cited Papers
- 1 June 2010
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2141-2148
- https://doi.org/10.1109/cvpr.2010.5539893
Abstract
We present an efficient and scalable technique for spatiotemporal segmentation of long video sequences using a hierarchical graph-based algorithm. We begin by over-segmenting a volumetric video graph into space-time regions grouped by appearance. We then construct a “region graph” over the obtained segmentation and iteratively repeat this process over multiple levels to create a tree of spatio-temporal segmentations. This hierarchical approach generates high quality segmentations, which are temporally coherent with stable region boundaries, and allows subsequent applications to choose from varying levels of granularity. We further improve segmentation quality by using dense optical flow to guide temporal connections in the initial graph. We also propose two novel approaches to improve the scalability of our technique: (a) a parallel out-of-core algorithm that can process volumes much larger than an in-core algorithm, and (b) a clip-based processing algorithm that divides the video into overlapping clips in time, and segments them successively while enforcing consistency. We demonstrate hierarchical segmentations on video shots as long as 40 seconds, and even support a streaming mode for arbitrarily long videos, albeit without the ability to process them hierarchically.Keywords
This publication has 17 references indexed in Scilit:
- Video object segmentation by tracking regionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- LIVEcut: Learning-based interactive video segmentation by evaluation of multiple propagated cuesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Real-time edge-aware image processing with the bilateral gridPublished by Association for Computing Machinery (ACM) ,2007
- Object based segmentation of video using color, motion and spatial informationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Video object cut and pastePublished by Association for Computing Machinery (ACM) ,2005
- Video tooningACM Transactions on Graphics, 2004
- Fast multiscale image segmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Mean shift: a robust approach toward feature space analysisIeee Transactions On Pattern Analysis and Machine Intelligence, 2002
- Stylized video cubesPublished by Association for Computing Machinery (ACM) ,2002
- A new motion-compensated reduced-order model Kalman filter for space-varying restoration of progressive and interlaced videoIEEE Transactions on Image Processing, 1998