Eye movement prediction and variability on natural video data sets
- 26 March 2012
- journal article
- research article
- Published by Informa UK Limited in Visual Cognition
- Vol. 20 (4-5), 495-514
- https://doi.org/10.1080/13506285.2012.667456
Abstract
We here study the predictability of eye movements when viewing high-resolution natural videos. We use three recently published gaze data sets that contain a wide range of footage, from scenes of almost still-life character to professionally made, fast-paced advertisements and movie trailers. Intersubject gaze variability differs significantly between data sets, with variability being lowest for the professional movies. We then evaluate three state-of-the-art saliency models on these data sets. A model that is based on the invariants of the structure tensor and that combines very generic, sparse video representations with machine learning techniques outperforms the two reference models; performance is further improved for two data sets when the model is extended to a perceptually inspired colour space. Finally, a combined analysis of gaze variability and predictability shows that eye movements on the professionally made movies are the most coherent (due to implicit gaze-guidance strategies of the movie directors), yet the least predictable (presumably due to the frequent cuts). Our results highlight the need for standardized benchmarks to comparatively evaluate eye movement prediction algorithms.Keywords
This publication has 44 references indexed in Scilit:
- TAM: Explaining off-object fixations and central fixation tendencies as effects of population averaging during searchVisual Cognition, 2012
- Eye guidance in natural vision: Reinterpreting salienceJournal of Vision, 2011
- Parallel visual search and rapid animal detection in natural scenesJournal of Vision, 2011
- Efficient coding and multiple motionsVision Research, 2010
- Visual attention guided bit allocation in video compressionImage and Vision Computing, 2010
- Effect of compressed offline foveated video on viewing behavior and subjective qualityACM Transactions on Multimedia Computing, Communications, and Applications, 2010
- Center-surround patterns emerge as optimal predictors for human saccade targetsJournal of Vision, 2009
- Objects predict fixations better than early saliencyJournal of Vision, 2008
- Neurocinematics: The Neuroscience of FilmProjections, 2008
- Image Encoding, Labeling, and Reconstruction from Differential GeometryCVGIP: Graphical Models and Image Processing, 1993