Spatial attention model‐modulated bi‐directional long short‐term memory for unsupervised video summarisation
Open Access
- 23 February 2021
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in Electronics Letters
- Vol. 57 (6), 252-254
- https://doi.org/10.1049/ell2.12111
Abstract
Compared with surveillance video, user-created videos contain more frequent shot changes, which lead to diversified backgrounds and a wide variety of content. The high redundancy among keyframes is a critical issue for the existing summarising methods in dealing with user-created videos. To address the critical issue, we designed a salient- area-size-based spatial attention model (SAM) on the observation that humans tend to focus on sizable and moving objects in videos. Moreover, the SAM is taken as guidance to refine frame-wise soft selected probability for the bi-directional long short-term memory model. The reinforcement learning framework, trained by the deep deterministic policy gradient algorithm, is adopted to do unsupervised training. Extensive experiments on the SumMe and TVSum datasets demonstrate that our method outperforms the state-of-the-art in terms of F-score.Keywords
Funding Information
- National Natural Science Foundation of China (62002130, 61702472, 2019AAA049)
This publication has 8 references indexed in Scilit:
- Unsupervised Video Summarization With Cycle-Consistent Adversarial LSTM NetworksIEEE Transactions on Multimedia, 2019
- Video Summarization With Attention-Based Encoder–Decoder NetworksIEEE Transactions on Circuits and Systems for Video Technology, 2019
- Video Salient Object Detection via Fully Convolutional NetworksIEEE Transactions on Image Processing, 2017
- Unsupervised Video Summarization with Adversarial LSTM NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- Video Summarization with Long Short-Term MemoryPublished by Springer Science and Business Media LLC ,2016
- See all by looking at a few: Sparse modeling for finding representative objectsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Additive Bernoulli Noise Linear Sequential CircuitsIEEE Transactions on Computers, 1972
- Is selective attention selective perception or selective response? A further test.Journal of Experimental Psychology, 1969