Self-Critical Sequence Training for Image Captioning
Top Cited Papers
- 1 July 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 1179-1195
- https://doi.org/10.1109/cvpr.2017.131
Abstract
Recently it has been shown that policy-gradient methods for reinforcement learning can be utilized to train deep end-to-end systems directly on non-differentiable metrics for the task at hand. In this paper we consider the problem of optimizing image captioning systems using reinforcement learning, and show that by carefully optimizing our systems using the test metrics of the MSCOCO task, significant gains in performance can be realized. Our systems are built using a new optimization approach that we call self-critical sequence training (SCST). SCST is a form of the popular REINFORCE algorithm that, rather than estimating a baseline to normalize the rewards and reduce variance, utilizes the output of its own test-time inference algorithm to normalize the rewards it experiences. Using this approach, estimating the reward signal (as actor-critic methods must do) and estimating normalization (as REINFORCE algorithms typically do) is avoided, while at the same time harmonizing the model with respect to its test-time inference procedure. Empirically we find that directly optimizing the CIDEr metric with SCST and greedy decoding at test-time is highly effective. Our results on the MSCOCO evaluation sever establish a new state-of-the-art on the task, improving the best result in terms of CIDEr from 104.9 to 114.7.Keywords
This publication has 7 references indexed in Scilit:
- Deep Residual Learning for Image RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Deep visual-semantic alignments for generating image descriptionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Show and tell: A neural image caption generatorPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- CIDEr: Consensus-based image description evaluationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine TranslationPublished by Association for Computational Linguistics (ACL) ,2014
- Long Short-Term MemoryNeural Computation, 1997
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992