Deep Large-Margin Rank Loss for Multi-Label Image Classification

Abstract

The large-margin technique has served as the foundation of several successful theoretical and empirical results in multi-label image classification. However, most large-margin techniques are only suitable to shallow multi-label models with preset feature representations and a few large-margin techniques of neural networks only enforce margins at the output layer, which are not well suitable for deep networks. Based on the large-margin technique, a deep large-margin rank loss function suitable for any network structure is proposed, which is able to impose a margin on any chosen set of layers of a deep network, allows choosing any ${\ell }_p$ norm ($p\ge 1$) on the metric measuring the margin between labels and is applicable to any network architecture. Although the complete computation of deep large-margin rank loss function has the $\mathcal{O}\left(C^2\right)$ time complexity, where C denotes the size of the label set, which would cause scalability issues when C is large, a negative sampling technique was proposed to make the loss function scale linearly to C. Experimental results on two large-scale datasets, VOC2007 and MS-COCO, show that the deep large-margin ranking function improves the robustness of the model in multi-label image classification tasks while enhancing the model’s anti-noise performance.