Multi-label Image Classification via Coarse-to-Fine Attention

Fan Lyu, Linyan Li, Victor S. Sheng, Qiming Fu, Fuyuan Hu

Research output: Contribution to journalArticle

Abstract

Great efforts have been made by using deep neural networks to recognize multi-label images. Since multi-label image classification is very complicated, many studies seek to use the attention mechanism as a kind of guidance. Conventional attention-based methods always analyzed images directly and aggressively, which is difficult to well understand complicated scenes. We propose a global/local attention method that can recognize a multi-label image from coarse to fine by mimicking how human-beings observe images. Our global/local attention method first concentrates on the whole image, and then focuses on its local specific objects. We also propose a joint max-margin objective function, which enforces that the minimum score of positive labels should be larger than the maximum score of negative labels horizontally and vertically. This function further improve our multi-label image classification method. We evaluate the effectiveness of our method on two popular multi-label image datasets (i.e., Pascal VOC and MS-COCO). Our experimental results show that our method outperforms state-of-the-art methods.

Original languageEnglish
Pages (from-to)1118-1126
Number of pages9
JournalChinese Journal of Electronics
Volume28
Issue number6
DOIs
StatePublished - Nov 10 2019

    Fingerprint

Keywords

  • Attention
  • Convolutional neural network
  • Multi-label classification
  • Recurrent neural network

Cite this