面向图像自动语句标注的注意力反馈模型

Translated title of the contribution: Feedback Attention Model for Image Captioning

Fan Lyu, Fuyuan Hu, Yanning Zhang, Zhenping Xia, Victor S. Sheng

Research output: Contribution to journalArticlepeer-review

Abstract

The image captioning problem aims to let machine generate relevant sentence of a given image, which has been applied to the service robot. To improve the performance of image captioning effectively, some researchers propose to leverage the attention mechanism. However, the mechanism often suffers from distraction and sentence-disorder. In this paper, we propose an image captioning model based on a novel feed-back attention mechanism. In generating the corresponding language for a given image, the proposed model uses the attention feedback from the generated language. With the feedback, the attention heatmap of the original image will be revised, and the generated sentence will also be better. We evaluate the proposed method on three benchmark datasets, i.e., Flickr8k, Flickr30k and MSCOCO, and the experimental results show the superiority of the proposed method.

Translated title of the contributionFeedback Attention Model for Image Captioning
Original languageChinese
Pages (from-to)1122-1129
Number of pages8
JournalJisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics
Volume31
Issue number7
DOIs
StatePublished - Jul 1 2019

Keywords

  • Attention feedback
  • Attention mechanism
  • Image captioning

Fingerprint Dive into the research topics of 'Feedback Attention Model for Image Captioning'. Together they form a unique fingerprint.

Cite this