Interactive Dual Generative Adversarial Networks for Image Captioning

• DOI: 10.1609/aaai.v34i07.6826
• Publication Date: 2020-06-29T18:34:46Z
• Authors: Junhao Liu; Kai Wang; Chunpu Xu; Zhou Zhao; Ruifeng Xu; Ying Shen; Min Yang
• Keywords: Closed captioning; Generative adversarial network; Natural Language Generation

Image captioning is usually built on either generation-based or retrieval-based approaches. Both ways have certain strengths but suffer from their own limitations. In this paper, we propose an Interactive Dual Generative Adversarial Network (IDGAN) for image captioning, which mutually combines the and methods to learn a better ensemble. IDGAN consists of two generators discriminators, where generation- benefit each other's complementary targets that are learned dual adversarial discriminators. Specifically, provide improved synthetic retrieved candidate captions with informative feedback signals respective discriminators trained distinguish generated true assign top rankings respectively, thus featuring merits both Extensive experiments MSCOCO dataset demonstrate proposed model significantly outperforms compared captioning.