Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation
2017-06-11 09:42
711 查看
Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation
Jean-Benoit Delbrouck, StephaneDupont
(Submitted on 23 Mar 2017)
In state-of-the-art Neural Machine Translation, an attention mechanism is used during decoding to enhance the translation. At every step, the decoder uses this mechanism to focus on different parts of the source sentence to gather the most useful information
before outputting its target word. Recently, the effectiveness of the attention mechanism has also been explored for multimodal tasks, where it becomes possible to focus both on sentence parts and image regions. Approaches to pool two modalities usually include
element-wise product, sum or concatenation. In this paper, we evaluate the more advanced Multimodal Compact Bilinear pooling method, which takes the outer product of two vectors to combine the attention features for the two modalities. This has been previously
investigated for visual question answering. We try out this approach for multimodal image caption translation and show improvements compared to basic combination methods.
Comments: | Submitted to ICLR Workshop 2017 |
Subjects: | Computation and Language (cs.CL) |
Cite as: | arXiv:1703.08084 [cs.CL] |
(or arXiv:1703.08084v1 [cs.CL] for this version) |
Submission history
From: Jean-Benoit Delbrouck [view email][v1] Thu, 23 Mar 2017 14:20:52 GMT (135kb,D)
相关文章推荐
- 阅读笔记(Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding)
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- 论文笔记 :Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Sampled Softmax 论文笔记:On Using Very Large Target Vocabulary for Neural Machine Translation
- On Using Very Large Target Vocabulary for Neural Machine Translation
- Sampled Softmax 论文笔记:On Using Very Large Target Vocabulary for Neural Machine Translation
- ACL 2016 | Modeling Coverage for Neural Machine Translation
- VQA 之 Multimodal Compact Bilinear Pooling
- VQA 之 Multimodal Compact Bilinear Pooling
- Modeling Coverage for Neural Machine Translation
- keras 2.0:Encoder-Decoder Sequence-to-Sequence Model for Neural Machine Translation
- Neural Networks for Machine Learning by Geoffrey Hinton (1~2)
- 论文笔记:Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
- Neural Networks for Machine Learning by Geoffrey Hinton (3)
- 神经机器翻译(Neural Machine Translation)系列教程 - (七)机器翻译-缩写-大攻略
- Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
- 机器学习中的神经网络Neural Networks for Machine Learning:Lecture 8 Quiz
- 《Neural Networks for Machine Learning》 by Hinton 学习笔记(一)
- TensorFlow 神经机器翻译教程-TensorFlow Neural Machine Translation Tutorial
- 神经机器翻译(Neural Machine Translation)系列教程 - (四)Ubuntu16.04 + cuda8.0安装