论文《NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE》总结
2017-06-28 20:53
656 查看
NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE
论文来源:Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
原文链接:摘要
神经机器翻译最近被提出用于机器翻译。与传统的统计机器翻译不同,神经机器翻译旨在构建一个神经网络能够共同联调最大化翻译效果。最近提出的神经机器翻译模型经常包括一类编码器和解码器,把一个源句子编码为一个固定长度的向量,解码器产生翻译。但是我们认为生成一个固定长度的向量是一个瓶颈,propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly。最后我们的模型取得了显著的效果。1.Introduction
神经机器翻译被提出(2013, 2014),不像传统的基于短语的翻译系统,包含很多小的子组件共同调整,神经机器翻译尝试去构建一个单个的神经网络系统,读一个句子并且输出一个正确的翻译。大部分已经提出的模型是属于编码器和解码器,把一个源句子编码为一个固定长度的向量,解码器产生翻译。每个语言有自己的编码-解码对,共同训练去最大化正确翻译的可能性。
将所有的信息转化为一个固定长度的向量,对于长句子会无法处理,尤其是比训练的句子更长的时候,研究表明,随着句子的加长,传统的编码-解码对的效果会恶化。
为了处理上述的问题,我们提出了一种方法,Each time the proposed model generates a word in a translation, it (soft-)searches for a set of positions in a source sentence where the most relevant information is concentrated. The model then predicts a target word based on the context vectors associated with these source positions and all the previous generated target words.
我们的模型在长句子上面效果很好,但是也使用与各种长度的句子,比现有的其他模型效果要好的多。
2.BACKGROUND: NEURAL MACHINE TRANSLATION
1.从概率的角度出发,翻译等价于给定一个原句子x,找到一个出现概率最大的目标句子y,我们使用句子对语料库来训练模型最近,2013-2014提出了一些神经网络模型,由两部分组成,编码器和解码器,例如两个RNN,一个用来编码,一个用来解码。
神经网络模型比传统的基于短语的翻译系统效果要好,加入神经网络组件到现有的翻译系统中,
。。。。之后继续更新。。。。。。。。。
单词
conjecture英 [kən’dʒektʃə] 美 [kən’dʒɛktʃɚ]
n. 推测;猜想
vi. 推测;揣摩
vt. 推测
bottleneck
英 [‘bɒt(ə)lnek] 美 [‘bɑtlnɛk]
n. 瓶颈;障碍物
In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving th
a828
e performance of this basic encoder–decoder architecture
deteriorate
英 [dɪ’tɪərɪəreɪt] 美 [dɪ’tɪrɪəret]
vi. 恶化,变坏
vt. 恶化
相关文章推荐
- 论文阅读:《Neural Machine Translation by Jointly Learning to Align and Translate》
- 【论文阅读】Neural Machine Translation By Jointly Learning To Align and Translate
- [ICLR2015]Neural Machine Translation by Jointly Learning to Align and Translate
- NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE
- NOTES of NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE
- Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
- Machine learning (Regression and neural networks) by Mark K Cowan
- Pattern Recoginition and Machine Learning Intro - Using neural nets to recognize handwritten digits
- Understand Machine Learning Algorithms By Implementing Them From Scratch (and tactics to get around
- 论文阅读:End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for H
- Machine Learning week 4 quiz: programming assignment-Multi-class Classification and Neural Networks
- 机器学习中的神经网络Neural Networks for Machine Learning:Programming assignment 3: Optimization and generaliza
- 论文笔记之:Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic
- Neural Networks and Deep Learning By Michael Nielsen
- 机器学习_论文笔记_1: A few useful things to know about machine learning
- 哈希检索方面的论文、代码和数据集Learning to Hash Paper, Code and Dataset
- Understanding Machine Learning By Shai Shalev-Shwartz and Shai Ben-David
- Introduction to Neural Machine Translation - part 2
- Neural Networks for Machine Learning by Geoffrey Hinton (6)
- ON THE EVOLUTION OF MACHINE LEARNING: FROM LINEAR MODELS TO NEURAL NETWORKS