Tree-to-Sequence Attentional Neural Machine Translation

@article{Eriguchi2016TreetoSequenceAN,
  title={Tree-to-Sequence Attentional Neural Machine Translation},
  author={Akiko Eriguchi and Kazuma Hashimoto and Yoshimasa Tsuruoka},
  journal={ArXiv},
  year={2016},
  volume={abs/1603.06075},
  url={https://api.semanticscholar.org/CorpusID:12851711}
}
This work proposes a novel end-to-end syntactic NMT model, extending a sequence- to-sequence model with the source-side phrase structure, which has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence.
View on ACL
aclweb.org
266 Citations
Highly Influential Citations
25
Background Citations
116
Methods Citations
115
Results Citations
4

Figures and Tables from this paper

266 Citations

Sequence-to-Dependency Neural Machine Translation

A novel Sequence-to-Dependency Neural Machine Translation (SD-NMT) method, in which the _target word sequence and its corresponding dependency structure are jointly constructed and modeled, and this structure is used as context to facilitate word generations.

Incorporating Source-Side Phrase Structures into Neural Machine Translation

This model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence, and is called a tree-to-sequence NMT model, extending a sequence- to-sequence model with the source-side phrase structure.

Incorporating Source-Side Phrase Structures into Neural Machine Translation

This model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence, and is called a tree-to-sequence NMT model, extending a sequence- to-sequence model with the source-side phrase structure.

Incorporating Syntactic Uncertainty in Neural Machine Translation with a Forest-to-Sequence Model

A forest-to-sequence NMT model to make use of exponentially many parse trees of the source sentence to compensate for the parser errors, and represents the collection of parse trees as a packed forest, and learns a neural transducer to translate from the input forest to the _target sentence.

Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder

This paper proposes a bidirectional tree encoder which learns both sequential and tree structured representations; a tree-coverage model that lets the attention depend on the source-side syntax and experiments demonstrate that the proposed models outperform the sequential attentional model.

Neural Machine Translation with Phrasal Attention

The proposed phrasal attention framework is simple yet effective, keeping the strength of phrase-based statistical machine translation (SMT) on the source side and able to statistically improve word-level attention-based NMT.

Neural Machine Translation with Source-Side Latent Graph Parsing

This paper presents a novel neural machine translation model which jointly learns translation and source-side latent graph representations of sentences which significantly outperforms the previous best models on the standard English-to-Japanese translation dataset.

A Hierarchy-to-Sequence Attentional Neural Machine Translation Model

A hierarchy-to-sequence attentional NMT model to handle segmenting a long sentence into short clauses, each of which can be easily translated by NMT, which can not only improve parameter learning, but also well explore different scopes of contexts for translation.

Incorporating Syntactic Uncertainty in Neural Machine Translation withForest-to-seuqence Mode

A forest-to-sequence Attentional Neural Machine Translation model which uses a forest instead of the 1-best tree to compensate parsing errors and demonstrates superiority of the method over the treeto-sequence and vanilla Attentional neural Machine Translation models.

Character-based Decoding in Tree-to-Sequence Attention-based Neural Machine Translation

This paper reports the systems (UT-AKY) submitted in the 3rd Workshop of Asian Translation 2016 and their results in the English-to-Japanese translation task, confirming that the character-based decoder can cover almost the full vocabulary in the _target language and generate translations much faster than the word-based model.
...

37 References

Evaluating Neural Machine Translation in English-Japanese Task

A simple workaround to find missing translations with a back-off sys-tem is described and a specific error pattern in NMT translations which omits some information and thus fail to preserve the complete meaning is demonstrated.

Agreement-Based Joint Training for Bidirectional Attention-Based Neural Machine Translation

This work proposes agreement-based joint training for bidirectional attention-based end-to-end neural machine translation, which encourages the two complementary models to agree on word alignment matrices on the same training data.

Neural Machine Translation by Jointly Learning to Align and Translate

It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a _target word, without having to form these parts as a hard segment explicitly.

Addressing the Rare Word Problem in Neural Machine Translation

This paper proposes and implements an effective technique to address the problem of end-to-end neural machine translation's inability to correctly translate very rare words, and is the first to surpass the best result achieved on a WMT’14 contest task.

Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation

A way to address the issue of a significant drop in translation quality when translating long sentences by automatically segmenting an input sentence into phrases that can be easily translated by the neural network translation model.

Effective Approaches to Attention-based Neural Machine Translation

A global approach which always attends to all source words and a local one that only looks at a subset of source words at a time are examined, demonstrating the effectiveness of both approaches on the WMT translation tasks between English and German in both directions.

Incorporating Structural Alignment Biases into an Attentional Neural Translation Model

The attentional neural translation model is extended to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions.

Sequence to Sequence Learning with Neural Networks

This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the _target sentence which made the optimization problem easier.

Recurrent Continuous Translation Models

We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences…

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

It is shown that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase.

Related Papers

Showing 1 through 3 of 0 Related Papers