TensorFlow: A system for large-scale machine learning
- Martín AbadiP. Barham Xiaoqiang Zhang
- 27 May 2016
Computer Science
The TensorFlow dataflow model is described and the compelling performance that TensorFlow achieves for several real-world applications is demonstrated.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Martín AbadiAshish Agarwal Xiaoqiang Zheng
- 14 March 2016
Computer Science, Engineering
The TensorFlow interface and an implementation of that interface that is built at Google are described, which has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields.
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Yonghui WuM. Schuster J. Dean
- 26 September 2016
Computer Science, Linguistics
GNMT, Google's Neural Machine Translation system, is presented, which attempts to address many of the weaknesses of conventional phrase-based translation systems and provides a good balance between the flexibility of "character"-delimited models and the efficiency of "word"-delicited models.
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
- Jonathan ShenRuoming Pang Yonghui Wu
- 16 December 2017
Computer Science
This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps…
Tacotron: Towards End-to-End Speech Synthesis
- Yuxuan WangR. Skerry-Ryan R. Saurous
- 29 March 2017
Computer Science
Tacotron is presented, an end-to-end generative text- to-speech model that synthesizes speech directly from characters that achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
- Melvin JohnsonM. Schuster Jeffrey Dean
- 14 November 2016
Computer Science, Linguistics
This work proposes a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages using a shared wordpiece vocabulary, and introduces an artificial token at the beginning of the input sentence to specify the required _target language.
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
- Yanping HuangYonglong Cheng Z. Chen
- 16 November 2018
Computer Science, Engineering
GPipe is introduced, a pipeline parallelism library that allows scaling any network that can be expressed as a sequence of layers by pipelining different sub-sequences of layers on separate accelerators, resulting in almost linear speedup when a model is partitioned across multiple accelerators.
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Dmitry LepikhinHyoukJoong Lee Z. Chen
- 30 June 2020
Computer Science
GShard enabled us to scale up multilingual neural machine translation Transformer model with Sparsely-Gated Mixture-of-Experts beyond 600 billion parameters using automatic sharding and it is demonstrated that such a giant model can efficiently be trained on 2048 TPU v3 accelerators in 4 days to achieve far superior quality for translation from 100 languages to English compared to the prior art.
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
- H. ZenViet Dang Yonghui Wu
- 5 April 2019
Computer Science, Linguistics
Experimental results show that neural end-to-end TTS models trained from the LibriTTS corpus achieved above 4.0 in mean opinion scores in naturalness in five out of six evaluation speakers.
Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Ye JiaYu Zhang Yonghui Wu
- 1 June 2018
Computer Science
It is shown that randomly sampled speaker embeddings can be used to synthesize speech in the voice of novel speakers dissimilar from those used in training, indicating that the model has learned a high quality speaker representation.
...
...