Fast Neural Machine Translation in C++
Basic example for training: The scripts for training a Edinburgh’s WMT16 system adapted from the Romanian-English sample from https://github.com/rsennrich/wmt16-scripts. The resulting system should be competitive or even slightly better than reported in that paper.
Training a transformer model: An example for training a Google-style transformer model introduced in Attention is all you need, Vaswani et al., 2017.
Training on raw texts with built-in SentencePiece: The example shows how to use Taku Kudo’s SentencePiece and Matt Post’s SacreBLEU to greatly simplify the training and evaluation process by providing ways to have reversible hidden preprocessing and repeatable evaluation.
Reconstructing Edinburgh’s WMT17 English-German system: The scripts show how to train a complete WMT-grade system based on Edinburgh’s WMT submission description for en-de.
Reconstructing top WMT17 system with Marian’s Transformer model: The scripts show how to train a complete better than (!) WMT-grade system based on Google’s Transformer model and Edinburgh’s WMT submission description for en-de. This example is a combination of reproducing Edinburgh’s WMT2017 system for en-de with Marian and the example for Transformer training.
Translating with Amun: The scripts demonstrate how to translate with Amun using Edinburgh’s German-English WMT2016 single model and ensemble.
The training with Marian guide is an overview aimed at new practitioners to machine translation. It covers the model training pipeline in general, and provides a foundation from which to better comprehend the other examples and tutorials presented here.
The Machine Translation Marathon 2019 Tutorial shows how to do efficient neural machine translation using the Marian toolkit by optimizing the speed, accuracy and use of resources for training and decoding of NMT models.
The Machine Translation Marathon 2018 Labs is a Marian tutorial that covers topics like downloading and compiling Marian, translating with a pretrained model, preparing training data and training a basic NMT model, and contains list of exercises introducing different features and model architectures available in Marian.