Variable Mini-Batch Sizing and Pre-Trained Embeddings

Abstract

This paper describes our submission to the WMT 2017 Neural MT Training Task. We modified the provided NMT system in order to allow for interrupting and con- tinuing the training of models.

This al- lowed mid-training batch size decremen- tation and incrementation at variable rates. In addition to the models with variable batch size, we tried different setups with pre-trained word2vec embeddings.

Aside from batch size incrementation, all our ex- periments performed below the baseline

Keywords

• variable
• mini
• batch
• sizing
• trained
• embeddings