Abstract
Sentence-aligned parallel bilingual corpora are the main and sometimes the only required resource for training Statistical and Neural Machine Translation systems (SMT, NMT). We propose an end-to-end deep neural architecture for language independent sentence alignment.
In addition to one-to-one alignment, our aligner can perform cross- and many-to-many alignment as well. We also present a case study which shows how simple linguistic analysis can improve the performance of a pure neural network significantly.
We used three language pairs from Europarl corpus (Koehn, 2005) and an English-Persian corpus (Pilevar et al., 2011) to generate an alignment dataset. Using this dataset, we tested our system individually and in an SMT system.
In both settings, we obtained significantly better results compared to an open source baseline.