Publication at Faculty of Mathematics and Physics |
2012
Abstract
This paper describes the creation process of an Indonesian-English parallel corpus (IDENTIC). The corpus contains 45,000 sentences collected from different sources in different genres.
Several manual text preprocessing tasks, such as alignment and spelling correction, are applied to the corpus to assure its quality. We also apply language specific text processing such as tokenization on both sides and clitic normalization on the Indonesian side.
The corpus is available in two different formats: ‘plain’, stored in text format and ‘morphologically enriched’, stored in CoNLL format. Some parts of the corpus are publicly available at the IDENTIC homepage.