The WMT'18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English

Franck Burlot (1) , Yves Scherrer (2) , Vinit Ravishankar (3) , Ondřej Bojar (3) , Stig-Arne Grönroos (4) , Maarit Koponen (5) , Tommi Nieminen (2) , François Yvon (1)

1 TLP - Traitement du Langage Parlé
2 Helsingin yliopisto = Helsingfors universitet = University of Helsinki
3 UK - Univerzita Karlova [Praha, Česká republika] = Charles University [Prague, Czech Republic]
4 Aalto University
5 University of Turku

Progress in the quality of machine translation output calls for new automatic evaluation procedures and metrics. In this paper, we extend the Morpheval protocol introduced by \citet{burlot-yvon-wmt2017} for the English-to-Czech and English-to-Latvian translation directions to three additional language pairs, and report its use to analyze the results of WMT 2018's participants for these language pairs. Considering additional, typologically varied source and target languages also enables us to draw some generalizations regarding this morphology-oriented evaluation procedure.

Mots clés: Machine translation; Machine Translation evaluation; Morphology

Domaines: Informatique et langage [cs.CL]

Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ondřej Bojar, Stig-Arne Grönroos, et al.. The WMT'18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English. 3rd Conference on Machine Translation (WMT 18), Oct 2018, Bruxelles, Belgium. pp.550-564, ⟨10.18653/v1/W18-64060⟩. ⟨hal-01910244⟩