Quality and Machine Translation: A realistic objective?

Abstract

Is Machine Translation (MT) output necessarily of lower quality than human translation? Taking a user guide in English as the source text, German as the target language, and IBM WebSphere as the MT system, we endeavour to answer this question. Eleven suitably qualified raters rated 30 source sentences, three translated versions and three post-edited versions for the parameters clarity, accuracy and style. The results show that the machine translated, post-edited output was judged to be of higher clarity and accuracy, while the translations were judged to be of better style. When asked to pick their “favourite” sentence, the majority of the evaluators chose translated (as opposed to machine translated) sentences. Further, sentences where Controlled Language (CL) rules had been applied scored higher on clarity and accuracy, adding further to the claim that the application of CL rules improves MT output.