How does the post-editing of Neural Machine Translation compare with from-scratch translation? A product and process study

Abstract

This study explores the post-editing process when working within the newly introduced neural machine translation (NMT) paradigm. To this end, an experiment was carried out to examine the differences between post-editing Google neural machine translation (GNMT) and from-scratch translation of English domain-specific and general language texts to Chinese. We analysed translation process and translation product data from 30 first-year postgraduate translation students. The analysis is based on keystroke logging, screen recording, questionnaires, retrospective protocols and target-text quality evaluations. The three main findings were: 1) post-editing GNMT was only significantly faster than from-scratch translation for domain-specific texts, but it significantly reduced the participants' cognitive effort for both text types; 2) post-editing GNMT generated translations of equivalent fluency and accuracy as those generated by from-scratch translations; 3) the student translators generally showed a positive attitude towards post-editing, but they also pointed to various challenges in the post-editing process. These were mainly due to the influence of their previous translation training, lack of experience in post-editing and the ambiguous wording of the post-editing guidelines.