Translation quality estimation is an important issue in machine translation, which has been attracting increasing interest from researchers, aiming at finding a good estimator for the quality of machine translation output.
The common approach for quality estimation is to treat the problem as a supervised regression/classification task using a quality-annotated noisy parallel corpus, called quality estimation data, as training data. However, the available size of quality estimation data remains small, due to the too expensive cost of creating such data. In addition, most conventional quality estimation approaches rely on manually designed features to model nonlinear relationships between feature vectors and corresponding quality labels.
To overcome these problems, this thesis proposes a novel neural network architecture for quality estimation task -- called the Predictor-Estimator -- that considers word prediction as an additional pre-task. The major component of the proposed neural architecture is a word prediction model based on a modified neural machine translation model -- a probabilistic model for predicting a target word conditioned on all the other source and target contexts. The underlying assumption is that the word prediction model is highly related to quality estimation models, and is therefore able to transfer useful knowledge to quality estimation tasks. Our proposed quality estimation method trains the following two types of neural models: 1) Predictor: Neural word prediction model trained from parallel corpora and 2) Estimator: Neural quality estimation model trained from quality estimation data. To transfer word prediction task to quality estimation task, we generate quality estimation feature vectors from the word prediction model and feed them into the quality estimation model.
Our proposed Predictor-Estimator has a potential problem: how to learn efficiently two-stage Predictor-Estimator model, because the Predictor-Estimator model is comprised of stacked main tasks (word prediction and quality estimation tasks) and parallel multi-level sub-tasks (sentence, word and phrase-level quality estimation sub-tasks). This thesis also proposes an efficient multi-task learning method for Predictor-Estimator model.
The experimental results on WMT17 quality estimation datasets indicated that our proposed method has potential as it achieves state-of-the-art performances for all subtasks (sentence, word, and phrase levels).