Application and analysis of string-similarity-based machine translation evaluation

To help developing a localization-oriented example based machine translation (EBMT) system, an automatic machine translation evaluation method is implemented which adopts sentence similarity as criteria. Experiment shows that the evaluation method distinguishes well between good translations and bad ones. To verify the consistency between automatic and human evaluation methods, 6 machine translation systems are scored using both methods and the evaluation results are compared. Correlation coefficient and significance tests are made to ensure the reliability of the results. Linear regression equations are calculated to map the automatic scoring results to human scorings, which can be utilized to predict human scoring of machine translation systems.