TY - GEN
T1 - Enriching SMT Training Data via Paraphrasing
AU - He, Wei
AU - Zhao, Shiqi
AU - Wang, Haifeng
AU - Liu, Ting
N1 - Publisher Copyright:
© 2011 AFNLP
PY - 2011
Y1 - 2011
N2 - This paper proposes a novel method to resolve the coverage problem of SMT system. The method generates paraphrases for source-side sentences of the bilingual parallel data, which are then paired with the target-side sentences to generate new parallel data. Within a statistical paraphrase generation framework, we employ an object function, named Sentence Novelty, to select paraphrases which having the most novel information to the bilingual training corpus of the SMT model. Meanwhile, the context is considered via a language model in the source language to ensure the fluency and accuracy of paraphrase substitution. Compared to a state-of-the-art phrase based SMT system (Moses), our method achieves an improvement of 1.66 points in terms of BLEU on a small training corpus which simulates a resource-poor environment, and 1.06 points on a training corpus of medium size.
AB - This paper proposes a novel method to resolve the coverage problem of SMT system. The method generates paraphrases for source-side sentences of the bilingual parallel data, which are then paired with the target-side sentences to generate new parallel data. Within a statistical paraphrase generation framework, we employ an object function, named Sentence Novelty, to select paraphrases which having the most novel information to the bilingual training corpus of the SMT model. Meanwhile, the context is considered via a language model in the source language to ensure the fluency and accuracy of paraphrase substitution. Compared to a state-of-the-art phrase based SMT system (Moses), our method achieves an improvement of 1.66 points in terms of BLEU on a small training corpus which simulates a resource-poor environment, and 1.06 points on a training corpus of medium size.
UR - https://www.scopus.com/pages/publications/84878197443
M3 - 会议稿件
AN - SCOPUS:84878197443
T3 - IJCNLP 2011 - Proceedings of the 5th International Joint Conference on Natural Language Processing
SP - 803
EP - 810
BT - IJCNLP 2011 - Proceedings of the 5th International Joint Conference on Natural Language Processing
A2 - Wang, Haifeng
A2 - Yarowsky, David
PB - Association for Computational Linguistics (ACL)
T2 - 5th International Joint Conference on Natural Language Processing, IJCNLP 2011
Y2 - 8 November 2011 through 13 November 2011
ER -