Skip to main navigation Skip to search Skip to main content

Alignment-enhanced transformer for constraining NMT with pre-specified translations

  • Kai Song
  • , Kun Wang
  • , Heng Yu
  • , Yue Zhang
  • , Zhongqiang Huang
  • , Weihua Luo
  • , Xiangyu Duan
  • , Min Zhang
  • Soochow University
  • Alibaba Group Holding Ltd.
  • Westlake University

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We investigate the task of constraining NMT with pre-specified translations, which has practical significance for a number of research and industrial applications. Existing works impose pre-specified translations as lexical constraints during decoding, which are based on word alignments derived from target-to-source attention weights. However, multiple recent studies have found that word alignment derived from generic attention heads in the Transformer is unreliable. We address this problem by introducing a dedicated head in the multi-head Transformer architecture to capture external supervision signals. Results on five language pairs show that our method is highly effective in constraining NMT with pre-specified translations, consistently outperforming previous methods in translation quality.

Original languageEnglish
Title of host publicationAAAI 2020 - 34th AAAI Conference on Artificial Intelligence
PublisherAAAI press
Pages8886-8893
Number of pages8
ISBN (Electronic)9781577358350
StatePublished - 2020
Externally publishedYes
Event34th AAAI Conference on Artificial Intelligence, AAAI 2020 - New York, United States
Duration: 7 Feb 202012 Feb 2020

Publication series

NameAAAI 2020 - 34th AAAI Conference on Artificial Intelligence

Conference

Conference34th AAAI Conference on Artificial Intelligence, AAAI 2020
Country/TerritoryUnited States
CityNew York
Period7/02/2012/02/20

Fingerprint

Dive into the research topics of 'Alignment-enhanced transformer for constraining NMT with pre-specified translations'. Together they form a unique fingerprint.

Cite this