Fast Structured Decoding for Sequence Models

• DOI: 10.48550/arxiv.1910.11555
• Publication Date: 2019-01-01
• Authors: Zhiqing Sun; Zhuohan Li; Haoqing Wang; Zi Lin; Di He; Zhihong Deng
• Keywords: Sequence (biology)

Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these suffer from heavy latency during inference. Recently, non-autoregressive were proposed reduce inference time. assume that decoding process of each token is conditionally independent others. Such a generation sometimes makes output sentence inconsistent, and thus learned could only inferior accuracy compared their counterparts. To improve then consistency cost at same time, we propose incorporate structured module into models. Specifically, design an efficient approximation for Conditional Random Fields (CRF) models, further dynamic transition technique model positional contexts CRF. Experiments translation show while increasing little (8~14ms), our significantly better than previous on different datasets. In particular, WMT14 En-De dataset, obtains BLEU score 26.80, which largely outperforms baselines 0.61 lower purely