Semi-supervised Structured Prediction with Neural CRF Autoencoder

• DOI: 10.18653/v1/d17-1179
• Publication Date: 2018-01-18T11:54:31Z
• Authors: Xiao Zhang; Yong Jiang; Hao Peng; Kewei Tu; Dan Goldwasser
• Keywords: 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

In this paper we propose an end-to-end neural CRF autoencoder (NCRF-AE) model for semi-supervised learning of sequential structured prediction problems. Our NCRF-AE consists of two parts: an encoder which is a CRF model enhanced by deep neural networks, and a decoder which is a generative model trying to reconstruct the input. Our model has a unified structure with different loss functions for labeled and unlabeled data with shared parameters. We developed a variation of the EM algorithm for optimizing both the encoder and the decoder simultaneously by decoupling their parameters. Our experimental results over the Part-of-Speech (POS) tagging task on eight different languages, show that the NCRF-AE model can outperform competitive systems in both supervised and semi-supervised scenarios. © 2017 Association for Computational Linguistics.