Transformer is the state-of-the-art model in recent machine translation evaluations. Two strands of research are promising to improve models this kind: first uses wide networks (a.k.a. Transformer-Big) and has been de facto standard for development system, other deeper language representation but faces difficulty arising from learning deep networks. Here, we continue line on latter. We claim that a truly can surpass Transformer-Big counterpart by 1) proper use layer normalization 2) novel...

Spoken Language Understanding (SLU), which typically involves intent determination and slot filling, is a core component of spoken dialogue systems. Joint learning has shown to be effective in SLU given that tags intents are supposed share knowledge with each other. However, most existing joint methods only consider by sharing parameters on surface level rather than semantic level. In this work, we propose novel self-attentive model gate mechanism fully utilize the correlation between...

Multimodal summarization with multimodal output (MSMO) is to generate a summary for news report, which has been proven effectively improve users' satisfaction. The existing MSMO methods are trained by the target of text modality, leading modality-bias problem that ignores quality model-selected image during training. To alleviate this problem, we propose objective function guidance reference use loss from generation and selection. Due lack data, present two strategies, i.e., ROUGE-ranking...

Transformer is the state-of-the-art model in recent machine translation evaluations. Two strands of research are promising to improve models this kind: first uses wide networks (a.k.a. Transformer-Big) and has been de facto standard for development system, other deeper language representation but faces difficulty arising from learning deep networks. Here, we continue line on latter. We claim that a truly can surpass Transformer-Big counterpart by 1) proper use layer normalization 2) novel...

In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information addition individual sentence. this paper, we investigate multi-encoder approaches document-level machine translation (NMT). Surprisingly, find that context encoder does not only encode surrounding sentences but also behaves as a noise generator. This makes us rethink real benefits of context-aware - some improvements come from robust training. We compare several methods introduce...

Sentiment analysis has now become a popular research problem to tackle in NLP field. However, there are very few researches conducted on sentiment for Chinese. Progress is held back due lack of large and labelled corpus powerful models. To remedy this deficiency, we build Chinese Treebank over social data. It concludes 13550 labeled sentences which from movie reviews. Furthermore, introduce novel Recursive Neural Deep Model (RNDM) predict label based recursive deep learning. We consider the...

To improve the classification performance for Chinese short text with automatic semantic feature selection, in this paper we propose Hybrid Attention Networks (HANs) which combines word- and character-level selective attentions. The model firstly applies RNN CNN to extract features of texts. Then it captures class-related attentive representation from features. Finally, all are concatenated fed into output layer classification. Experimental results on 32-class 5-class datasets show that, our...

With the booming of content "re-creation" in social media platforms, character-orientedvideo summary has become a crucial form user-generated video content. However, artificial extraction could be time-consuming with high missing rate, while traditional techniques on person search may incur heavy burden computing resources. At same time, videos are usually accompanied rich textual information, e.g., subtitles or bullet-screen comments which provide multi-view description videos. Thus, there...

Currently, as a basic task of military document information extraction, Named Entity Recognition (NER) for documents has received great attention. In 2020, China Conference on Knowledge Graph and Semantic Computing (CCKS) System Engineering Research Institute Academy Military Sciences (AMS) issued the NER test evaluation, which requires recognition four types entities including Test Elements (TE), Performance Indicators (PI), Components (SC) Task Scenarios (TS). Due to particularity...

In recent years, the generation of conversation content based on deep neural networks has attracted many researchers. However, traditional language models tend to generate general replies, lacking logical and emotional factors. This paper proposes a model that combines reinforcement learning with editing constraints more meaningful customizable replies. The divides replies into three clauses pre-generated keywords uses editor further optimize final reply. multi-task multiple indicator...

Neural architecture search (NAS) has advanced significantly in recent years but most NAS systems restrict to learning architectures of a recurrent or convolutional cell. In this paper, we extend the space NAS. particular, present general approach learn both intra-cell and inter-cell (call it ESS). For better result, design joint method perform simultaneously. We implement our model differentiable system. neural language modeling, outperforms strong baseline on PTB WikiText data, with new...

In secure communication, finding bits that used for steganography is essential effective and high-quality speech information hiding. This paper presents an adaptive hiding method based on K-means, which can quickly find hidden bits. uses K-means algorithm to detect through clustering embeds secret with different matrices achieve good efficiency utilization of proposed according the result matrix coding at transmitting end extracts corresponding embedding ate receiving end. Thereby...

The field of conversation generation using neural networks has attracted increasing attention from researchers for several years. However, traditional language models tend to generate a generic reply with poor semantic logic and no emotion. This article proposes an emotional model based on Bayesian deep network that can replies rich emotions, clear themes, diverse sentences. topic keywords the are pregenerated by introducing commonsense knowledge in model. is divided into multiple clauses,...

Paraphrases are sentences or phrases that convey the same meaning using different words. Paraphrase recognition is of interest for many current Natural Language Processing (NLP) tasks. As understood in linguistics, phenomenon paraphrases difficult to characterize. In this article, we present a novel approach task paraphrase identification. The proposed measures similarity between two based on both lexical and semantic levels, via combining neural networks keywords jointly. particular, employ...

In recent decades, neural network based methods have significantly improved the performance of speech enhancement. Most them estimate time-frequency (T-F) representation target directly or indirectly, then resynthesize waveform using estimated T-F representation. this work, we proposed temporal convolutional recurrent (TCRN), an end-to-end model that map noisy to clean waveform. The TCRN, which is combined convolution and network, able efficiently effectively leverage short-term ang...

Xinze Guo, Chang Liu, Xiaolong Li, Yiran Wang, Guoliang Feng Zhitao Xu, Liuyi Yang, Li Ma, Changliang Li. Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1). 2019.

Topic classification plays an important role in facilitating security-related applications, which can help people reduce data scope and acquire key information quickly. Conversation is one of the ways communication between people. The utterances a conversation may contain vital clues, such as people's opinions, emotions political slants. To explore more effective approaches for Chinese conversational topic classification, this paper, we propose neural network architecture with pre-trained...