How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data? Existing techniques often attempt transfer powerful machine (MT) capabilities ST, but neglect the discrepancy across modalities. In this paper, we propose Speech-TExt Manifold Mixup (STEMM) method calibrate such discrepancy. Specifically, mix up sequences of different modalities, and take both unimodal multimodal mixed as input model in parallel, regularize their output...

How can we learn unified representations for spoken utterances and their written text? Learning similar semantically speech text is important translation. To this end, propose ConST, a cross-modal contrastive learning method end-to-end speech-to-text We evaluate ConST variety of previous baselines on popular benchmark MuST-C. Experiments show that the proposed consistently outperforms methods, achieves an average BLEU 29.4. The analysis further verifies indeed closes representation gap...

Although sequence-to-sequence attentional neural machine translation (NMT) has achieved great progress recently, it is confronted with two challenges: learning optimal model parameters for long parallel sentences and well exploiting different scopes of contexts. In this paper, partially inspired by the idea segmenting a sentence into short clauses, each which can be easily translated NMT, we propose hierarchy-to-sequence NMT to handle these challenges. Our encoder takes segmented clause...

Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu. Proceedings of the 53rd Annual Meeting Association for Computational Linguistics and 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

We propose to enhance the RNN decoder in a neural machine translator (NMT) with external memory, as natural but powerful extension state decoding RNN.This memory-enhanced is called MEMDEC.At each time during decoding, MEMDEC will read from this memory and write once, both content-based addressing.Unlike unbounded previous work (Bahdanau et al., 2014) store representation of source sentence, matrix predetermined size designed better capture information important for process at step.Our...

Photopolymers hold great promise for the preparation of transparent volume holographic gratings (VHG), which are core optical elements in many application fields. To improve recording property a two-stage photopolymer, four new (meth)acrylate monomers (CTA, CTMA, CTBA, CTBMA) with high refractive indices (1.59–1.63) designed and synthesized this study. Using them as one writing monomer, series photopolymer samples different formulations thicknesses fabricated recording. Among them,...

The Ni-rich LiNi

Deep Neural Networks (DNNs) have provably enhanced the state-of-the-art Machine Translation (NMT) with its capability in modeling complex functions and capturing linguistic structures. However NMT deep architecture encoder or decoder RNNs often suffer from severe gradient diffusion due to non-linear recurrent activations, which makes optimization much more difficult. To address this problem we propose a novel linear associative units (LAU) reduce propagation path inside unit. Different...

The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language with a heuristically chosen source context window, achieving state-of-the-art performance in SMT. In this paper, we give more systematic treatment by summarizing relevant information through convolutional architecture guided information. With different guiding signals during decoding, our specifically designed convolution+gating architectures can pinpoint parts of sentence that...

This paper proposes three distortion models to explicitly incorporate the word reordering knowledge into attention-based Neural Machine Translation (NMT) for further improving translation performance. Our proposed enable attention mechanism attend source words regarding both semantic requirement and penalty. Experiments on Chinese-English show that approaches can improve alignment quality achieve significant improvements over a basic NMT by large margins. Compared with previous works...

In this study, we first investigate a novel capsule network with dynamic routing for linear time Neural Machine Translation (NMT), referred as \textsc{CapsNMT}. \textsc{CapsNMT} uses an aggregation mechanism to map the source sentence into matrix pre-determined size, and then applys deep LSTM decode target sequence from representation. Unlike previous work \cite{sutskever2014sequence} store passive bottom-up way, policy encodes iterative process decide credit attribution between nodes lower...

Metal compounds play important roles in the formation of organic pollutants during thermal-related processes. However, metal-catalyzed predominant have not previously been characterized nor any detailed catalytic mechanisms clarified. Here, we preciously distinguished multiple free radical intermediates on metal catalyst surfaces pollutant through laboratory and theoretical studies. Differences between intermediate species, concentrations, under catalysis different were investigated. The...

How can speech-to-text translation (ST) perform as well machine (MT)? The key point is to bridge the modality gap between speech and text so that useful MT techniques be applied ST.Recently, approach of representing with unsupervised discrete units yields a new way ease problem. This motivates us propose Discrete Unit Back-translation(DUB) answer two questions (1) Is it better represent than continuous features in direct ST? (2) much benefit bring With DUB, back-translation technique...

Many tasks in natural language processing, ranging from machine translation to question answering, can be reduced the problem of matching two sentences or more generally short texts. We propose a new approach problem, called Deep Match Tree (DeepMatch$_{tree}$), under general setting. The consists components, 1) mining algorithm discover patterns for short-texts, defined product space dependency trees, and 2) deep neural network texts using mined patterns, as well learning build having...

Chlorinated organic chemicals are produced and used extensively worldwide, their risks to the biology environment of increasing concern. However, chlorinated byproducts [e.g., polychlorinated dibenzo-p-dioxins dibenzofurans (PCDD/Fs)] formed during commercial manufacturing processes present in organochlorine products rarely reported. The knowledge on occurrences fate unintentional persistent chemical is necessary for accurate assessment production. Here, PCDD/Fs were tracked throughout...