Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, Xudong Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint (EMNLP-IJCNLP). 2019.

MixUp (Zhang et al. 2017) is a recently proposed dataaugmentation scheme, which linearly interpolates random pair of training examples and correspondingly the one-hot representations their labels. Training deep neural networks with such additional data shown capable significantly improving predictive accuracy current art. The power MixUp, however, primarily established empirically its working effectiveness have not been explained in any depth. In this paper, we develop an understanding for...

Mixup, a recent proposed data augmentation method through linearly interpolating inputs and modeling targets of random samples, has demonstrated its capability significantly improving the predictive accuracy state-of-the-art networks for image classification. However, how this technique can be applied to what is effectiveness on natural language processing (NLP) tasks have not been investigated. In paper, we propose two strategies adaption Mixup sentence classification: one performs...

In human conversations, individuals can indicate relevant regions within a scene while addressing others. turn, the other person then respond by referring to specific if necessary. This natural referential ability in dialogue remains absent current Multimodal Large Language Models (MLLMs). To fill this gap, paper proposes an MLLM called Shikra, which handle spatial coordinate inputs and outputs language. Its architecture consists of vision encoder, alignment layer, LLM. It is designed be...

Few-shot text classification has recently been promoted by the meta-learning paradigm which aims to identify target classes with knowledge transferred from source sets of small tasks named episodes. Despite their success, existing works building meta-learner based on Prototypical Networks are unsatisfactory in learning discriminative representations between similar classes, may lead contradictions during label prediction. In addition, task-level and instance-level overfitting problems...

Contrastive learning has achieved remarkable success in representation via self-supervision unsupervised settings. However, effectively adapting contrastive to supervised tasks remains as a challenge practice. In this work, we introduce dual (DualCL) framework that simultaneously learns the features of input samples and parameters classifiers same space. Specifically, DualCL regards augmented associating different labels then exploits between samples. Empirical studies on five benchmark text...

Unsupervised sentence representation learning is a fundamental problem in natural language processing. Recently, contrastive has made great success on this task. Existing constrastive based models usually apply random sampling to select negative examples for training. Previous work computer vision shown that hard help achieve faster convergency and better optimization learning. However, the importance of negatives yet be explored. In study, we prove are essential maintaining strong gradient...

Mashing up Web services and RESTful APIs is a novel programming approach to develop new applications. As the number of available resources increasing rapidly, discover potential or getting difficult. Therefore, it vital relieve mashup developers burden service discovery. In this paper, we propose probabilistic model assist creators by recommending list that may be used compose required given descriptions mashup. Specifically, relational topic exploited characterize relationship among...

The knowledge base (KB) completion problem is usually formulated as a link prediction problem. Such formulation incapable of capturing certain application scenarios when the KB contains multi-fold relations. In this paper, we present new completion, called instance reconstruction. Unlike its link-prediction counterpart, which has linear complexity in size KB, behave high-degree polynomial. This presents significant challenge developing scalable reconstruction algorithms. novel embedding...

Multitask learning has shown promising performance in multiple related tasks simultaneously, and variants of model architectures have been proposed, especially for supervised classification problems. One goal multitask is to extract a good representation that sufficiently captures the relevant part input about output each task. To achieve this objective, paper we design architecture based on observation correlations exist between outputs some (e.g. entity recognition relation extraction...

Web service discovery is a vital problem in computing with the increasing number of services. Existing approaches merely focus on WSDL-based keyword search, semantic matching based domain knowledge or ontologies, QoS-based recommendations. The search omits underlying correlations and QoS information not always available. In this paper, we propose probabilistic approach to help web users retrieve related services improve performance. Specifically, apply model characterize latten topics...

The models developed to date for knowledge base embedding are all based on the assumption that relations contained in bases binary. For training and testing of these models, multi-fold (or n-ary) relational data converted triples (e.g., FB15K dataset) interpreted as instances binary relations. This paper presents a canonical representation containing We show existing popular datasets correspond sub-optimal modelling framework, resulting loss structural information. advocate novel which...

A large majority of approaches have been proposed to leverage the dependency tree in relation classification task. Recent works focused on pruning irrelevant information from tree. The state-of-the-art Attention Guided Graph Convolutional Networks (AGGCNs) transforms into a weighted-graph distinguish relevance nodes and edges for classification. However, their approach, graph is fully connected, which destroys structure original How effectively make use relevant while ignoring trees remains...

We propose a novel strategy to encode the syntax parse tree of sentence into learnable distributed representation. The proposed encoding scheme is provably information-lossless. In specific, an embedding vector constructed for each word in sentence, path corresponding word. one-to-one correspondence between these "syntax-embedding" vectors and words (hence their vectors) makes it easy integrate such representation with all word-level NLP models. empirically show benefits embeddings on...

Aspect-level sentiment classification (ALSC) aims at predicting the polarity of a specific aspect term occurring in sentence. This task requires learning representation by aggregating relevant contextual features concerning term. Existing methods cannot sufficiently leverage syntactic structure sentence, and hence are difficult to distinguish different sentiments for multiple aspects We perceive limitations previous propose hypothesis about finding crucial information with help structure....

Effective regularization techniques are highly desired in deep learning for alleviating overfitting and improving generalization. This work proposes a new scheme, based on the understanding that flat local minima of empirical risk cause model to generalize better. scheme is referred as adversarial perturbation (AMP), where instead directly minimizing risk, an alternative "AMP loss" minimized via SGD. Specifically, AMP loss obtained from by applying "worst" norm-bounded each point parameter...

In the context of green economy, Closed-Loop Supply Chain (CLSC) competition is intensifying. This study aims to help companies operating in supply chains determine optimal remanufacturing strategies when competing with other firms. We examine decision-making problem CLSCs competitive environments facing multiple mode options. The research constructs a Prisoner’s dilemma model for dual CLSCs, where each chain has three strategic choices: independent remanufacturing, outsourced and authorized...

Supervised multimodal classification has been proven to outperform unimodal in the image-text domain. However, this task is highly dependent on abundant labeled data. To perform data-insufficient scenarios, study, we explore semi-supervised (SSMC) that only requires a small amount of data and plenty unlabeled Specifically, first design baseline SSMC models by combining known semi supervised pseudo-labeling methods with two most commonly used modal fusion strategies, i.e. feature-level...

Word order difference between source and target languages is a major obstacle to cross-lingual transfer, especially in the dependency parsing task. Current works are mostly based on order-agnostic models or word reordering mitigate this problem. However, such methods either do not leverage grammatical information naturally contained computationally expensive as permutation space grows exponentially with sentence length. Moreover, reordered an unnatural may be form of noising that harms model...

Weakly supervised phrase grounding tasks aim to learn alignments between phrases and regions with coarse image-caption match information. One branch of previous methods established pseudo-label relationships based on the Expectation-Maximization (EM) algorithm combined contrastive learning. However, adopting a simplified batch-level local update (partial) pseudo-labels in E-step is sub-optimal, while extending it global requires inefficiently numerous computations. In addition, their failure...

Web services, as loosely-coupled software systems, are increasingly being published to the web and there a large number of services with similar functions. Therefore, service users compare non-functional properties e.g., Quality Service (QoS), when they make selection. This paper aims at generating more comprehensive recommendation novel approach fulfill accurate prediction unknown services' QoS values. We accomplish by using fuzzy clustering method calculating users' similarity. Our...