Abstract Background In recent years, deep learning methods have been applied to many natural language processing tasks achieve state-of-the-art performance. However, in the biomedical domain, they not out-performed supervised word sense disambiguation (WSD) based on support vector machines or random forests, possibly due inherent similarities of medical senses. Results this paper, we propose two deep-learning-based models for WSD: a model bi-directional long short-term memory (BiLSTM)...

The success of language models based on the Transformer architecture appears to be inconsistent with observed anisotropic properties representations learned by such models. We resolve this showing, contrary previous studies, that do not occupy a narrow cone, but rather drift in common directions. At any training step, all embeddings except for ground-truth target embedding are updated gradient same direction. Compounded over set, and share components, manifested their shape we have...

To improve the generalization of representations for natural language processing tasks, words are commonly represented using vectors, where distances among vectors related to similarity words. While word2vec, state-of-the-art implementation skip-gram model, is widely used and improves performance many its mechanism not yet well understood. In this work, we derive learning rules model establish their close relationship competitive learning. addition, provide global optimal solution...

We explore the idea of using pre-trained BERT as a source factual knowledge, analyze which components model are responsible for its ability to answer questions requiring and study transferability knowledge downstream tasks. Our experiments show that Language Modeling Head is indispensable predicting facts, implying any captured in limited. While dominant approach researching how stored language models focuses on tailoring question formulation optimize retrieval quality, we find patterns...

Predicting the risk of mortality for patients with acute myocardial infarction (AMI) using electronic health records (EHRs) data can help identify risky who might need more tailored care. In our previous work, we built computational models to predict one-year admitted an intensive care unit (ICU) AMI or post syndrome. Our prior work only used structured clinical from MIMIC-III, a publicly available ICU database. this study, enhanced by adding word embedding features free-text discharge...

Predicting the risk of mortality for patients with acute myocardial infarction (AMI) using electronic health records (EHRs) data can help identify risky who might need more tailored care. In our previous work, we built computational models to predict one-year admitted an intensive care unit (ICU) AMI or post syndrome. Our prior work only used structured clinical from MIMIC-III, a publicly available ICU database. this study, enhanced by adding word embedding features free-text discharge...

In this paper, we propose a novel deep neural network architecture for supervised medical word sense disambiguation. Our is based on layered bidirectional LSTM network, upon which max-pooling along multiple time steps are performed so that dense representation of the context created. addition, introduced four different adjustments to output in order find most suitable input form layer. Results show best model outperforms current state-of-the-art MSH WSD dataset. Moreover, also train an...

Natural language processing has improved substantially in the last few years due to increased computational power and availability of text data. Bidirectional Encoder Representations from Transformers (BERT) have further performance by using an auto-encoding model that incorporates larger bidirectional contexts. However, underlying mechanisms BERT for its effectiveness are not well understood. In paper we investigate how architecture pretraining protocol affect geometry embeddings features...

Unexpected responses or repeated clarification questions from conversational agents detract the users’ experience with technology meant to streamline their daily tasks. To reduce these frictions, Query Rewriting (QR) techniques replace transcripts of faulty queries alternatives that lead thatsatisfy needs. Despite successes, existing QR approaches are limited in ability fix require considering personal preferences.We improve by proposing Personalized Adaptive Interactions Graph Encoder...