A5085563247- Topic Modeling
- Natural Language Processing Techniques
- Advanced Text Analysis Techniques
- Text Readability and Simplification
- Multimodal Machine Learning Applications
- Artificial Intelligence in Healthcare and Education
- Domain Adaptation and Few-Shot Learning
- Educational Technology and Assessment
- Intelligent Tutoring Systems and Adaptive Learning
- Human Pose and Action Recognition
- Generative Adversarial Networks and Image Synthesis
- Chronic Disease Management Strategies
- Speech Recognition and Synthesis
- Image Enhancement Techniques
- Biomedical Text Mining and Ontologies
- Anomaly Detection Techniques and Applications
- Advanced Neural Network Applications
- Web Data Mining and Analysis
- Nutrition and Health in Aging
- Epigenetics and DNA Methylation
- Speech and dialogue systems
- Context-Aware Activity Recognition Systems
- Online Learning and Analytics
- Machine Learning and Data Classification
- Handwritten Text Recognition Techniques
Zhejiang Chinese Medical University
2025
University of Science and Technology of China
2020-2025
Baidu (China)
2023
China Tourism Academy
2022
Entities, as the essential elements in relation extraction tasks, exhibit certain structure. In this work, we formulate such entity structure distinctive dependencies between mention pairs. We then propose SSAN, which incorporates these structural within standard self-attention mechanism and throughout overall encoding stage. Specifically, design two alternative transformation modules inside each building block to produce attentive biases so adaptively regularize its attention flow. Our...
With the great success of pre-trained language models, pretrain-finetune paradigm now becomes undoubtedly dominant solution for natural understanding (NLU) tasks. At fine-tune stage, target task data is usually introduced in a completely random order and treated equally. However, examples NLU tasks can vary greatly difficulty, similar to human learning procedure, models benefit from an easy-to-difficult curriculum. Based on this idea, we propose our Curriculum Learning approach. By reviewing...
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting prompts. In this paper, we propose ExpertPrompting to elicit the potential LLMs answer as distinguished experts. We first utilize In-Context Learning automatically synthesize detailed and customized descriptions expert identity for each specific instruction, then ask provide conditioned on such agent background. Based augmented prompting strategy, produce a new set...
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural processing tasks that were previously thought to be exclusive humans. In this work, we introduce Qwen, first installment our large model series. Qwen is a comprehensive series encompasses distinct with varying parameter counts. It includes base pretrained models, and Qwen-Chat, chat finetuned human alignment techniques. The consistently demonstrate superior performance across multitude...
Relational triple extraction is challenging for its difficulty in capturing rich correlations between entities and relations. Existing works suffer from 1) heterogeneous representations of relations, 2) modeling entity-entity interactions entity-relation interactions. Therefore, the are not fully exploited by existing works. In this paper, we propose UniRel to address these challenges. Specifically, unify relations jointly encoding them within a concatenated natural language sequence, with...
Benfeng Xu, Quan Wang, Yajuan Lyu, Yabing Shi, Yong Zhu, Jie Gao, Zhendong Mao. Proceedings of the 2022 Conference North American Chapter Association for Computational Linguistics: Human Language Technologies. 2022.
Introduction Epigenetic biomarkers are molecular indicators of epigenetic changes, and some studies have suggested that these predictive power for disease risk. This study aims to analyze the relationship between 30 risk diabetes cancer using machine learning modeling. Methods The data this were sourced from NHANES database, which includes DNA methylation arrays biomarker datasets. Nine algorithms used build models: AdaBoost, GBM, KNN, lightGBM, MLP, RF, SVM, XGBoost, logistics. Model...
In-Context Learning (ICL), which formulates target tasks as prompt completion conditioned on in-context demonstrations, has become the prevailing utilization of LLMs. In this paper, we first disclose an actual predicament for typical usage that it can not scale up with training data due to context length restriction. Besides, existing works have shown ICL also suffers from various biases and requires delicate calibration treatment. To address both challenges, advocate a simple effective...
While large language models (LLMs) have exhibited impressive instruction-following capabilities, it is still unclear whether and to what extent they can respond explicit constraints that might be entailed in various instructions. As a significant aspect of LLM alignment, thus important formulate such specialized set instructions as well investigate the resulting behavior LLMs. To address this vacancy, we propose new benchmark CoDI-Eval systematically comprehensively evaluate LLMs' responses...
With the notable success of pretrained language models, pretraining-fine-tuning paradigm has become a dominant solution for natural understanding (NLU) tasks. Typically, training instances target NLU task are introduced in completely random order and treated equally at fine-tuning stage. However, these can vary greatly difficulty, similar to human learning procedures, models benefit from an easy-to-difficult curriculum. Based on this concept, we propose curriculum (CL) framework. Our...
Pre-trained language models (PLMs), such as BERT and GPT, have revolutionized the field of NLP, not only in general domain but also biomedical domain. Most prior efforts building PLMs resorted simply to adaptation focused mainly on English. In this work we introduce eHealth, a Chinese PLM built from scratch with new pre-training framework. This framework pre-trains eHealth discriminator through both token- sequence-level discrimination. The former is detect input tokens corrupted by...
Current relation extraction methods suffer from the inadequacy of large-scale annotated data.While distant supervision alleviates problem data quantities, there still exists domain disparity in qualities due to its reliance on domain-restrained knowledge bases. In this work, we propose S2ynRE, a framework two-stage Self-training with Synthetic for Relation Extraction.We first leverage capability large language models adapt target and automatically synthesize quantities coherent, realistic...
Language models pretrained on general domain corpora usually exhibit considerable degradation when generalizing to downstream tasks of specialized domains. Existing approaches try construct PLMs for each specific domains either from scratch or through further pretraining, which not only costs substantial resources, but also fails cover all target at various granularity. In this work, we propose RADA, a novel Retrieval-Augmented framework Domain Adaptation. We first textual that covers the...
While large language models (LLMs) have exhibited impressive instruction-following capabilities, it is still unclear whether and to what extent they can respond explicit constraints that might be entailed in various instructions. As a significant aspect of LLM alignment, thus important formulate such specialized set instructions as well investigate the resulting behavior LLMs. To address this vacancy, we propose new benchmark CoDI-Eval systematically comprehensively evaluate LLMs' responses...
Although most of existing works for sensor-based Human Activity Recognition rely on the temporal view, we argue that spectral view also provides complementary prior and accordingly benchmark a standard multi-view framework with extensive experiments to demonstrate its consistent superiority over single-view opponents. We then delve into intrinsic mechanism representation fusion, propose ModalDrop as novel modality-aware regularization method learn exploit representations both views...
Entities, as the essential elements in relation extraction tasks, exhibit certain structure. In this work, we formulate such structure distinctive dependencies between mention pairs. We then propose SSAN, which incorporates these structural within standard self-attention mechanism and throughout overall encoding stage. Specifically, design two alternative transformation modules inside each building block to produce attentive biases so adaptively regularize its attention flow. Our experiments...
Enhancing the instruction-following ability of Large Language Models (LLMs) primarily demands substantial instruction-tuning datasets. However, sheer volume these imposes a considerable computational burden and annotation cost. To investigate label-efficient instruction tuning method that allows model itself to actively sample subsets are equally or even more effective, we introduce self-evolving mechanism DiverseEvol. In this process, iteratively augments its training subset refine own...
As large language models attract increasing attention and find widespread application, concurrent challenges of reliability also arise at the same time. Confidence calibration, an effective analysis method for gauging deep models, serves as a crucial tool assessing improving their reliability. However, such investigation has been comparatively underexplored. In this work, we conduct systematic examination calibration aligned throughout entire construction process, including pretraining...
As large language models attract increasing attention and find widespread application, concurrent challenges of reliability also arise at the same time. Confidence calibration, an effective analysis method for gauging deep models, serves as a crucial tool assessing improving their reliability. However, such investigation has been comparatively underexplored. In this work, we conduct systematic examination calibration aligned throughout entire construction process, including pretraining...
Relational triple extraction is challenging for its difficulty in capturing rich correlations between entities and relations. Existing works suffer from 1) heterogeneous representations of relations, 2) modeling entity-entity interactions entity-relation interactions. Therefore, the are not fully exploited by existing works. In this paper, we propose UniRel to address these challenges. Specifically, unify relations jointly encoding them within a concatenated natural language sequence, with...