A5100606824- Muscle activation and electromyography studies
- Prosthetics and Rehabilitation Robotics
- Power Systems and Renewable Energy
- Stroke Rehabilitation and Recovery
- Microgrid Control and Optimization
- Topic Modeling
- Smart Grid and Power Systems
- Robot Manipulation and Learning
- Advanced Graph Neural Networks
- Privacy-Preserving Technologies in Data
- Gaze Tracking and Assistive Technology
- High-Voltage Power Transmission Systems
- Power Systems and Technologies
- Multimodal Machine Learning Applications
- Hand Gesture Recognition Systems
- Soft Robotics and Applications
- Natural Language Processing Techniques
- Smart Grid Energy Management
- Recommender Systems and Techniques
- Generative Adversarial Networks and Image Synthesis
- Video Analysis and Summarization
- Blockchain Technology Applications and Security
- Optimal Power Flow Distribution
- Motor Control and Adaptation
- Transportation Planning and Optimization
Nagoya University
2014-2024
University of North Carolina at Chapel Hill
2024
Northwest University
2024
Hefei University of Technology
2010-2023
Tsinghua University
2018-2023
Nara Institute of Science and Technology
2008-2023
Guilin University of Electronic Technology
2023
Robert Bosch (Germany)
2023
Data61
2022
University of Technology Sydney
2022
Graph representation learning has emerged as a powerful technique for addressing real-world problems. Various downstream graph tasks have benefited from its recent developments, such node classification, similarity search, and classification. However, prior arts on focus domain specific problems train dedicated model each dataset, which is usually non-transferable to out-of-domain data. Inspired by the advances in pre-training natural language processing computer vision, we design...
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding. We propose CogView, 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem. also demonstrate finetuning strategies for various downstream tasks, e.g. style learning, super-resolution, text-image ranking fashion design, methods stabilize pretraining, eliminating NaN losses. CogView achieves state-of-the-art FID on...
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source 100B-scale at least as good GPT-3 (davinci) unveil how models of such scale can be successfully pre-trained. Over the course this effort, we face numerous unexpected technical engineering challenges, particularly on loss spikes divergence. In paper, training process GLM-130B including its design choices, strategies for both efficiency stability,...
Prompting a pretrained language model with natural patterns has been proved effective for understanding (NLU). However, our preliminary study reveals that manual discrete prompts often lead to unstable performance—e.g., changing single word in the prompt might result substantial performance drop. We propose novel method P-Tuning employs trainable continuous embeddings concatenation prompts. Empirically, not only stabilizes training by minimizing gap between various prompts, but also improves...
Prompting a pretrained language model with natural patterns has been proved effective for understanding (NLU). However, our preliminary study reveals that manual discrete prompts often lead to unstable performance -- e.g., changing single word in the prompt might result substantial drop. We propose novel method P-Tuning employs trainable continuous embeddings concatenation prompts. Empirically, not only stabilizes training by minimizing gap between various prompts, but also improves sizeable...
We propose a new CogQA framework for multi-hop reading comprehension question answering in web-scale documents. Founded on the dual process theory cognitive science, gradually builds graph an iterative by coordinating implicit extraction module (System 1) and explicit reasoning 2). While giving accurate answers, our further provides explainable paths. Specifically, implementation based BERT neural network efficiently handles millions of documents questions HotpotQA fullwiki dataset,...
Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, Jie Tang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint (EMNLP-IJCNLP). 2019.
Recent advances in network embedding has revolutionized the field of graph and mining. However, (pre-)training embeddings for very large-scale networks is computationally challenging most existing methods. In this work, we present ProNE---a fast, scalable, effective model, whose single-thread version 10--400x faster than efficient benchmarks with 20 threads, including LINE, DeepWalk, node2vec, GraRep, HOPE. As a concrete example, single-version ProNE requires only 29 hours to embed hundreds...
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements. In this work, we present systematical reproduction 12 HGNNs using official codes, datasets, settings, hyperparameters, revealing surprising findings about progress HGNNs. We find that simple homogeneous GNNs, e.g., GCN GAT, are largely underestimated due to improper settings. GAT with...
The development of the transformer-based text-to-image models are impeded by its slow generation and complexity for high-resolution images. In this work, we put forward a solution based on hierarchical transformers local parallel auto-regressive generation. We pretrain 6B-parameter transformer with simple flexible self-supervised task, Cross-modal general language model (CogLM), finetune it fast super-resolution. new system, CogView2, shows very competitive compared to concurrent...
Large-scale pretrained transformers have created milestones in text (GPT-3) and text-to-image (DALL-E CogView) generation. Its application to video generation is still facing many challenges: The potential huge computation cost makes the training from scratch unaffordable; scarcity weak relevance of text-video datasets hinder model understanding complex movement semantics. In this work, we present 9B-parameter transformer CogVideo, trained by inheriting a model, CogView2. We also propose...
With the increasing use of large-scale grid-connected photovoltaic system, accurate forecast approach for power output system has become an important issue. In order to a at 24-hour-ahead without any complex modeling and complicated calculation, artificial neural network based is proposed in this paper. The improved back-propagation learning algorithm adopted overcome shortcomings standard algorithm. Similar day selection on information improve accuracy different weather types. Forecasting...
Graph representation learning has been extensively studied in recent years, which sampling is a critical point. Prior arts usually focus on positive node pairs, while the strategy for negative left insufficiently explored. To bridge gap, we systematically analyze role of from perspectives both objective and risk, theoretically demonstrating that as important determining optimization resulted variance. best our knowledge, are first to derive theory quantify nice distribution pn(u|v) ∝...
We investigate how generative adversarial nets (GANs) can help semi-supervised learning on graphs. first provide insights working principles of over graphs and then present GraphSGAN, a novel approach to In generator classifier networks play competitive game. At equilibrium, generates fake samples in low-density areas between subgraphs. order discriminate from the real, implicitly takes density property subgraph into consideration. An efficient algorithm has been developed improve...
Recently, federated learning (FL) has emerged as a promising distributed machine (ML) technology, owing to the advancing computational and sensing capacities of end-user devices, however with increasing concerns on users' privacy. As special architecture in FL, vertical FL (VFL) is capable constructing hyper ML model by embracing sub-models from different clients. These are trained locally vertically partitioned data distinct attributes. Therefore, design VFL fundamentally that conventional...
Microgrids integrate distributed renewable energy resources, controllable loads and storage in a more economic reliable fashion. Battery units are essential for microgrid operation, which make become strong coupling system the time domain. Hence, traditional methods of static dispatch no longer suitable microgrids. This paper proposes dynamic method. Considering as discrete system, is to find optimal control strategy finite period. Based on this idea, model microgrids established, then...
Healthy individuals modulate muscle activation patterns according to their intended movement and external environment. Persons with neurological disorders (e.g., stroke spinal cord injury), however, have problems in control due primarily inability pattern an appropriate manner. A functionality test at the level of individual muscles that investigates activity a interest on various motor tasks may enable muscle-level force grading. To date there is no extant work focuses application...
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements. In this work, we present systematical reproduction 12 HGNNs using official codes, datasets, settings, hyperparameters, revealing surprising findings about progress HGNNs. We find that simple homogeneous GNNs, e.g., GCN GAT, are largely underestimated due to improper settings. GAT with...
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1.9TB images and 292GB texts that cover a wide range domains. We propose cross-modal method called M6, referring to Multi-Modality Multitask Mega-transformer, unified on data single modality multiple modalities. scale model size up 10 billion 100 parameters, build pretrained Chinese. apply series downstream applications, demonstrate its outstanding performance comparison with strong...
Using large-scale training data to build a pre-trained language model (PLM) with larger volume of parameters can significantly improve downstream tasks. For example, OpenAI trained the GPT3 175 billion on 570 GB English data, enabling applications building only small number samples. However, there is lack Chinese corpus support PLMs. This paper introduces super corpora WuDaoCorpora, containing about 3 TB and 1.08 trillion characters. We also release base version 200 72 As baseline, we train...
Graph neural network-based recommendation systems are blossoming recently, and its core component is aggregation methods that determine neighbor embedding learning. Prior arts usually focus on how to aggregate information from the perspective of spatial structure information, but temporal about neighbors left insufficiently explored.