A5038654477- Advanced Graph Neural Networks
- Machine Learning in Materials Science
- Topic Modeling
- Semantic Web and Ontologies
- Machine Learning in Bioinformatics
- Genetics, Bioinformatics, and Biomedical Research
- Computational Drug Discovery Methods
- Natural Language Processing Techniques
- Recommender Systems and Techniques
- Human Pose and Action Recognition
- Medical Imaging Techniques and Applications
- Protein Structure and Dynamics
- Radiomics and Machine Learning in Medical Imaging
- Bayesian Modeling and Causal Inference
- Domain Adaptation and Few-Shot Learning
- Bioinformatics and Genomic Networks
- Adversarial Robustness in Machine Learning
- Face recognition and analysis
- Biomedical Text Mining and Ontologies
- Speech and dialogue systems
- Generative Adversarial Networks and Image Synthesis
- Advanced Neural Network Applications
- Intelligent Tutoring Systems and Adaptive Learning
- Data Quality and Management
- Model-Driven Software Engineering Techniques
Mila - Quebec Artificial Intelligence Institute
2019-2024
Centre Universitaire de Mila
2023-2024
Université de Montréal
2023-2024
University of Hong Kong
2023
Tsinghua University
2019-2021
Peking University
2016-2018
Abstract Pre-trained language representation models (PLMs) cannot well capture factual knowledge from text. In contrast, embedding (KE) methods can effectively represent the relational facts in graphs (KGs) with informative entity embeddings, but conventional KE take full advantage of abundant textual information. this paper, we propose a unified model for Knowledge Embedding and LanguagERepresentation (KEPLER), which not only better integrate into PLMs also produce effective text-enhanced...
Molecular graph generation is a fundamental problem for drug discovery and has been attracting growing attention. The challenging since it requires not only generating chemically valid molecular structures but also optimizing their chemical properties in the meantime. Inspired by recent progress deep generative models, this paper we propose flow-based autoregressive model called GraphAF. GraphAF combines advantages of both approaches enjoys: (1) high flexibility data density estimation; (2)...
Link prediction is a very fundamental task on graphs. Inspired by traditional path-based methods, in this paper we propose general and flexible representation learning framework based paths for link prediction. Specifically, define the of pair nodes as generalized sum all path representations, with each product edge representations path. Motivated Bellman-Ford algorithm solving shortest problem, show that proposed formulation can be efficiently solved algorithm. To further improve capacity...
Learning continuous representations of nodes is attracting growing interest in both academia and industry recently, due to their simplicity effectiveness a variety applications. Most existing node embedding algorithms systems are capable processing networks with hundreds thousands or few millions nodes. However, how scale them that have tens even remains challenging problem. In this paper, we propose GraphVite, high-performance CPU-GPU hybrid system for training embeddings, by co-optimizing...
Pre-trained language representation models (PLMs) cannot well capture factual knowledge from text. In contrast, embedding (KE) methods can effectively represent the relational facts in graphs (KGs) with informative entity embeddings, but conventional KE take full advantage of abundant textual information. this paper, we propose a unified model for Knowledge Embedding and LanguagE Representation (KEPLER), which not only better integrate into PLMs also produce effective text-enhanced strong...
Machine learning has huge potential to revolutionize the field of drug discovery and is attracting increasing attention in recent years. However, lacking domain knowledge (e.g., which tasks work on), standard benchmarks data preprocessing pipelines are main obstacles for machine researchers this domain. To facilitate progress discovery, we develop TorchDrug, a powerful flexible platform built on top PyTorch. TorchDrug variety important including molecular property prediction, pretrained...
We are now witnessing significant progress of deep learning methods in a variety tasks (or datasets) proteins. However, there is lack standard benchmark to evaluate the performance different methods, which hinders this field. In paper, we propose such called PEER, comprehensive and multi-task for Protein sEquence undERstanding. PEER provides set diverse protein understanding including function prediction, localization structure protein-protein interaction protein-ligand prediction. types...
We present an advanced dialog state tracking system designed for the 5th Dialog State Tracking Challenge (DSTC5). The main task of DSTC5 is to track in a human-human dialog. For each utterance, tracker emits frame slot-value pairs considering full history up current turn. Our includes encoder-decoder architecture with attention mechanism map input word sequence set semantic labels, i.e., pairs. This handles problem unknown alignment between utterances and labels. By combining attention-based...
Answering complex first-order logic (FOL) queries on knowledge graphs is a fundamental task for multi-hop reasoning. Traditional symbolic methods traverse complete graph to extract the answers, which provides good interpretation each step. Recent neural learn geometric embeddings queries. These can generalize incomplete graphs, but their reasoning process hard interpret. In this paper, we propose Graph Neural Network Query Executor (GNN-QE), neural-symbolic model that enjoys advantages of...
Recently, pre-trained foundation models have enabled significant advancements in multiple fields. In molecular machine learning, however, where datasets are often hand-curated, and hence typically small, the lack of with labeled features, codebases to manage those datasets, has hindered development models. this work, we present seven novel categorized by size into three distinct categories: ToyMix, LargeMix UltraLarge. These push boundaries both scale diversity supervised labels for...
When prompted with a few examples and intermediate steps, large language models (LLMs) have demonstrated impressive performance in various reasoning tasks. However, prompting methods that rely on implicit knowledge an LLM often hallucinate incorrect answers when the is wrong or inconsistent task. To tackle this problem, we present Hypotheses-to-Theories (HtT), framework learns rule library for LLMs. HtT contains two stages, induction stage deduction stage. In stage, first asked to generate...
Foundation models that can perform inference on any new task without requiring specific training have revolutionized machine learning in vision and language applications. However, applications involving graph-structured data remain a tough nut for foundation models, due to challenges the unique feature- label spaces associated with each graph. Traditional graph ML such as neural networks (GNNs) trained graphs cannot feature different from ones. Furthermore, existing learn functions...
Reasoning on large-scale knowledge graphs has been long dominated by embedding methods. While path-based methods possess the inductive capacity that embeddings lack, their scalability is limited exponential number of paths. Here we present A*Net, a scalable method for graph reasoning. Inspired A* algorithm shortest path problems, our A*Net learns priority function to select important nodes and edges at each iteration, reduce time memory footprint both training inference. The ratio selected...
Large Language Models (LLMs) have gained the ability to assimilate human knowledge and facilitate natural language interactions with both humans other LLMs. However, despite their impressive achievements, LLMs not made significant advancements in realm of graph machine learning. This limitation arises because graphs encapsulate distinct relational data, making it challenging transform them into that understand. In this paper, we bridge gap a novel framework, GraphText, translates language....
Recently, doc2vec has achieved excellent results in different tasks. In this paper, we present a context aware variant of doc2vec. We introduce novel weight estimating mechanism that generates weights for each word occurrence according to its contribution the context, using deep neural networks. Our model can achieve similar compared initialized byWikipedia trained vectors, while being much more efficient and free from heavy external corpus. Analysis shows they are kind enhanced IDF capture...
Complex logical query answering (CLQA) is a recently emerged task of graph machine learning that goes beyond simple one-hop link prediction and solves far more complex multi-hop reasoning over massive, potentially incomplete graphs in latent space. The received significant traction the community; numerous works expanded field along theoretical practical axes to tackle different types queries modalities with efficient systems. In this paper, we provide holistic survey CLQA detailed taxonomy...
Foundation models in language and vision have the ability to run inference on any textual visual inputs thanks transferable representations such as a vocabulary of tokens language. Knowledge graphs (KGs) different entity relation vocabularies that generally do not overlap. The key challenge designing foundation KGs is learn enable graph with arbitrary vocabularies. In this work, we make step towards present ULTRA, an approach for learning universal representations. ULTRA builds relational...
Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at compositional queries comprised of multiple projections operations. Existing CLQA methods that learn parameters bound to certain entity or relation vocabularies can only be applied the graph they are trained on which requires substantial training time before being deployed a new graph. Here we present UltraQuery, an inductive reasoning model zero-shot answer any KG. The core idea...
Foundation models such as GPT-4 for natural language processing (NLP), Flamingo computer vision (CV), have set new benchmarks in AI by delivering state-of-the-art results across various tasks with minimal task-specific data. Despite their success, the application of these to graph domain is challenging due relational nature graph-structured To address this gap, we propose Graph Model (GFM) Workshop, first workshop GFMs, dedicated exploring adaptation and development foundation specifically...
Abstract Understanding complex interactions in biomedical networks is crucial for advancements biomedicine, but traditional link prediction (LP) methods are limited capturing this complexity. Representation-based learning techniques improve accuracy by mapping nodes to low-dimensional embeddings, yet they often struggle with interpretability and scalability. We present BioPathNet, a novel graph neural network framework based on the Neural Bellman-Ford Network (NBFNet), addressing these...
<title>Abstract</title> Understanding complex interactions in biomedical networks is crucial for advancements biomedicine, but traditional link prediction (LP) methods are limited capturing this complexity. Representation-based learning techniques improve accuracy by mapping nodes to low-dimensional embeddings, yet they often struggle with interpretability and scalability. We present BioPathNet, a novel graph neural network framework based on the Neural Bellman-Ford Network (NBFNet),...
Reasoning, the ability to logically draw conclusions from existing knowledge, is a hallmark of human. Together with perception, they constitute two major themes artificial intelligence. While deep learning has pushed limit perception beyond human-level performance, progress in reasoning domains way behind. One fundamental reason that problems usually have flexible structures for both knowledge and queries, many models only perform well on seen during training. Here we aim push boundary by...