From CNN, RNN, to ViT, we have witnessed remarkable advancements in video prediction, incorporating auxiliary inputs, elaborate neural architectures, and sophisticated training strategies. We admire these progresses but are confused about the necessity: is there a simple method that can perform comparably well? This paper proposes SimVp, prediction model completely built upon CNN trained by MSE loss an end-to-end fashion. Without introducing any additional tricks complicated strategies,...

Deep learning on graphs has recently achieved remarkable success a variety of tasks, while such relies heavily the massive and carefully labeled data. However, precise annotations are generally very expensive time-consuming. To address this problem, self-supervised (SSL) is emerging as new paradigm for extracting informative knowledge through well-designed pretext tasks without relying manual labels. In survey, we extend concept SSL, which first emerged in fields computer vision natural...

Deep generative models have unlocked another profound realm of human creativity. By capturing and generalizing patterns within data, we entered the epoch all-encompassing Artificial Intelligence for General Creativity (AIGC). Notably, diffusion models, recognized as one paramount materialize ideation into tangible instances across diverse domains, encompassing imagery, text, speech, biology, healthcare. To provide advanced comprehensive insights diffusion, this survey comprehensively...

Spatiotemporal predictive learning aims to generate future frames by from historical frames. In this paper, we investigate existing methods and present a general framework of spatiotemporal learning, in which the spatial encoder decoder capture intra-frame features middle temporal module catches inter-frame correlations. While mainstream employ recurrent units long-term dependencies, they suffer low computational efficiency due their unparallelizable architectures. To parallelize module,...

Recent years have witnessed the prosperity of pre-training graph neural networks (GNNs) for molecules. Typically, atom types as node attributes are randomly masked and GNNs then trained to predict in AttrMask \citep{hu2020strategies}, following Masked Language Modeling (MLM) task BERT~\citep{devlin2019bert}. However, unlike MLM where vocabulary is large, does not learn informative molecular representations due small unbalanced `vocabulary'. To amend this problem, we propose a variant...

Deep generative models have unlocked another profound realm of human creativity. By capturing and generalizing patterns within data, we entered the epoch all-encompassing Artificial Intelligence for General Creativity (AIGC). Notably, diffusion models, recognized as one paramount materialize ideation into tangible instances across diverse domains, encompassing imagery, text, speech, biology, healthcare. To provide advanced comprehensive insights diffusion, this survey comprehensively...

Spatio-temporal forecasting is challenging attributing to the high nonlinearity in temporal dynamics as well complex location-characterized patterns spatial domains, especially fields like weather forecasting. Graph convolutions are usually used for modeling dependency meteorology handle irregular distribution of sensors' location. In this work, a novel graph-based convolution imitating meteorological flows proposed capture local patterns. Based on assumption smoothness patterns, we propose...

Antibodies are Y-shaped proteins that protect the host by binding to specific antigens, and their is mainly determined Complementary Determining Regions (CDRs) in antibody. Despite great progress made CDR design, existing computational methods still encounter several challenges: 1) poor capability of modeling complex CDRs with long sequences due insufficient contextual information; 2) conditioned on pre-given antigenic epitopes static interaction target antibody; 3) neglect specificity...

Recent years have witnessed remarkable advances in spatiotemporal predictive learning, incorporating auxiliary inputs, elaborate neural architectures, and sophisticated training strategies. Although impressive, the system complexity of mainstream methods is increasing as well, which may hinder convenient applications. This paper proposes SimVP, a simple baseline model that completely built upon convolutional networks without recurrent architectures trained by common mean squared error loss...

Spatio-temporal predictive learning is a paradigm that enables models to learn spatial and temporal patterns by predicting future frames from given past in an unsupervised manner. Despite remarkable progress recent years, lack of systematic understanding persists due the diverse settings, complex implementation, difficult reproducibility. Without standardization, comparisons can be unfair insights inconclusive. To address this dilemma, we propose OpenSTL, comprehensive benchmark for...

Graph neural networks (GNNs) have been playing important roles in various graph-related tasks. However, most existing GNNs are based on the assumption of homophily, so they cannot be directly generalized to heterophily settings where connected nodes may different features and class labels. Moreover, real-world graphs often arise from highly entangled latent factors, but tend ignore this simply denote heterogeneous relations between as binary-valued homogeneous edges. In article, we propose a...

Recent years have witnessed great success in handling graph-related tasks with Graph Neural Networks (GNNs). Despite their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">academic</i> success, Multi-Layer Perceptrons (MLPs) remain the primary workhorse for practical xmlns:xlink="http://www.w3.org/1999/xlink">industrial</i> applications. One reason such an academic-industry gap is neighborhood-fetching latency incurred by data dependency GNNs....

How can we design protein sequences folding into the desired structures effectively and efficiently? AI methods for structure-based have attracted increasing attention in recent years; however, few simultaneously improve accuracy efficiency due to lack of expressive features autoregressive sequence decoder. To address these issues, propose PiFold, which contains a novel residue featurizer PiGNN layers generate one-shot way with improved recovery. Experiments show that PiFold could achieve...

Compound-Protein Interaction (CPI) prediction aims to predict the pattern and strength of compound-protein interactions for rational drug discovery. Existing deep learning-based methods utilize only single modality protein sequences or structures lack co-modeling joint distribution two modalities, which may lead significant performance drops in complex real-world scenarios due various factors, e.g., missing domain shifting. More importantly, these model at a fixed scale, neglecting more...

Recent advances in contrastive learning have enlightened diverse applications across various semi-supervised fields. Jointly training supervised and unsupervised with a shared feature encoder becomes common scheme. Though it benefits from taking advantage of both feature-dependent information self-supervised label-dependent learning, this scheme remains suffering bias the classifier. In work, we systematically explore relationship between study how helps robust data-efficient deep learning....

<title>Abstract</title> While cryo-electron microscopy (Cryo-EM) yields high-resolution density maps for complex structures, accurate determination of the corresponding three-dimensional atomic structures still necessitates significant expertise and labor-intensive manual interpretation. Recently, AI-based methods have emerged to streamline this process in biological community; however, several challenges persist. First, existing typically require multi-stage training inference, causing...

Discovering the genotype-phenotype relationship is crucial for genetic engineering, which will facilitate advances in fields such as crop breeding, conservation biology, and personalized medicine. Current research usually focuses on single species small datasets due to limitations phenotypic data collection, especially traits that require visual assessments or physical measurements. Deciphering complex composite phenotypes, morphology, from at scale remains an open question. To break through...

Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed challenge due to the contrasting modeling paradigms of discrete sequences. We introduce FoldTokenizer represent sequence-structure as symbols. This approach involves projecting residue types into space, guided reconstruction loss for information preservation. name learned symbols FoldToken, sequence FoldTokens serves new language,...

The development of therapeutic antibodies heavily relies on accurate predictions how antigens will interact with antibodies. Existing computational methods in antibody design often overlook crucial conformational changes that undergo during the binding process, significantly impacting reliability resulting To bridge this gap, we introduce dyAb, a flexible framework incorporates AlphaFold2-driven to model pre-binding antigen structures and specifically addresses dynamic nature conformation...

Spatio-temporal predictive learning plays a crucial role in self-supervised learning, with wide-ranging applications across diverse range of fields. Previous approaches for temporal modeling fall into two categories: recurrent-based and recurrent-free methods. The former, while meticulously processing frames one by one, neglect short-term spatio-temporal information redundancies, leading to inefficiencies. latter naively stack sequentially, overlooking the inherent dependencies. In this...