Multi-Scale Protein Language Model for Unified Molecular Modeling

• DOI: 10.48550/arxiv.2403.12995
• Publication Date: 2024-03-05
• Authors: Kangjie Zheng; Siyu Long; Tianyu Lu; Junwei Yang; Xinyu Dai; Ming Zhang; Zaiqing Nie; Wei‐Ying Ma; Hao Zhou
• Keywords: Molecular model; Scale model

Protein language models have demonstrated significant potential in the field of protein engineering. However, current primarily operate at residue scale, which limits their ability to provide information atom level. This limitation prevents us from fully exploiting capabilities for applications involving both proteins and small molecules. In this paper, we propose ms-ESM (multi-scale ESM), a novel approach that enables multi-scale unified molecular modeling. achieves by pre-training on code-switch sequences utilizing position encoding capture relationships among residues atoms. Experimental results indicate surpasses previous methods protein-molecule tasks, demonstrating full utilization models. Further investigations reveal through modeling, not only gains knowledge but also retains its understanding proteins.