Me LLaMA: Foundation Large Language Models for Medical Applications

• DOI: 10.48550/arxiv.2402.12749
• Publication Date: 2024-02-20
• Authors: Qianqian Xie; Qingyu Chen; Aokun Chen; Peng Cheng; Yan Hu; Fongci Lin; Xueqing Peng; Jimin Huang; Jeffrey Zhang; Vipina Keloth; Huan He; Lucila Ohno-Machido; Yonghui Wu; Hua Xu; Jiang Bian
• Keywords: Foundation (evidence)

Recent large language models (LLMs) like ChatGPT and LLaMA have shown great promise in many AI applications. However, their performance on medical tasks is suboptimal can be further improved by training domain-specific datasets. This study introduces Me LLaMA, a LLM family including foundation - 13/70B chat-enhanced versions 13/70B-chat, developed through the continual pre-training instruction tuning of LLaMA2 using data. Our data suite for evaluation, includes large-scale dataset with 129B tokens, an 214k samples, evaluation benchmark (MIBE) across six 14 extensive MIBE shows that surpass existing open-source LLMs zero-shot few-shot learning outperform commercial giants 6 out 8 datasets GPT-4 3 In addition, we empirically investigated catastrophic forgetting problem, our results show other LLMs. one first largest foundational designed domain, both biomedical clinical It exhibits superior general compared to LLMs, rendering it attractive choice All resources are available at: https://github.com/BIDS-Xu-Lab/Me-LLaMA.