EyeGPT: Ophthalmic Assistant with Large Language Models

• DOI: 10.48550/arxiv.2403.00840
• Publication Date: 2024-02-29
• Authors: Xiaolan Chen; Ziwei Zhao; Weiyi Zhang; Pusheng Xu; Le Gao; Mingpu Xu; Yue Wu; Yinwen Li; Danli Shi; Mingguang He

Artificial intelligence (AI) has gained significant attention in healthcare consultation due to its potential improve clinical workflow and enhance medical communication. However, owing the complex nature of information, large language models (LLM) trained with general world knowledge might not possess capability tackle medical-related tasks at an expert level. Here, we introduce EyeGPT, a specialized LLM designed specifically for ophthalmology, using three optimization strategies including role-playing, finetuning, retrieval-augmented generation. In particular, proposed comprehensive evaluation framework that encompasses diverse dataset, covering various subspecialties different users, inquiry intents. Moreover, considered multiple metrics, accuracy, understandability, trustworthiness, empathy, proportion hallucinations. By assessing performance EyeGPT variants, identify most effective one, which exhibits comparable levels empathy human ophthalmologists (all Ps>0.05). Overall, ur study provides valuable insights future research, facilitating comparisons evaluations developing LLMs ophthalmology. The benefits include enhancing patient experience eye care optimizing ophthalmologists' services.