Foundation Metrics: Quantifying Effectiveness of Healthcare Conversations powered by Generative AI

• DOI: 10.48550/arxiv.2309.12444
• Publication Date: 2023-01-01
• Authors: Mahyar Abbasian; Elahe Khatibi; Iman Azimi; David Oniani; Zahra Shakeri Hos...; Alexander Thieme; Zhongqi Yang; Yanshan Wang; Bryant Lin; Olivier Gevaert; Li-Jia Li; Ramesh Jain; Amir M. Rahmani
• Keywords: FOS: Computer and information sciences; 0301 basic medicine; Computer Science - Computation and Language; patient care; Computer applications to medicine. Medical informatics; R858-859.7; 610; healthcare; 8.1 Organisation and delivery of services; Health Services; 3. Good health; 004; 03 medical and health sciences; Generative Artificial Intelligence; Good Health and Well Being; Networking and Information Technology R&D (NITRD); Clinical Research; Health Services and Systems; Health Sciences; Perspective; Machine Learning and Artificial Intelligence; Health services and systems; Generic health relevance; Computation and Language (cs.CL)

Generative Artificial Intelligence is set to revolutionize healthcare delivery by transforming traditional patient care into a more personalized, efficient, and proactive process. Chatbots, serving as interactive conversational models, will probably drive this patient-centered transformation in healthcare. Through the provision of various services, including diagnosis, personalized lifestyle recommendations, mental health support, objective substantially augment outcomes, all while mitigating workload burden on providers. The life-critical nature applications necessitates establishing unified comprehensive evaluation metrics for models. Existing proposed generic large language models (LLMs) demonstrate lack comprehension regarding medical concepts their significance promoting patients' well-being. Moreover, these neglect pivotal user-centered aspects, trust-building, ethics, personalization, empathy, user comprehension, emotional support. purpose paper explore state-of-the-art LLM-based that are specifically applicable assessment Subsequently, we present an designed thoroughly assess performance chatbots from end-user perspective. These encompass processing abilities, impact real-world clinical tasks, effectiveness user-interactive conversations. Finally, engage discussion concerning challenges associated with defining implementing metrics, particular emphasis confounding factors such target audience, methods, prompt techniques involved