The utility of generative artificial intelligence Chatbot (ChatGPT) in generating teaching and learning material for anesthesiology residents

• DOI: 10.3389/frai.2025.1582096
• Publication Date: 2025-05-21T05:24:10Z
• Authors: Zhaosheng Jin; Ramon Abola; Vincent Bargnes; Alexandra Tsivitis; Sadiq Rahman; Jonathon Schwartz; Sergio D. Bergese; Joy E. Schabel
• Keywords: Vetting; Chatbot; Stethoscope

The popularization of large language chatbots such as ChatGPT has led to growing utility in various biomedical fields. It been shown that can provide reasonably accurate responses medical exam style questions. On the other hand, have known limitations which may hinder their education. We conducted a pragmatically designed study evaluate accuracy and completeness generated styles prompts, based on entry-level anesthesiology topics. Ninety-five unique prompts were constructed using topics from Anesthesia Knowledge Test 1 (AKT-1), standardized undertaken by US residents after month specialty training. A combination focused open-ended was used its ability present organize information. also included for journal references, lecture outlines, well biased (medically inaccurate) prompts. independently scored 3-point Likert scale, two board-certified anesthesiologists with extensive experience Fifty-two (55%) rated completely both evaluators. For longer most deemed complete. Notably, chatbot frequently inaccurate when asked specific literature references input prompt contained deliberate errors (biased prompts). Another recurring observation conflation adjacent concepts (e.g., characteristic attributed wrong drug under same pharmacological class). Some inaccuracies could potentially result significant harm if applied clinical situations. While generate medically cases, reliability is not yet suited Content will require validation prior use.