Readability and Presentation Suitability of ChatGPT's Medical Responses to Patient Questions: Cross-Sectional Study (Preprint)

• DOI: 10.2196/preprints.53046
• Publication Date: 2023-09-26T20:21:25Z
• Authors: Chan Woong Jang; Myungeun Yoo; Yoon Ghil Park
• Keywords: Health Literacy; Presentation (obstetrics); Grade level

<sec> <title>BACKGROUND</title> Online medical information, like ChatGPT, is crucial for patients making health decisions. However, many struggle with low literacy skills when using such content. To help, we need to ensure that the information easily readable average adult. Surprisingly, there's been no research on how well ChatGPT delivers in text form. </sec> <title>OBJECTIVE</title> assess readability and presentation suitability of responses most commonly asked patient questions, as ChatGPT's ability improve readability. <title>METHODS</title> This study involves two phases. First, evaluated 30 knee osteoarthritis (OA)-related questions March 20, 2023. We applied Flesch-Kincaid Grade Level (FKGL) Simple Measure Of Gobbledygook (SMOG) formulas. Additionally, used three evaluation tools: Suitability Assessment Materials (SAM) scores, Ensuring Quality Information Patients (EQIP), modified DISCERN (mDISCERN) overall quality scores. Secondly, assessed improvement answers 50 stroke-related by providing both detailed simple instructions into ChatGPT. In this phase, also utilized FKGL SMOG tests. <title>RESULTS</title> assessment, mean (standard deviation, SD) scores regarding OA were follows: FKGL, 13.65 (1.80) reading grade SMOG, 15.62 (1.55) grade, all which statistically higher than recommended sixth-grade level (P &lt; 0.001). SAM score was 55.00 (10.64), considered “adequate.” The EQIP mDISCERN 43.72 (5.78) 2.83 (0.59), respectively, none high quality. Upon implementing stroke, ANOVA test results indicate significant differences among groups: pre-intervention, post-intervention instructions, Post-hoc analysis revealed pre-intervention group differed significantly from groups assessments 0.001, respectively). there difference between = 0.96 0.86 SMOG). <title>CONCLUSIONS</title> discovered are hard read have quality, may discomfort patients, despite their adequate information. Furthermore, lacks information's As technology advances, enhancing user-friendliness will increase its usefulness patients. <title>CLINICALTRIAL</title> Not applicable.