ChatGPT outscored human candidates in a virtual objective structured clinical examination in obstetrics and gynecology
Interface (matter)
DOI:
10.1016/j.ajog.2023.04.020
Publication Date:
2023-04-22T23:46:46Z
AUTHORS (33)
ABSTRACT
Natural language processing is a form of artificial intelligence that allows human users to interface with machine without using complex codes. The ability natural systems, such as ChatGPT, successfully engage healthcare systems requiring fluid reasoning, specialist data interpretation, and empathetic communication in an unfamiliar evolving environment poorly studied. This study investigated whether the ChatGPT could complete mock objective structured clinical examination simulating assessment for membership Royal College Obstetricians Gynaecologists.This aimed determine additional training, would achieve score at least equivalent achieved by candidates who sat virtual examinations Singapore.This was conducted 2 phases. In first phase, total 7 discussion questions were selected from historical cohorts (cohorts A B) questions. examined these responses recorded script. Of note, (acting anonymizers) on same videoconferencing, their transcribed verbatim into written scripts. 3 sets response scripts mixed, each set allocated 1 actors. second actors used presenting examiners These blind scored 14 qualified examiners. scores unblinded compared candidate performance scores.The average given 77.2%. (n=26 candidates) 73.7 %. demonstrated sizable improvements over several subject domains. median time taken station 2.54 minutes, well before 10 minutes allowed.ChatGPT generated factually accurate contextually relevant answers based settings within very short period. outperformed knowledge areas. Not all able discern between responses. Our highlight emergent models demonstrate reasoning environments compete have undergone extensive training.
SUPPLEMENTAL MATERIAL
Coming soon ....
REFERENCES (40)
CITATIONS (81)
EXTERNAL LINKS
PlumX Metrics
RECOMMENDATIONS
FAIR ASSESSMENT
Coming soon ....
JUPYTER LAB
Coming soon ....