A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation

• DOI: 10.48550/arxiv.2401.12208
• Publication Date: 2024-01-01
• Authors: Chen, Zhihong; Varma, Maya; Xu, Justin; Paschali, Magdalini; Van Veen, Dave; Johnston, Andrew; Youssef, Alaa; Blankemeier, Louis; Bluethgen, Christian; Altmayer, Stephan; Valanarasu, Jeya ...; Muneer, Mohamed S...; Reis, Eduardo Pontes; Cohen, Joseph Paul; Olsen, Cameron; Abraham, Tanishq ...; Tsai, Emily B.; Beaulieu, Christo...; Jitsev, Jenia; Gatidis, Sergios; Delbrouck, Jean-B...; Chaudhari, Akshay S.; Langlotz, Curtis P.
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; Computation and Language (cs.CL)

26 pages, 8 figures<br/>Over 1.4 billion chest X-rays (CXRs) are performed annually due to their cost-effectiveness as an initial diagnostic test. This scale of radiological studies provides a significant opportunity to streamline CXR interpretation and documentation. While foundation models are a promising solution, the lack of publicly available large-scale datasets and benchmarks inhibits their iterative development and real-world evaluation. To overcome these challenges, we constructed a large-scale dataset (CheXinstruct), which we utilized to train a vision-language foundation model (CheXagent). We systematically demonstrated competitive performance across eight distinct task types on our novel evaluation benchmark (CheXbench). Beyond technical validation, we assessed the real-world utility of CheXagent in directly drafting radiology reports. Our clinical assessment with eight radiologists revealed a 36% time saving for residents using CheXagent-drafted reports, while attending radiologists showed no significant time difference editing resident-drafted or CheXagent-drafted reports. The CheXagent-drafted reports improved the writing efficiency of both radiology residents and attending radiologists in 81% and 61% of cases, respectively, without loss of quality. Overall, we demonstrate that CheXagent can effectively perform a variety of CXR interpretation tasks and holds potential to assist radiologists in routine clinical workflows.<br/>