FluoroSAM: A Language-aligned Foundation Model for X-ray Image Segmentation

• DOI: 10.48550/arxiv.2403.08059
• Publication Date: 2024-03-12
• Authors: Benjamin D. Killeen; Liam J. Wang; Han Zhang; Mehran Armand; Russell H. Taylor; Greg Osgood; Mathias Unberath
• Keywords: Foundation (evidence)

Automated X-ray image segmentation would accelerate research and development in diagnostic interventional precision medicine. Prior efforts have contributed task-specific models capable of solving specific analysis problems, but the utility these is restricted to their particular task domain, expanding broader use requires additional data, labels, retraining efforts. Recently, foundation (FMs) -- machine learning trained on large amounts highly variable data thus enabling broad applicability emerged as promising tools for automated analysis. Existing FMs medical focus scenarios modalities where objects are clearly defined by visually apparent boundaries, such surgical tool endoscopy. imaging, contrast, does not generally offer delineated boundaries or structure priors. During formation, complex 3D structures projected transmission onto imaging plane, resulting overlapping features varying opacity shape. To pave way toward an FM comprehensive arbitrary images, we develop FluoroSAM, a language-aligned variant Segment-Anything Model, from scratch 1.6M synthetic images. FluoroSAM including masks 128 organ types 464 non-anatomical objects, implants. In real images cadaveric specimens, able segment bony anatomical based text-only prompting with 0.51 0.79 DICE point-based refinement, outperforming competing SAM variants all structures. also zero-shot generalization segmenting classes beyond training set thanks its language alignment, which demonstrate full lung chest X-rays.