PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology

• DOI: 10.48550/arxiv.2502.08916
• Publication Date: 2025-02-12
• Authors: Fatemeh Ghezloo; Mehmet Saygın Sey...; Rustin Soraki; Wisdom O. Ikezogwo; Beibin Li; Tejoram Vivekanandan; Joann G. Elmore; Ranjay Krishna; Linda G. Shapiro
• Keywords: Pathfinder; Histopathology; Medical Decision Making

Diagnosing diseases through histopathology whole slide images (WSIs) is fundamental in modern pathology but challenged by the gigapixel scale and complexity of WSIs. Trained histopathologists overcome this challenge navigating WSI, looking for relevant patches, taking notes, compiling them to produce a final holistic diagnostic. Traditional AI approaches, such as multiple instance learning transformer-based models, fail short holistic, iterative, multi-scale diagnostic procedure, limiting their adoption real-world. We introduce PathFinder, multi-modal, multi-agent framework that emulates decision-making process expert pathologists. PathFinder integrates four agents, Triage Agent, Navigation Description Diagnosis collaboratively navigate WSIs, gather evidence, provide comprehensive diagnoses with natural language explanations. The Agent classifies WSI benign or risky; if risky, Agents iteratively focus on significant regions, generating importance maps descriptive insights sampled patches. Finally, synthesizes findings determine patient's classification. Our Experiments show outperforms state-of-the-art methods skin melanoma diagnosis 8% while offering inherent explainability descriptions diagnostically Qualitative analysis pathologists shows Agent's outputs are high quality comparable GPT-4o. also first AI-based system surpass average performance challenging classification task 9%, setting new record efficient, accurate, interpretable AI-assisted diagnostics pathology. Data, code models available at https://pathfinder-dx.github.io/