Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding

• DOI: 10.1109/lra.2022.3146544
• Publication Date: 2022-01-27T22:31:02Z
• Authors: Lalithkumar Seeni...; Sai Mitheran; Mobarakol Islam; Hongliang Ren
• Keywords: FOS: Computer and information sciences; Computer Science - Robotics; 03 medical and health sciences; 0302 clinical medicine; Image and Video Processing (eess.IV); FOS: Electrical engineering, electronic engineering, information engineering; Electrical Engineering and Systems Science - Image and Video Processing; Robotics (cs.RO)

Global and local relational reasoning enable scene understanding models to perform human-like analysis understanding. Scene enables better semantic segmentation object-to-object interaction detection. In the medical domain, a robust surgical model allows automation of skill evaluation, real-time monitoring surgeon's performance post-surgical analysis. This paper introduces globally-reasoned multi-task capable performing instrument tool-tissue Here, we incorporate global in latent space introduce multi-scale (neighborhood) coordinate improve segmentation. Utilizing setup, visual-semantic graph attention network detection is further enhanced through reasoning. The features from module are introduced into network, allowing it detect interactions based on both node-to-node Our reduces computation cost compared running two independent single-task by sharing common modules, which indispensable for practical applications. Using sequential optimization technique, proposed outperforms other state-of-the-art MICCAI endoscopic vision challenge 2018 dataset. Additionally, also observe when trained using knowledge distillation technique. official code implementation made available GitHub.