SimCol3D — 3D reconstruction during colonoscopy challenge

• DOI: 10.1016/j.media.2024.103195
• Publication Date: 2024-05-15T16:37:44Z
• Authors: Anita Rau; Sophia Bano; Yueming Jin; Pablo Azagra; Javier Morlana; Rawen Kader; Edward Sanderson; Bogdan J. Matusze...; Jae Young Lee; Dong-Jae Lee; Erez Posner; Netanel Frank; Varshini Elangovan; Sista Raviteja; Zhengwen Li; Jiquan Liu; Seenivasan Lalith...; Mobarakol Islam; Hongliang Ren; Laurence B. Lovat; José M.M. Montiel; Danail Stoyanov
• Keywords: Camera pose estimation; Depth prediction; FOS: Computer and information sciences; Surgical data science; Computer Vision and Pattern Recognition (cs.CV); I.4.5; Computer Science - Computer Vision and Pattern Recognition; Computer-assisted interventions; 3D reconstruction; Colonoscopy; Navigation

Colorectal cancer is one of the most common cancers in the world. While colonoscopy is an effective screening technique, navigating an endoscope through the colon to detect polyps is challenging. A 3D map of the observed surfaces could enhance the identification of unscreened colon tissue and serve as a training platform. However, reconstructing the colon from video footage remains difficult. Learning-based approaches hold promise as robust alternatives, but necessitate extensive datasets. Establishing a benchmark dataset, the 2022 EndoVis sub-challenge SimCol3D aimed to facilitate data-driven depth and pose prediction during colonoscopy. The challenge was hosted as part of MICCAI 2022 in Singapore. Six teams from around the world and representatives from academia and industry participated in the three sub-challenges: synthetic depth prediction, synthetic pose prediction, and real pose prediction. This paper describes the challenge, the submitted methods, and their results. We show that depth prediction from synthetic colonoscopy images is robustly solvable, while pose estimation remains an open research question.