Data from Multi-institutional prognostic modelling in head and neck cancer: evaluating impact and generalizability of deep learning and radiomics

• DOI: 10.1158/2767-9764.c.6684845.v1
• Publication Date: 2023-06-07T14:23:26Z
• Authors: Michal Kazmierski; Mattea Welch; Sejin Kim; Chris McIntosh; Katrina Rey-McIntyre; Shao Hui Huang; Tirth Patel; Tony Tadic; Michael Milosevic; Fei-Fei Liu; Adam Ryczkowski; Joanna Kazmierska; Zezhong Ye; Deborah Plana; Hugo JWL Aerts; Benjamin H Kann; Scott V. Bratman; Andrew J Hope; Benjamin Haibe-Kains

<div>Abstract<p>Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging ({radiomics}). However, the development of prognostic models is complex as no modelling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records and pre-treatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or electronic medical record (EMR) data. The model with the highest accuracy used multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best-performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks.</p></div>