Deep Learning in current Neuroimaging: a multivariate approach with power and type I error control but arguable generalization ability

• DOI: 10.48550/arxiv.2103.16685
• Publication Date: 2021-01-01
• Authors: C. Jiménez-Mesa; Javier Ramı́rez; John Suckling; Jonathan Vöglein; Johannes Levin; J. M. Górriz; Alzheimer’s Disea...; Dominantly Inheri...
• Keywords: Overfitting; Discriminative model; Resampling; Statistical power; Cross-validation

Discriminative analysis in neuroimaging by means of deep/machine learning techniques is usually tested with validation techniques, whereas the associated statistical significance remains largely under-developed due to their computational complexity. In this work, a non-parametric framework proposed that estimates classifications using deep architectures. particular, combination autoencoders (AE) and support vector machines (SVM) applied to: (i) one-condition, within-group designs often normal controls (NC) and; (ii) two-condition, between-group which contrast, for example, Alzheimer's disease (AD) patients NC (the extension multi-class analyses also included). A random-effects inference based on label permutation test both studies cross-validation (CV) resubstitution upper bound correction (RUB) as methods. This allows false positives classifier overfitting be detected well estimating power test. Several experiments were carried out Disease Neuroimaging Initiative (ADNI) dataset, Dominantly Inherited Alzheimer Network (DIAN) MCI prediction dataset. We found CV RUB methods offer positive rate close level an acceptable (although lower cross-validation). large separation between training accuracies was observed, especially one-condition designs. implies low generalization ability model fitted not informative respect set. propose solution applying RUB, whereby similar results are obtained those set, but considering whole set cost per iteration.