Relationship between prediction accuracy and feature importance reliability: An empirical and theoretical study

• DOI: 10.1016/j.neuroimage.2023.120115
• Publication Date: 2023-04-23T23:53:57Z
• Authors: Jianzhong Chen; Leon Qi Rong Ooi; Trevor Wei Kiat Tan; Shaoshi Zhang; Jingwei Li; Christopher L. As...; Simon B Eickhoff; Danilo Bzdok; Avram J Holmes; B.T. Thomas Yeo
• Keywords: Feature (linguistics); Univariate; Predictive modelling

There is significant interest in using neuroimaging data to predict behavior. The predictive models are often interpreted by the computation of feature importance, which quantifies relevance an imaging feature. Tian and Zalesky (2021) suggest that importance estimates exhibit low split-half reliability, as well a trade-off between prediction accuracy reliability across parcellation resolutions. However, it unclear whether universal. Here, we demonstrate that, with sufficient sample size, (operationalized Haufe-transformed weights) can achieve fair excellent reliability. With size 2600 participants, weights average intra-class correlation coefficients 0.75, 0.57 0.53 for cognitive, personality mental health measures respectively. much more reliable than original regression univariate FC-behavior correlations. Original not even participants. Intriguingly, strongly positively correlated phenotypes. Within particular behavioral domain, there no clear relationship performance models. Furthermore, show mathematically necessary, but sufficient, error. In case linear models, lower error related Therefore, higher might yield accuracy. Finally, discuss how our theoretical results relate features measures. Overall, current study provides empirical insights into