This paper proposes a method to modify traditional convolutional neural network (CNN) into an interpretable CNN, in order clarify knowledge representations high conv-layers of the CNN. In each filter conv-layer represents specific object part. Our CNNs use same training data as ordinary without need for any annotations parts or textures supervision. The CNN automatically assigns with part during learning process. We can apply our different types various structures. explicit representation...

This paper aims to quantitatively explain the rationales of each prediction that is made by a pre-trained convolutional neural network (CNN). We propose learn decision tree, which clarifies specific reason for CNN at semantic level. I.e., tree decomposes feature representations in high conv-layers into elementary concepts object parts. In this way, tells people parts activate filters and how much part contributes score. Such quantitative explanations predictions have values beyond...

This paper learns a graphical model, namely an explanatory graph, which reveals the knowledge hierarchy hidden inside pre-trained CNN. Considering that each filter in conv-layer of CNN usually represents mixture object parts, we propose simple yet efficient method to automatically disentangles different part patterns from filter, and construct graph. In node pattern, edge encodes co-activation relationships spatial between patterns. More importantly, learn graph for unsupervised manner,...

The frequency and intensity of natural disasters has significantly increased over the past decades this trend is predicted to continue. Facing these possible unexpected disasters, accurately predicting human emergency behavior their mobility will become critical issue for planning effective humanitarian relief, disaster management, long-term societal reconstruction. In paper, we build up a large database (GPS records 1.6 million users one year) several different datasets capture analyze...

This paper presents a method to interpret the success of knowledge distillation by quantifying and analyzing task-relevant task-irrelevant visual concepts that are encoded in intermediate layers deep neural network (DNN). More specifically, three hypotheses proposed as follows. 1. Knowledge makes DNN learn more than learning from raw data. 2. ensures is prone various simultaneously. Whereas, scenario data, learns sequentially. 3. yields stable optimization directions Accordingly, we design...

Given a pre-trained CNN without any testing samples, this paper proposes simple yet effective method to diagnose feature representations of the CNN. We aim discover representation flaws caused by potential dataset bias. More specifically, when is trained estimate image attributes, we mine latent relationships between different attributes inside Then, compare mined attribute with ground-truth CNN's blind spots and failure modes due In fact, bias cannot be examined conventional evaluation...

The Great East Japan Earthquake and the Fukushima nuclear accident cause large human population movements evacuations. Understanding predicting these is critical for planning effective humanitarian relief, disaster management, long-term societal reconstruction. In this paper, we construct a mobility database that stores manages GPS records from mobile devices used by approximately 1.6 million people throughout 1 August 2010 to 31 July 2011. By mining enormous set of Auto-GPS sensor data,...

Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Yu Cheng, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, Zhouhan Lin. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.

This paper proposes a learning strategy that embeds object-part concepts into pre-trained convolutional neural network (CNN), in an attempt to 1) explore explicit semantics hidden CNN units and 2) gradually transform the semantically interpretable graphical model for hierarchical object understanding. Given part annotations on very few (e.g., 3-12) objects, our method mines certain latent patterns from associates them with different semantic parts. We use four-layer And-Or graph organize...

In recent decades, the frequency and intensity of natural disasters has increased significantly, this trend is expected to continue. Therefore, understanding predicting human behavior mobility during a disaster will play vital role in planning effective humanitarian relief, management, long-term societal reconstruction. However, such research very difficult perform owing uniqueness various unavailability reliable large-scale data. study, we collect big heterogeneous data (e.g., GPS records...

This paper presents a method to pursue semantic and quantitative explanation for the knowledge encoded in convolutional neural network (CNN). The estimation of specific rationale each prediction made by CNN key issue understanding networks, it is significant values real applications. In this study, we propose distill from into an explainable additive model, which explains quantitatively. We discuss problem biased interpretation predictions. To overcome interpretation, develop prior losses...

This paper proposes a generic method to learn interpretable convolutional filters in deep neural network (CNN) for object classification, where each filter encodes features of specific part. Our does not require additional annotations parts or textures supervision. Instead, we use the same training data as traditional CNNs. automatically assigns high conv-layer with an part certain category during learning process. Such explicit knowledge representations conv-layers CNN help people clarify...

Various attribution methods have been developed to explain deep neural networks (DNNs) by inferring the attribution/importance/contribution score of each input variable final output. However, existing are often built upon different heuristics. There remains a lack unified theoretical understanding why these effective and how they related. To this end, for first time, we formulate core mechanisms fourteen methods, which were designed on heuristics, into same mathematical system, i.e., system...

This study proposes a set of generic rules to revise existing neural networks for 3D point cloud processing rotation-equivariant quaternion (REQNNs), in order make feature representations be and permutation-invariant. Rotation equivariance features means that the computed on rotated input is same as applying rotation transformation original cloud. We find rotation-equivariance naturally satisfied, if network uses features. Interestingly, we prove such revision also makes gradients REQNN...

In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover confusing samples DNN, which are represented non-generalizable interactions, determined low-layer parameters. comparison, other factors, such as high-layer parameters and architecture, have much less impact on composition samples. Two DNNs with different usually fully sets samples, even though they similar performance. This finding extends...

This paper aims to analyze the generalization power of deep neural networks (DNNs) from perspective interactions. Unlike previous analysis a DNN's in highdimensional feature space, we find that DNN can be explained as We found generalizable interactions follow decay-shaped distribution, while non-generalizable spindle-shaped distribution. Furthermore, our theory effectively disentangle these two types DNN. have verified well match real experiments.

By mining "big GPS records" of 1.6 million users, an intelligent system automatically discovers, analyzes, and simulates population evacuations during the Great East Japan Earthquake Fukushima Daiichi nuclear accident.

In this paper, we use the interaction inside adversarial perturbations to explain and boost transferability. We discover prove negative correlation between transferability perturbations. The is further verified through different DNNs with various inputs. Moreover, can be regarded as a unified perspective understand current transferability-boosting methods. To end, that some classic methods of enhancing essentially decease interactions Based on this, propose directly penalize during attacking...

Compared to traditional learning from scratch, knowledge distillation sometimes makes the DNN achieve superior performance. In this paper, we provide a new perspective explain success of based on information theory, i.e., quantifying points encoded in intermediate layers for classification. To end, consider signal processing as layer-wise process discarding information. A point is referred an input unit, which discarded much less than that other units. Thus, propose three hypotheses...

The frequency and intensity of natural disasters has significantly increased over the past decades this trend is predicted to continue. Facing these possible unexpected disasters, understanding simulating human emergency mobility following will becomethe critical issue for planning effective humanitarian relief, disaster management, long-term societal reconstruction. However, due uniquenessof various unavailability reliable large scale data, such kind research very difficult be performed....

This paper aims to explain deep neural networks (DNNs) from the perspective of multivariate interactions. In this paper, we define and quantify significance interactions among multiple input variables DNN. Input with strong usually form a coalition reflect prototype features, which are memorized used by DNN for inference. We based on Shapley value, is designed assign attribution value each variable have conducted experiments various DNNs. Experimental results demonstrated effectiveness...