Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

• DOI: 10.1016/j.neuroimage.2016.03.063
• Publication Date: 2016-04-02T21:52:24Z
• Authors: Radoslaw Martin C...; Aditya Khosla; Dimitrios Pantazis; Aude Oliva
• Keywords: Adult; Male; 0301 basic medicine; Spatial layout; Cognitive Neuroscience; Models, Neurological; Deep neural network; Article; Young Adult; 03 medical and health sciences; 0302 clinical medicine; Humans; Representational similarity analysis; Cerebral Cortex; Brain Mapping; Magnetoencephalography; Neurology; Pattern Recognition, Visual; Multivariate Analysis; Female; Neural Networks, Computer; Scene perception; Photic Stimulation

ABSTRACTHuman scene recognition is a rapid multistep process evolving over time from single scene image to spatial layout processing. We used multivariate pattern analyses on magnetoencephalography (MEG) data to unravel the time course of this cortical process. Following an early signal for lower-level visual analysis of single scenes at ~100ms, we found a marker of real-world scene size, i.e. spatial layout processing, at ~250ms indexing neural representations robust to changes in unrelated scene properties and viewing conditions. For a quantitative explanation that captures the complexity of scene recognition, we compared MEG data to a deep neural network model trained on scene classification. Representations of scene size emerged intrinsically in the model, and resolved emerging neural scene size representation. Together our data provide a first description of an electrophysiological signal for layout processing in humans, and a novel quantitative model of how spatial layout representations may emerge in the human brain.