Connectivity-inspired network for context-aware recognition

• DOI: 10.48550/arxiv.2409.04360
• Publication Date: 2024-01-01
• Authors: Carloni G.; Colantonio S.
• Keywords: I.2; FOS: Computer and information sciences; I.4; J.3; I.5; Computer Science - Artificial Intelligence; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; Electrical Engineering and Systems Science - Image and Video Processing; J.6; Context-aware recognition, Biological inspiration, Attention; Artificial Intelligence (cs.AI); I.2; I.4; I.5; J.3; J.6; FOS: Electrical engineering, electronic engineering, information engineering

The aim of this paper is threefold. We inform the AI practitioner about the human visual system with an extensive literature review; we propose a novel biologically motivated neural network for image classification; and, finally, we present a new plug-and-play module to model context awareness. We focus on the effect of incorporating circuit motifs found in biological brains to address visual recognition. Our convolutional architecture is inspired by the connectivity of human cortical and subcortical streams, and we implement bottom-up and top-down modulations that mimic the extensive afferent and efferent connections between visual and cognitive areas. Our Contextual Attention Block is simple and effective and can be integrated with any feed-forward neural network. It infers weights that multiply the feature maps according to their causal influence on the scene, modeling the co-occurrence of different objects in the image. We place our module at different bottlenecks to infuse a hierarchical context awareness into the model. We validated our proposals through image classification experiments on benchmark data and found a consistent improvement in performance and the robustness of the produced explanations via class activation. Our code is available at https://github.com/gianlucarloni/CoCoReco.<br/>ECCV 2024 - HCV Workshop, Accepted for presentation, Submitted Manuscript Version (adapted to include author names, Acknowledgements, and reference DOIs): the version of the manuscript improved after peer review will appear in the Proceedings later<br/>