Hierarchical Attention Network for Action Recognition in Videos

• DOI: 10.48550/arxiv.1607.06416
• Publication Date: 2016-01-01
• Authors: Yilin Wang; Suhang Wang; Jiliang Tang; Neil O’Hare; Yi Chang; Baoxin Li
• Keywords: Action Recognition

Understanding human actions in wild videos is an important task with a broad range of applications. In this paper we propose novel approach named Hierarchical Attention Network (HAN), which enables to incorporate static spatial information, short-term motion information and long-term video temporal structures for complex action understanding. Compared recent convolutional neural network based approaches, HAN has following advantages (1) can efficiently capture longer range; (2) able reveal transitions between frame chunks different time steps, i.e. it explicitly models the frames as well segments (3) multiple step attention mechanism, automatically learns regions video. The proposed model trained evaluated on standard benchmarks, i.e., UCF-101 HMDB-51, significantly outperforms state-of-the arts