Online Action Detection with Gated Cross Attention Module

Online action detection aims to identify action occurring in the most recent frame of streaming video. Due to its online nature and the presence of abundant background information in streaming videos, online action detection is quite challenging. In this paper, we introduce a solution to the challenging task of online action detection. We propose a gated cross attention module, which is concurrently applied to both the long-term memory encoder and the short-term memory decoder. This module dynamically adjusts attention weights by generating gated tensors and actively participating in attention calculation, thereby facilitating effective information reweighting. Moreover, we integrate background classification with online action detection to bolster action modeling. Besides, an online inference acceleration method is realized for our proposed algorithm. Experimental results demonstrate that our method achieves competitive performance against other state-of-the-art methods.