Skip to main content

Time-attentive fusion network: An efficient model for online detection of action start

Publication ,  Journal Article
Hu, X; Wang, S; Li, M; Li, Y; Du, S
Published in: IET Image Processing
May 29, 2024

Online detection of action start is a significant and challenging task that requires prompt identification of action start positions and corresponding categories within streaming videos. This task presents challenges due to data imbalance, similarity in boundary content, and real-time detection requirements. Here, a novel Time-Attentive Fusion Network is introduced to address the requirements of improved action detection accuracy and operational efficiency. The time-attentive fusion module is proposed, which consists of long-term memory attention and the fusion feature learning mechanism, to improve spatial-temporal feature learning. The temporal memory attention mechanism captures more effective temporal dependencies by employing weighted linear attention. The fusion feature learning mechanism facilitates the incorporation of current moment action information with historical data, thus enhancing the representation. The proposed method exhibits linear complexity and parallelism, enabling rapid training and inference speed. This method is evaluated on two challenging datasets: THUMOS’14 and ActivityNet v1.3. The experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art methods in terms of both detection accuracy and inference speed.

Duke Scholars

Published In

IET Image Processing

DOI

EISSN

1751-9667

ISSN

1751-9659

Publication Date

May 29, 2024

Volume

18

Issue

7

Start / End Page

1892 / 1902

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4607 Graphics, augmented reality and games
  • 4603 Computer vision and multimedia computation
  • 0906 Electrical and Electronic Engineering
  • 0801 Artificial Intelligence and Image Processing
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Hu, X., Wang, S., Li, M., Li, Y., & Du, S. (2024). Time-attentive fusion network: An efficient model for online detection of action start. IET Image Processing, 18(7), 1892–1902. https://doi.org/10.1049/ipr2.13071
Hu, X., S. Wang, M. Li, Y. Li, and S. Du. “Time-attentive fusion network: An efficient model for online detection of action start.” IET Image Processing 18, no. 7 (May 29, 2024): 1892–1902. https://doi.org/10.1049/ipr2.13071.
Hu X, Wang S, Li M, Li Y, Du S. Time-attentive fusion network: An efficient model for online detection of action start. IET Image Processing. 2024 May 29;18(7):1892–902.
Hu, X., et al. “Time-attentive fusion network: An efficient model for online detection of action start.” IET Image Processing, vol. 18, no. 7, May 2024, pp. 1892–902. Scopus, doi:10.1049/ipr2.13071.
Hu X, Wang S, Li M, Li Y, Du S. Time-attentive fusion network: An efficient model for online detection of action start. IET Image Processing. 2024 May 29;18(7):1892–1902.

Published In

IET Image Processing

DOI

EISSN

1751-9667

ISSN

1751-9659

Publication Date

May 29, 2024

Volume

18

Issue

7

Start / End Page

1892 / 1902

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4607 Graphics, augmented reality and games
  • 4603 Computer vision and multimedia computation
  • 0906 Electrical and Electronic Engineering
  • 0801 Artificial Intelligence and Image Processing