Skip to main content

A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators

Publication ,  Conference
Ju, R; Chu, H; Wang, Y; Deng, Q; Cheng, M; Li, M
Published in: ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction
October 18, 2021

As elevator accidents do great damage to people's lives and property, taking immediate responses to emergent calls for help is necessary. In most emergency cases, passengers must use the "SOS"button to contact the remote safety guard. However, this method is unreliable when passengers lose the ability of body movement. To address this problem, we define a novel task of identifying real and fake calls for help in elevator scenes. Given that the limited call for help dataset collected in elevators contains multimodal data of real and fake categories, we collected and constructed an audiovisual dataset dedicated to the proposed task. Moreover, we present a novel instance-modality-wise dynamic framework to efficiently use the information from each modality and make inferences. Experimental results show that our multimodal network improves the performance on the call for help multimodal dataset by 2.66% (accuracy) and 1.25% (F1 Score) with respect to the pure audio model. Besides, our method outperforms other methods on our dataset.

Duke Scholars

Published In

ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction

DOI

Publication Date

October 18, 2021

Start / End Page

112 / 120
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Ju, R., Chu, H., Wang, Y., Deng, Q., Cheng, M., & Li, M. (2021). A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators. In ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction (pp. 112–120). https://doi.org/10.1145/3461615.3491112
Ju, R., H. Chu, Y. Wang, Q. Deng, M. Cheng, and M. Li. “A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators.” In ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction, 112–20, 2021. https://doi.org/10.1145/3461615.3491112.
Ju R, Chu H, Wang Y, Deng Q, Cheng M, Li M. A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators. In: ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction. 2021. p. 112–20.
Ju, R., et al. “A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators.” ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction, 2021, pp. 112–20. Scopus, doi:10.1145/3461615.3491112.
Ju R, Chu H, Wang Y, Deng Q, Cheng M, Li M. A Multimodal Dynamic Neural Network for Call for Help Recognition in Elevators. ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction. 2021. p. 112–120.

Published In

ICMI 2021 Companion - Companion Publication of the 2021 International Conference on Multimodal Interaction

DOI

Publication Date

October 18, 2021

Start / End Page

112 / 120