Violence Detection Through Fusing Visual Information to Auditory Scene

Hongwei Li; Lin Ma; Xinyu Min; Haifeng Li

doi:10.1007/978-981-99-2401-1_19

Violence Detection Through Fusing Visual Information to Auditory Scene

Hongwei Li
, Lin Ma
, Xinyu Min
, Haifeng Li^*

^*Corresponding author for this work

Faculty of Computing, Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In the field of audio and video detection, violence detection is a crucial task with significant theoretical and practical implications. In order to solve the present issue of the lack of violent audio datasets, we first created our own audio violent dataset named VioAudio. Then, we proposed a CNN-ConvLSTM network model for audio violence detection, which obtained an accuracy of 91.5% on VioAudio and a MAP value of 16.47% on the MediaEval 2015 dataset. Meanwhile, this paper integrated self-attention mechanisms and visual information into CNN-ConvLSTM network in order to address the issue of modality singularity in violence detection, and then confirmed them on MediaEval2015 dataset. The experimental results demonstrate that after fusing visual and auditory information, the CNN-LSTM network model greatly enhanced recognition accuracy, attaining a 31.25% MAP value, which is 1.94% higher than the best result. The method proposed in this paper considerably increased the accuracy of violence detection and offered fresh perspectives on how to integrate multimodal information to identify violence.

Original language	English
Title of host publication	Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings
Editors	Ling Zhenhua, Gao Jianqing, Yu Kai, Jia Jia
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	208-220
Number of pages	13
ISBN (Print)	9789819924004
DOIs	https://doi.org/10.1007/978-981-99-2401-1_19
State	Published - 2023
Externally published	Yes
Event	17th National Conference on Man-Machine Speech Communication, NCMMSC 2022 - Hefei, China Duration: 15 Dec 2022 → 18 Dec 2022

Publication series

Name	Communications in Computer and Information Science
Volume	1765 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	17th National Conference on Man-Machine Speech Communication, NCMMSC 2022
Country/Territory	China
City	Hefei
Period	15/12/22 → 18/12/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 16 Peace, Justice and Strong Institutions

Keywords

Auditory and Visual Information Fusion
Convolution Neural Network
Long-Short Term Memory Network
Violence Detection

Access to Document

10.1007/978-981-99-2401-1_19

Cite this

Li, H., Ma, L., Min, X., & Li, H. (2023). Violence Detection Through Fusing Visual Information to Auditory Scene. In L. Zhenhua, G. Jianqing, Y. Kai, & J. Jia (Eds.), Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings (pp. 208-220). (Communications in Computer and Information Science; Vol. 1765 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-2401-1_19

Li, Hongwei ; Ma, Lin ; Min, Xinyu et al. / Violence Detection Through Fusing Visual Information to Auditory Scene. Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings. editor / Ling Zhenhua ; Gao Jianqing ; Yu Kai ; Jia Jia. Springer Science and Business Media Deutschland GmbH, 2023. pp. 208-220 (Communications in Computer and Information Science).

@inproceedings{f9273a3dced3442ca46a445901ab4c9c,

title = "Violence Detection Through Fusing Visual Information to Auditory Scene",

abstract = "In the field of audio and video detection, violence detection is a crucial task with significant theoretical and practical implications. In order to solve the present issue of the lack of violent audio datasets, we first created our own audio violent dataset named VioAudio. Then, we proposed a CNN-ConvLSTM network model for audio violence detection, which obtained an accuracy of 91.5\% on VioAudio and a MAP value of 16.47\% on the MediaEval 2015 dataset. Meanwhile, this paper integrated self-attention mechanisms and visual information into CNN-ConvLSTM network in order to address the issue of modality singularity in violence detection, and then confirmed them on MediaEval2015 dataset. The experimental results demonstrate that after fusing visual and auditory information, the CNN-LSTM network model greatly enhanced recognition accuracy, attaining a 31.25\% MAP value, which is 1.94\% higher than the best result. The method proposed in this paper considerably increased the accuracy of violence detection and offered fresh perspectives on how to integrate multimodal information to identify violence.",

keywords = "Auditory and Visual Information Fusion, Convolution Neural Network, Long-Short Term Memory Network, Violence Detection",

author = "Hongwei Li and Lin Ma and Xinyu Min and Haifeng Li",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 17th National Conference on Man-Machine Speech Communication, NCMMSC 2022 ; Conference date: 15-12-2022 Through 18-12-2022",

year = "2023",

doi = "10.1007/978-981-99-2401-1\_19",

language = "英语",

isbn = "9789819924004",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "208--220",

editor = "Ling Zhenhua and Gao Jianqing and Yu Kai and Jia Jia",

booktitle = "Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings",

address = "德国",

}

Li, H , Ma, L, Min, X & Li, H 2023, Violence Detection Through Fusing Visual Information to Auditory Scene. in L Zhenhua, G Jianqing, Y Kai & J Jia (eds), Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings. Communications in Computer and Information Science, vol. 1765 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 208-220, 17th National Conference on Man-Machine Speech Communication, NCMMSC 2022, Hefei, China, 15/12/22. https://doi.org/10.1007/978-981-99-2401-1_19

Violence Detection Through Fusing Visual Information to Auditory Scene. / Li, Hongwei ; Ma, Lin; Min, Xinyu et al.
Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings. ed. / Ling Zhenhua; Gao Jianqing; Yu Kai; Jia Jia. Springer Science and Business Media Deutschland GmbH, 2023. p. 208-220 (Communications in Computer and Information Science; Vol. 1765 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Violence Detection Through Fusing Visual Information to Auditory Scene

AU - Li, Hongwei

AU - Ma, Lin

AU - Min, Xinyu

AU - Li, Haifeng

PY - 2023

Y1 - 2023

N2 - In the field of audio and video detection, violence detection is a crucial task with significant theoretical and practical implications. In order to solve the present issue of the lack of violent audio datasets, we first created our own audio violent dataset named VioAudio. Then, we proposed a CNN-ConvLSTM network model for audio violence detection, which obtained an accuracy of 91.5% on VioAudio and a MAP value of 16.47% on the MediaEval 2015 dataset. Meanwhile, this paper integrated self-attention mechanisms and visual information into CNN-ConvLSTM network in order to address the issue of modality singularity in violence detection, and then confirmed them on MediaEval2015 dataset. The experimental results demonstrate that after fusing visual and auditory information, the CNN-LSTM network model greatly enhanced recognition accuracy, attaining a 31.25% MAP value, which is 1.94% higher than the best result. The method proposed in this paper considerably increased the accuracy of violence detection and offered fresh perspectives on how to integrate multimodal information to identify violence.

AB - In the field of audio and video detection, violence detection is a crucial task with significant theoretical and practical implications. In order to solve the present issue of the lack of violent audio datasets, we first created our own audio violent dataset named VioAudio. Then, we proposed a CNN-ConvLSTM network model for audio violence detection, which obtained an accuracy of 91.5% on VioAudio and a MAP value of 16.47% on the MediaEval 2015 dataset. Meanwhile, this paper integrated self-attention mechanisms and visual information into CNN-ConvLSTM network in order to address the issue of modality singularity in violence detection, and then confirmed them on MediaEval2015 dataset. The experimental results demonstrate that after fusing visual and auditory information, the CNN-LSTM network model greatly enhanced recognition accuracy, attaining a 31.25% MAP value, which is 1.94% higher than the best result. The method proposed in this paper considerably increased the accuracy of violence detection and offered fresh perspectives on how to integrate multimodal information to identify violence.

KW - Auditory and Visual Information Fusion

KW - Convolution Neural Network

KW - Long-Short Term Memory Network

KW - Violence Detection

UR - https://www.scopus.com/pages/publications/85161580036

U2 - 10.1007/978-981-99-2401-1_19

DO - 10.1007/978-981-99-2401-1_19

M3 - 会议稿件

AN - SCOPUS:85161580036

SN - 9789819924004

T3 - Communications in Computer and Information Science

SP - 208

EP - 220

BT - Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings

A2 - Zhenhua, Ling

A2 - Jianqing, Gao

A2 - Kai, Yu

A2 - Jia, Jia

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th National Conference on Man-Machine Speech Communication, NCMMSC 2022

Y2 - 15 December 2022 through 18 December 2022

ER -

Li H , Ma L, Min X, Li H. Violence Detection Through Fusing Visual Information to Auditory Scene. In Zhenhua L, Jianqing G, Kai Y, Jia J, editors, Man-Machine Speech Communication - 17th National Conference, NCMMSC 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 208-220. (Communications in Computer and Information Science). doi: 10.1007/978-981-99-2401-1_19