Electronic and Cyber Defense

Electronic and Cyber Defense

Violent behavior detection in surveillance cameras using convolutional and memory neural networks

Document Type : Original Article

Authors
Ahmad Dolat Khah 1
Milad Asadpour 2
Raheb Hashempour 3
behnam dorostkar 4
1 Researcher, Amin University , Tehran, Iran
2 PhD Student, University of Tehran, Tehran, Iran
3 Master's degree, Amin University .Tehran, iran
4 Assistant Professor, Amin University . Tehran. Iran
Abstract
The existence of security is mandatory in any society and it is the basis for the progress and development of a country as easily and quickly as possible, so all countries try to establish stable security by controlling the level of violence and strife in the society. On the other hand, due to the limitation of manpower, it is not possible to carry out the entire process of providing security through the traditional and common methods of the past, and in this regard, new and up-to-date equipment and technologies must be used. and advanced countries of the world, the use of closed-circuit and surveillance cameras in public places is in this research, an expert system based on two sets of neural network ResNet101 and memory LSTM with the aim of reducing the amount of computation while maintaining proper accuracy, ResNet101 network is presented With a total of 347 layers and through the transfer learning method, it extracts the spatio-temporal features of consecutive video frames, and then the LSTM network with a total of 9 layers is responsible for detecting violent behavior in the video. These two sets have been optimized in terms of the type of layer arrangement, the way of connection and the number of cells in each layer so that they can have the best performance in all video conditions, including low quality, presence of noise and short video, etc. As a result of this intelligent system, they can detect violent behavior in closed-circuit cameras with an accuracy of 86.28% in real-time and instantaneously in low-quality video images of 224x3x224 pixels, and in case of violence, report it to the relevant people. inform In the end, it should be mentioned that the designed system, by reducing the amount of computing while maintaining the accuracy, has been able to perform effective and appropriate online monitoring of low-quality surveillance cameras by using only 22 frames per 5 seconds of video.
Keywords
violence detection
machine vision
artificial neural networks
surveillance cameras
Subjects

Smiley face

 

[1] I. Serrano Gracia, O. Deniz Suarez, G. Bueno Garcia, and T.-K. Kim, "Fast fight detection," PloS one, vol. 10, no. 4, p. e0120448, 2015.
[2] S. U. Khan, I. U. Haq, S. Rho, S. W. Baik, and M. Y. Lee, "Cover the violence: A novel Deep-Learning-Based approach towards violence-detection in movies," Applied Sciences, vol. 9, no. 22, p. 4963, 2019.
 [3] P. Wu, J. Liu, Y. Shi, Y. Sun, F. Shao, Z. Wu, and Z. Yang, "Not only look, but also listen: Learning multimodal violence detection under weak supervision," in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, 2020: Springer, pp. 322-339.
[4] J. Mahmoodi and A. Salajeghe, "A classification method based on optical flow for violence detection," Expert systems with applications, vol. 127, pp. 121-127, 2019.
[5] I. Febin, K. Jayasree, and P. T. Joy, "Violence detection in videos for an intelligent surveillance system using MoBSIFT and movement filtering algorithm," Pattern Analysis and Applications, vol. 23, no. 2, pp. 611-623, 2020.
[6] R. A. Pratama, N. Yudistira, and F. A. Bachtiar, "Violence recognition on videos using two-stream 3D CNN with custom spatiotemporal crop," Multimedia Tools and Applications, pp. 1-23, 2023.
[7] A. Traoré and M. A. Akhloufi, "2D bidirectional gated recurrent unit convolutional neural networks for end-to-end violence detection in videos," in International Conference on Image Analysis and Recognition, 2020: Springer, pp. 152-160.
[8] A. Srivastava, T. Badal, and R. Singh, "Real life violence detection in surveillance videos using spatiotemporal features," in 2021 Thirteenth International Conference on Contemporary Computing (IC3-2021), 2021, pp. 262-266.
[9] S. Chaudhary, M. A. Khan, and C. Bhatnagar, "Multiple anomalous activity detection in videos," Procedia Computer Science, vol. 125, pp. 336-345, 2018.
[10] C. Ding, S. Fan, M. Zhu, W. Feng, and B. Jia, "Violence detection in video by using 3D convolutional neural networks," in Advances in Visual Computing: 10th International Symposium, ISVC 2014, Las Vegas, NV, USA, December 8-10, 2014, Proceedings, Part II 10, 2014: Springer, pp. 551-558.
[11] F. Santos, D. Durães, F. S. Marcondes, N. Hammerschmidt, S. Lange, J. Machado, and P. Novais, "In-car violence detection based on the audio signal," in International conference on intelligent data engineering and automated learning, 2021: Springer, pp. 437-445.
[12] T. Zhang, Z. Yang, W. Jia, B. Yang, J. Yang, and X. He, "A new method for violence detection in surveillance scenes," Multimedia Tools and Applications, vol. 75, pp. 7327-7349, 2016.
[13] A. Mumtaz, A. B. Sargano, and Z. Habib, "Violence detection in surveillance videos with deep network using transfer learning," in 2018 2nd European Conference on Electrical Engineering and Computer Science (EECS), 2018: IEEE, pp. 558-563.
[14] Torabipour, Tobi, Siadat, Sayeda Safiya. A method for predicting the stock price of Tehran stock market in relation to knowledge. electronic and cyber defense, 1401; 10 (4): 91-100[in persian]
[15] F. J. Rendón-Segador, J. A. Álvarez-García, F. Enríquez, and O. Deniz, "Violencenet: Dense multi-head self-attention with bidirectional convolutional lstm for detecting violence," Electronics, vol. 10, no. 13, p. 1601, 2021.
[16] B. M. Peixoto, S. Avila, Z. Dias, and A. Rocha, "Breaking down violence: A deep-learning strategy to model and classify violence in videos," in Proceedings of the 13th International Conference on Availability, Reliability and Security, 2018, pp. 1-7.
[17] A. Jain and D. K. Vishwakarma, "Deep NeuralNet for violence detection using motion features from dynamic images," in 2020 third international conference on smart systems and inventive technology (ICSSIT), 2020: IEEE, pp. 826-831.
[18] Z. Islam, M. Rukonuzzaman, R. Ahmed, M. H. Kabir, and M. Farazi, "Efficient two-stream network for violence detection using separable convolutional lstm," in 2021 International Joint Conference on Neural Networks (IJCNN), 2021: IEEE, pp. 1-8.
[19] S. Vosta and K.-C. Yow, "A cnn-rnn combined structure for real-world violence detection in surveillance cameras," Applied Sciences, vol. 12, no. 3, p. 1021, 2022.
[20] Irfanullah, T. Hussain, A. Iqbal, B. Yang, and A. Hussain, "Real time violence detection in surveillance videos using Convolutional Neural Networks," Multimedia Tools and Applications, vol. 81, no. 26, pp. 38151-38173, 2022.
[21]. V. D. Huszár, V. K. Adhikarla, I. Négyesi, and C. Krasznay, "Toward Fast and Accurate Violence Detection for Automated Video Surveillance Applications," IEEE Access, vol. 11, pp. 18772-18793, 2023.
[22] S. Singh and B. Tyagi, "Computational Comparison of CNN Based Methods for Violence Detection," 2023.
[23] A. L. Detzel, H. Liu, J. Strauss, G. Zhou, and Y. Zhu, "Bitcoin: Learning and predictability via technical analysis," in Paris December 2018 Finance Meeting EUROFIDAI-AFFI, 2019, vol. 1, p. 314.
[24] T. Zhang, Z. Yang, W. Jia, B. Yang, J. Yang, and X. He, "A new method for violence detection in surveillance scenes," Multimedia Tools and Applications, vol. 75, pp. 7327-7349, 2016.
[25] A. Sherstinsky, "Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network," Physica D: Nonlinear Phenomena, vol. 404, p. 132306, 2020.
Electronic and Cyber Defense
Volume 12, Issue 4 - Serial Number 48
Winter
Winter 2025
Pages 95-106

  • Receive Date 20 August 2024
  • Revise Date 08 December 2024
  • Accept Date 02 January 2025
  • Publish Date 20 January 2025