Weapons detection for security and video surveillance

  1. Summary
  2. Public datasets
  3. Published studies
    1. Automatic handgun detection alarm in videos using deep learning
    2. Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos with deep learning
    3. A binocular image fusion approach for minimizing false positives in handgun detection with deep learning
    4. Object Detection Binary Classifiers methodology based on deep learning to identify small objects handled similarly: Application in video surveillance
  4. Papers
  5. Authors

1. Summary

There are many places where the crime rate caused by firearms or knives is very high, especially in places where they are allowed. The early detection of potentially violent situations is of paramount importance for citizens security. One way to prevent these situations is by detecting the presence of dangerous objects such as handguns and knives in surveillance videos.

Current surveillance and control systems still require human monitoring and intervention. We present a system of automatic detection of weapons on video appropriate for surveillance and control purposes.

Our proposal is the early detection of these weapons using deep learning techniques through video security in real time and minimizing the computational burden. 

Different researching works carried out in this line are described below. We list the different image datasets built in these works in Section 2, as well as the proposed methods for improving the weapon detection system and results are illustrated in Section 3. Finally, the published works information is provided for citation and research purposes and the authors in Section 4 and 5.

2. Public datasets

The weapon detection task can be performed through different approaches that determine the type of required images. Therefore, the created datasets follow the image classification and object detection scheme and annotation including different objects:

  • Handguns
  • Knives
  • Weapons vs similar handled object

All dataset are depicted and public researching purpose, more information in provided in OpenData Weapon Detection.

3. Published studies

In an automatic weapon detection system the alarm must be activated when the system is completely confident about the presence of weapons in the scene. We describe below the published studies focussed on making a weapon detection system more robust by minimizing false positives and maximizing detection skills.

3.1 Automatic handgun detection alarm in videos using deep learning

R.Olmos et al. (2017) Automatic handgun detection alarm in video surveillance using deep learning. Neurocomputing. doi.org/10.1016/j.neucom.2017.05.012

This work presents a novel automatic handgun detection system in videos appropriate for both, surveillance and control purposes. We reformulate this detection problem into the problem of minimizing false positives and solve it by i) building the key training data-set guided by the results of a deep Convolutional Neural Network (CNN) classifier and ii) assessing the best classification model under two approaches, the sliding window approach and region proposal approach. 
The most promising results are obtained by Faster RCNN based model trained on our new database. The best detector shows a high potential even in low quality youtube videos and provides satisfactory results as automatic alarm system. Among 30 scenes, it successfully activates the alarm after five successive true positives in a time interval smaller than 0.2 s, in 27 scenes. We also define a new metric, Alarm Activation Time per Interval (AATpI), to assess the performance of a detection model as an automatic detection system in videos.

3.2 Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos with deep learning

A. Castillo et al. (2019) Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos with deep learning. Neurocomputing. doi.org/10.1016/j.neucom.2018.10.076

The automatic detection of cold steel weapons handled by one or multiple persons in surveillance videos can help reducing crimes. However, the detection of these metallic objects in videos faces an important problem: their surface reflectance under medium to high illumination conditions blurs their shapes in the image and hence makes their detection impossible. 
The objective of this work is two-fold: (i) To develop an automatic cold steel weapon detection model for video surveillance using Convolutional Neural Networks(CNN) and (ii) strengthen its robustness to light conditions by proposing a brightness guided preprocessing procedure called DaCoLT (Darkening and Contrast at Learning and Test stages). 
The obtained detection model provides excellent results as a cold steel weapon detector and as an automatic alarm system in video surveillance.

Results of this study are ilustrated in next content: Results videos

3.3 A binocular image fusion approach for minimizing false positives in handgun detection with deep learning

R.Olmos et al. (2019) A binocular image fusion approach for minimizing false positives in handgun detection with deep learning. Information fusion. doi.org/10.1016/j.inffus.2018.11.015

Object detection models have known important improvements in the recent years. The state-of-the art detectors are end-to-end CNN based models that reach good mean average precisions, around 73%, on benchmarks of high quality images. However, these models still produce a large number of false positives in low quality videos such as, surveillance videos. This paper proposes a novel image fusion approach to make the detection model focus on the area of interest where the action is more likely to happen in the scene. 
We propose building a low cost symmetric dual camera system to compute the disparity map and exploit this information to improve the selection of candidate regions from the input frames. From our results, the proposed approach not only reduces the number of false positives but also improves the overall performance of the detection model which makes it appropriate for object detection in surveillance videos.

3.4 Object Detection Binary Classifiers methodology based on deep learning to identify small objects handled similarly: Application in video surveillance

F. Pérez-Hernández et al. (2020) Object detection binary classifiers methodology based on deep learning to identify small objects handled similarly: application in video surveillance. Knowledge-Based-Systems. doi.org/10.1016/j.knosys.2020.105590

The capability of distinguishing between small objects when manipulated with hand is essential in many fields, especially in video surveillance. To date, the recognition of such objects in images using Convolutional Neural Networks (CNNs) remains a challenge. In this paper, we propose improving robustness, accuracy and reliability of the detection of small objects handled similarly using binarization techniques. We propose improving their detection in videos using a two level methodology based on deep learning, called Object Detection with Binary Classifiers. The first level selects the candidate regions from the input frame and the second level applies a binarization technique based on a CNN-classifier with One-Versus-All or One-Versus-One. In particular, we focus on the video surveillance problem of detecting weapons and objects that can be confused with a handgun or a knife when manipulated with hand. We create a database considering six objects: pistol, knife, smartphone, bill, purse and card. The experimental study shows that the proposed methodology reduces the number of false positives with respect to the baseline multi-class detection model.

Results of this study are ilustrated in next content: Results videos

4. Papers

Olmos, R., Tabik, S., & Herrera, F. (2018).
Automatic handgun detection alarm in videos using deep learning.
Neurocomputing, 275, 66-72.
Link to the article - PDF

Castillo, A., Tabik, S., Pérez, F., Olmos, R., Herrera, F. (2019).
Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos using deep learning.
Neurocomputing, 330, 151-161.
Link to the article - PDF

Olmos, R., Tabik, S., Lamas, A., Pérez-Hernández, F., Herrera, F. (2019).
A binocular image fusion approach for minimizing false positives in handgun detection with deep learning
Information Fusion, 49, 271-280.
Link to the article - PDF

Pérez-Hernández, F., Tabik, S., Lamas, A., Olmos, R., Fujita, H., Herrera, F. (2020).
Object Detection Binary Classifiers methodology based on deep learning to identify small objects handled similarly: Application in video surveillance.
Knowledge-Based Systems, 194, 105590.
Link to the article - PDF

5. Authors

 Francisco Herrera Triguero Google Scholar Researcher ID Orcid E-Mail

 Siham Tabik Google Scholar Researcher ID Orcid E-Mail

 Alberto Castillo Lamas Google Scholar Researcher ID Orcid E-Mail

 Francisco Pérez Hernández Google Scholar Researcher ID Orcid E-Mail

 Roberto Olmos Pimentel Google Scholar E-Mail

 

Page Maintained by Francisco Pérez Hernández y Alberto Castillo Lamas