Citation Author(s):
Mainak Chakraborty Defence Institute of Advanced Technology (DIAT), Girinagar, Pune 411025, India Harish C.Kumawat Defence Institute of Advanced Technology (DIAT), Girinagar, Pune 411025, India Sunita Vikrant Dhavale Defence Institute of Advanced Technology (DIAT), Girinagar, Pune 411025, India A. ArockiaBazil Raj Defence Institute of Advanced Technology (DIAT), Girinagar, Pune 411025, India
Submitted by:Mainak ChakrabortyLast updated:Thu, 10/27/2022 – 23:57DOI:10.21227/015m-7415Data Format:
Image JPG File (.JPG) and .MAT
Research Article Link:
DIAT-μRadHAR (Micro-Doppler Signature Dataset) & μRadNet (A Lightweight DCNN)—For Human Suspicious Activity Recognition
Links:
DIAT-RadHARNet: A Lightweight DCNN for Radar Based Classification of Human Suspicious Activities
Application of DNN for radar micro-doppler signature-based human suspicious activity recognition
License:
Creative Commons Attribution
5212 Views
Categories:
Artificial Intelligence
Signal Processing
Machine Learning
Sensors
Image Processing
Computational Intelligence
Computer Vision
Keywords:
convolutional neural network, deep convolution neural network (DCNN)-based classification, human suspicious activity, micro-Doppler (m-Doppler) signatures, X-band continuous wave (CW) radar.
ABSTRACT
In the view of national security, radar micro-Doppler (m-D) signatures-based recognition of suspicious human activities becomes significant. In connection to this, early detection and warning of terrorist activities at the country borders, protected/secured/guarded places and civilian violent protests is mandatory. Designing an automated human suspicious activities: army crawling, army jogging, jumping with holding a gun, army marching, boxing, and stone-pelting/grenades-throwing, recognition system using a suitable deep convolutional neural network (DCNN) model is rapidly growing due to its inherent in-depth features extraction capability. As a value addition to this research, an X-band continuous wave (CW) 10 GHz radar has been developed at our radar systems laboratory and used to acquire the m-D signatures, to prepare a dataset (DIAT-μRadHAR) corresponding to above mentioned suspicious activities. In order to prepare a realistic dataset, human targets of different heights, weights, and gender are directed to perform the suspicious activities in front of the radar at different ranges between 10 m – 0.5 km and at different target aspect angles (0°, ±15°, ±30° and ±45°).
I. Introduction
Human activity recognition (HAR) plays a vital role in national security, military surveillance [1], protest-crowd assessment, rescue missions, and health care, etc. [2]. The HAR using computer vision-based techniques have numerous inherent limitations, such as; lighting requirement, short-range operation, bleariness in IR/night-vision camera images, need of clear weather conditions, and background light influence, etc., [3]. Annotation of human minute activities from vision-based images and/or videos is a labor-intensive and more time-consuming task [4]. Hence, HAR using radar is an emerging research topic, nowadays, due to its robustness for reliable operations even; in low light conditions, in adverse weather conditions, for long-range operations [5], for air and/or foliage targets imaging [6], for ground penetrative detections, and through-wall imaging etc., [7]–[9]. Each part of the human body’s movements are dissimilar for different activities, and they generate unique micro-Doppler (m-D) signatures when we illuminate the human target by a radar system [10]. Radar-based HAR using the deep learning (DL) models can provide the promising solution even for the mission of long-range and/or adverse weather conditions [11], [12]. Thus, introducing the DL technique in the radar receiver chain will be useful to process/extract the necessary features from the m-D signatures and automatically recognize suspicious human activities. From the national security point of view, human suspicious activities detection becomes significant for activating the necessary counter-attack/measure systems [2], [13]. In the protest place, country borders, terrorist attack/bomb-blast locations, the more common human suspicious activities, from the military/police point of view are (i) person fight punching (boxing) during the one-to-one attack, (ii) person intruding for pre-attack surveillance (army marching), (iii) person training (army jogging), (iv) person shooting (or escaping) with a rifle (jumping with holding a gun), (v) stone/hand-grenade throwing for damage/blasting (stone-pelting/grenades-throwing), and (vi) person hidden translation for attack execution or escape (army crawling) [14]–[16]. Detecting/classifying these kinds of suspicious human activities correctly, better at the earliest, by processing their m-D signatures is very much essential. Thus, generating a dataset (DIAT-RadHAR) containing sufficient amount of experimentally collected m-D signature images, designing a lightweight deep convolutional neural network (DCNN) model (RadNet), and testing/analyzing the performance of the developed DCNN model with open-field experiments are significant to have an automated, accurate human suspicious activities detection and recognition system; which are the main contributions presented in this paper.
INSTRUCTIONS:
In our dataset, the total number of spectrogram images generated using the open-field experiments is 3780, and the class-wise details can be found in our journal articles (1) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. A. B. Raj, “DIAT-μ RadHAR (Micro-Doppler Signature Dataset) & μ RadNet (A Lightweight DCNN)—For Human Suspicious Activity Recognition,” in IEEE Sensors Journal, vol. 22, no. 7, pp. 6851-6858, 1 April1, 2022, doi: 10.1109/JSEN.2022.3151943. (2) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. B. Raj A., “DIAT-RadHARNet: A Lightweight DCNN for Radar Based Classification of Human Suspicious Activities,” in IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-10, 2022, Art no. 2505210, doi: 10.1109/TIM.2022.3154832. (3) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. B. Raj A., “Application of DNN for radar micro-doppler signature-based human suspicious activity recognition.” in Pattern Recognition Letters, vol. 162 , pp. 1-6, 2022, doi: https://doi.org/10.1016/j.patrec.2022.08.005.
The dataset consist of 3780 spectrogram images (Image JPG File (.JPG) and .MAT) corresponding to micro-Doppler signatures of different human activities; namely (a) army marching, (b) Stone pelting/Grenades throwing, (c) jumping with holding a gun, (d) army Jogging, (e) army crawling and (f) boxing activities.
The DIAT-μRadHAR dataset is completely open to academic research. To use the dataset, please cite the following base/original papers:
(1) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. A. B. Raj, “DIAT-μ RadHAR (Micro-Doppler Signature Dataset) & μ RadNet (A Lightweight DCNN)—For Human Suspicious Activity Recognition,” in IEEE Sensors Journal, vol. 22, no. 7, pp. 6851-6858, 1 April1, 2022, doi: 10.1109/JSEN.2022.3151943.
(2) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. B. Raj A., “DIAT-RadHARNet: A Lightweight DCNN for Radar Based Classification of Human Suspicious Activities,” in IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-10, 2022, Art no. 2505210, doi: 10.1109/TIM.2022.3154832.
(3) M. Chakraborty, H. C. Kumawat, S. V. Dhavale and A. B. Raj A., “Application of DNN for radar micro-doppler signature-based human suspicious activity recognition.” in Pattern Recognition Letters, vol. 162 , pp. 1-6, 2022, doi: https://doi.org/10.1016/j.patrec.2022.08.00