Multimodal Collaborative Sensing and Identification Using Deep Learning for Intelligence, Surveillance and Reconnaissance (ISR) Applications
Project Status: Active (Started: 2014)
Research Description
Ubiquitous sensors of various types have drastically enhanced human beings’ sensing and perceptive power so that people can see, listen and feel from very long distances, in dark night, under hazardous conditions, and in the world measured by nanometers. The seamless integration of sensing and computing has made many revolutionary breakthroughs recently, such as the driverless cars and unmanned drones in active service for the military operations. In this proposed work, we will endeavor to combine the penetrating sensing power of Laser Doppler Vibrometry (LDV) sensors and other sensing modalities such as Optical-Electro (EO), Infrared (IR), 3D laser scanners, and micro-phone arrays on stationary and/or mobile platforms to perform collaborative sensing. With the immense computing power of computational platforms and deep learning algorithms, the proposed work aims to integrate multimodal and collaborative sensing with big data analytics and deep learning. The main objective of this proposed project is to gain deeper insights into both the sensing phenomenology and the corresponding computing procedures using signals collected by multi-modal sensors and exploited them effectively in a broad spectrum of ISR applications such as vehicle identification and facility inspection.
Sponsors
AFOSR Grant (Award#19RT0416) on Dynamic data driven applications systems with multimodal sensing, collaborative perception and deep computing (PI: Jie Wei; CoI: Zhigang Zhu, 2021- 2024) [CUNY Newswire] [CCNY News] [GSOE LinkedIn] [CCNY Tweet]
DoD ARO (Award# 12037379/P68871-CS-REP), Acquisition of Laser Doppler Vibrometers for a Computational Vibrometry System: from Microscopy Analysis to Remote Surveillance and Inspection. PI: Jie Wei (CS); Co-PIs: Bingmei Fu (BME), Karen Hubbard (Bio), Akira Kawaguchi (CS), Zhigang Zhu (CS). Duration: One year (06/02/2016-06/01/2017)
Air Force Research Laboratory under the Research Collaboration Program (RCP) (Award #FA8650-13-C-5800), “Multimodal Collaborative Sensing and Identification Using Deep Learning for Intelligence, Surveillance and Reconnaissance (ISR) Applications” (PI: Jie Wei; Co-PI: Zhigang Zhu), 9/1/2016 to 9/30/2018, renewable up to five years.
NSF I/UCRC: Center for Surveillance Research (Award # IIP-0968922), Research and Development of Multimodal Features and Algorithms for Vehicle Classification, PI: Jie Wei, Co-PI: Zhigang Zhu, 09/01/2014-08/31/2016