Drone and Controller Detection and Localization: Trends and Challenges

Abstract

Unmanned aerial vehicles (UAVs) have emerged as a rapidly growing technology seeing unprecedented adoption in various application sectors due to their viability and low cost. However, UAVs have also been used to perform illegal and malicious actions, which have recently increased. This creates a need for technologies capable of detecting, classifying, and deactivating malicious and unauthorized drones. This paper reviews the trends and challenges of the most recent UAV detection methods, i.e., radio frequency-based (RF), radar, acoustic, and electro-optical, and localization methods. Our research covers different kinds of drones with a major focus on multirotors. The paper also highlights the features and limitations of the UAV detection systems and briefly surveys the UAV remote controller detection methods.

1. Introduction

In recent years, there has been a significant advancement in unmanned aerial vehicles (UAVs). UAVs are widely used for commercial, civilian, and military applications due to their low cost, spatiotemporal coverage, and remote sensing capability. They have been specifically popular for collecting information in remote and inaccessible areas, such as military surveillance and search and rescue in floods or earthquakes [1,2,3].

An aircraft without onboard human command and control is called a UAV, also called a drone. Command and control are achieved autonomously by the embedded autopilot or remotely by the operators through a ground station [1]. Moreover, autonomous and remote controls can be integrated as a single UAV control mechanism. Over the years, the technology and features of UAVs have improved tremendously to address the varying requirements of different applications. In addition, ongoing research has been successful in finding ways to improve the performance of the UAV. Various designs and features that support their assigned missions in different fields and sectors have been proposed, such as shape structures, take-off, and landing techniques [2,4].

Surveillance applications use UAV technology to be integrated as a standalone, connected platform for information gathering. The human detection system in [4] was achieved through input from thermal images and videos from a thermal camera connected to a UAV. These images and videos are categorized by reference to a thermal dataset in the system and are processed by sequence operations to achieve the final result. In the military, some geographic areas are difficult to reach for monitoring and detecting unwanted signals or entities. The proposed system in [5] overcomes this demand.

Moreover, smart farming utilizes UAV technology for real-time monitoring and data acquisition of crop parameters, e.g., plant height, presence of weeds, or fungus. Tsouros et al. [1] discussed different types of UAVs and explored multiple applications of UAVs in precision agriculture, crop health, and growth monitoring. Moreover, this work reviews data acquisition technologies and aerial image processing methods. Further, civil engineering utilizes UAV technologies for seismic risk assessment, transportation management, disaster response, construction management, surveying and mapping, and flood monitoring and evaluation [2].

To summarize, UAV technologies have numerous features that enable their usage in multiple sectors and applications. The benefits of such technologies include: (1) reducing human risk, (2) lower energy consumption, (3) a lower cost, (4) flexibility, and (5) accuracy of data collection. The advances in electronics and sensor technology have widened the scope of UAV applications for their likely invaluable inclusion in police, fire brigades, and disaster management operations. However, in recent years, UAVs have been used to perform malicious actions, such as drug smuggling, intelligence gathering, and suicide attacks [2,3,5]. UAVs also pose a threat to surpassing restricted government or military sites. In addition, with the prevalence of smaller UAVs, concerns over public privacy are rising.

All these threats warrant an urgent need for research into UAV detection methods. It becomes strategic to detect and localize UAVs to prevent such malicious actions. Recently, various detection algorithms have been researched, such as active radar probes, acoustic recognition, infrared spectrum identification, visual recognition, and radio frequency (RF) signal detection [1,2,3,4,5,6,7]. This study aims to provide a detailed literature review of these detection methods, identify their strengths, explore various applications where they were used, and compare these methods for the major relevant studies in the open literature. Our study scope includes the detection and localization of multirotor and other UAV types. The study also reviews the techniques for the UAV controller localization of the detected drones. This review aims to survey the quickly evolving field, record what is notable and popular within this sector, and provide recommendations for future investigators.

Table 1. Summary of reviewed topics for drones and their controller detection.

Detection Technologies		Ref.
UAV Architecture and Security Concerns		[3,4,5,6,7,8]
UAE Detection Technologies	RF	[7,8,9,10,11,12,13,14,15,16,17,18,19,20]
	Radar	[21,22,23,24,25,26,27,28,29]
	Acoustic	[30,31,32,33,34,35,36,37,38,39,40]
	Electro-optical	[41,42,43,44,45,46]
	Hybrid fusion	[40,41,47,48]
Controller Detection and Localization		[12,18,49,50]

summarizes the covered topics in different sections of this study.

The rest of the study is organized as follows: Section 2 details the architecture of UAVs and associated security concerns with drones. A comprehensive review of UAV detection technologies is outlined in Section 3. The studies about drone controller localization are reviewed in Section 4. Lastly, Section 5 concludes the findings of the study.

2. UAV Architecture and Security Concerns

2.1. UAV Architecture

UAVs have multiple subsystems integrated to perform various operations, such as launch, fly, operate, process, transmit, and receive commands from remote or ground stations [3,5]. Four main UAV subsystems should be considered: (1) a power unit, (2) a communication module, (3) the main computing device, and (4) a sensor board. The power unit is designed to provide a longer lifetime for UAV operation without charging it [4]. The high-level architecture of the UAV system is illustrated in Figure 1, including UAV’s main computer processes commands based on the collected data from other subsystems or components (GPS, sensors, gyroscopes, accelerometers, antennas, receivers, etc.). These data or commands are transferred through a communication link between the UAV and the ground control station (GCS). This communication is mainly monitored to detect UAVs based on RF and radar-based technologies (details in Section 3.1 and Section 3.2).

A brief description of the major components of UAE architecture is as follows:

• UAV’s structure/airframe: There are many common features of a UAV’s chassis, such as lightweight, small size, endurance, aerodynamic flexibility, etc.
• Main computer: The critical part responsible for autonomous functioning and flight control. The computing subsystem processes sensed information, transmits it back, manages flight operations, and communicates with the control base.
• Sensors/payloads: UAVs can be equipped with a range of possible lightweight sensors per the application’s needs, including RGB cameras, thermal sensors, LiDAR sensors, and multispectral and hyperspectral sensors. All of them are connected to the flight controller to gather real-time data and process it for the missions’ execution.
• Communication link: UAVs are equipped with a high-quality wireless communication unit, including 5G, WiFi, Bluetooth, and radio-frequency identification (RFID), to facilitate communication with the GCS or the internet.
• Ground control station (GCS): This base station is mainly employed to monitor and control the UAV during its operation. Flight operation is continuously monitored and can be controlled to alter the mission.

2.2. Security Concerns

Regarding UAV security, two main topics are discussed in the literature: the security and safety of UAVs and the potential misuse of UAVs against critical infrastructures and privacy-related issues.

Threats to UAV security are well-researched concerning targeting its hardware, software, and communication module. In [6], threats to various components of the UAV system are discussed. GCS’s physical, network, and cloud security have been highlighted as vulnerabilities that can be exploited. Moreover, threats exist against the UAV communication according to the communication medium technology or type (WiFi, cellular network, GPS, and other RF solutions). The possible common attacks are eavesdropping, jamming, replay, denial of service, hijacking, etc. Other threats include mission disruption and itinerary tracking [7,8].

Despite promising application benefits using UAVs, threats also exist by their prevalence in the public domain. UAV security threats and incidents are mainly caused by privacy violations of sensitive sites, airplane flight disruption, damage and explosion in targeted areas, and sensitive data leakage through eavesdropping [7,8].

3. UAV Detection Methods

As mentioned in Section 2, there are many sectors in which UAVs have been explored and adopted, utilizing their practical and advanced features. The continuous development and improvement of UAV’s main systems and components, i.e., flight controller, sensors, gyroscopes, cameras, GPS, etc., increased the demand and reliance on UAVs for accomplishing different civilian and military missions. Moreover, they are widely available in the market at a reasonable cost compared to other solutions.

Research has been dedicated to designing, developing, and implementing systems for detecting malicious UAVs. Techniques of these systems are classified into passive and active. RF-based, acoustic, and vision-based techniques are among the passive technologies, whereas radar-based techniques are defined as active technologies. These technologies vary in operational conditions, covering range, consistency, accuracy, and many other parameters. This section focuses on UAV detection technologies and discusses the general framework and related work.

3.1. RF-Based

RF is used for UAV remote command and control communication. RF-based detection technologies rely on real-time sensing, capturing, processing, analyzing, and retrieving data from UAV’s RF-emitted signals. Acquired RF data are intended to identify, track and classify the detected UAV and localize the controller. RF-based techniques analyze the captured spectrum between the UAV and operators using circular or linear array antennas to detect both the drone and its controller in all-weather environments. As most of the communication between a drone and its controller occurs in the ISM band, around 2.4 GHz, the implementation cost of such a system is much lower compared to a radar-based solution [21,22,23,24,25,26,27,28,51].

In RF-based detection technologies, RF and WiFi-based fingerprinting techniques are major verification systems. RF-based techniques include studying and analyzing the characteristics of the captured transmitted RF signal from UAVs or UAVs’ controllers. However, WiFi-based fingerprinting is related to the WiFi links and traffic between the UAV and its remote controller. The reviewed studies include the analysis of RF spectrogram (fingerprinting) [7,9,10,19], angle of arrival (AOA) (MUSIC) [12], and direction of arrival (DOA) [40] methods for the identification and localization of drones using conventional as well machine learning algorithms [9,10,13,14,15].

In [7], the technique proposes a complete UAV detection and identification system framework designed to work in the 2.4 GHz frequency band. The system starts with capturing the wireless signals in the test area. Then, the captured signal is processed based on a 4-level Haar wavelet transform analysis. The standard deviation of the processed signal is calculated to define the UAV detection condition. After the detection of the UAV, the RF fingerprinting stage is activated, and three main features are extracted: (1) fractal dimension (FD), (2) square integrated bispectra (SIB), and (3) axially integrated bispectra (AIB). These features are adjusted and weighted using principal component analysis (PCA) and neighborhood component analysis (NCA) algorithms. The final RF fingerprints are stored as the training data for a set of machine learning algorithms used to classify the UAV.

Based on indoor and outdoor experimental scenarios, the average identification accuracy of UAVs is summarized with respect to three fingerprinting features. Furthermore, in [8], the WiFi network traffic is monitored, and the UAV detection method is based on WiFi fingerprint analysis. The extracted features are related to the captured traffic’s duration, behavior, and distribution. Different scenarios are applied to evaluate the system’s performance in UAV detection, where the average precision is about 96%.

The authors of [15,19,20] fed the extracted time-domain characteristics (shape factor, skewness, kurtosis, and variance) of recorded RF signals to the machine learning data processing units to detect and classify UAVs. In [9], indoor experimental testing is conducted for data collection using the RF fingerprints of the transmitted signal from the micro-UAV controller to the UAV for UAV detection and classification. Different micro-UAV controllers (a total of 14) operating at the 2.4 GHz frequency band were used to create the dataset (a total of 100 RF signals) and test the proposed detection and classification technique. Each micro-UAV controller has a different transmitted signal, categorized with its unique transmitter characteristics, excluding the traditional threshold-based detection technique. The Markov model algorithm is later used for UAV detection and energy transient signal approach for feature extraction and UAV classification. The performance and accuracy of the system were found to be 96.3%.

UAVs use the Industrial Scientific and Medical (ISM) frequency bands, i.e., 2.4 GHz and 5.8 GHz bands, to communicate with their remote controllers [11]. Multiple passive RF sensors support these frequency bands and are used for non-invasive surveillance operations, including UAV monitoring, detection, localization, and tracking. In [11], the UAV detection system consists of a sensor node, Keysight RF sensor N6841A, operating in the range of 20 MHz–6 GHz, broadband antenna, and GPS tracker linked with geolocation software, N6854A. The RF signals are detected and collected within a radius of 2 km from the sensor node. A GPS antenna also records the time stamps for these collected signals. The localization of the UAV is performed using a detection algorithm and time difference of arrival (TDOA) measurements. Extended Kalman filter (EKF) framework and fitting motion models (MM) address these errors and improve localization performance.

Furthermore, the research work in [10] illustrates a system model and architecture followed by experimental validation of the proposed direction finding (DF) method of sparse de-noising auto-encoder (SDAE) for UAV surveillance. This method consists of a single channel for a receiver and a directional phased array antenna. The mechanism of the system works as follows. First, the transmitted signal from the drone to its ground controller gets processed using an RF switching mechanism to measure the received signals output power at each directional phased array antenna. Next, the acquired output power values from the N-antennas of the phased directional array are input to the proposed SDAE-based deep neural network (DNN). The first network layer extracts received wattage values. Then the remaining network utilizes sparse representation to categorize UAVs’ signal directions. The system diagram of the proposed method in [10] is depicted in Figure 2. To summarize, the wattage power values are passed to the proposed deep network, followed by the DF method, which exploits both the sparsity parameter of the transmitted UAV signal and the gain variation parameters of the directional antenna array.

In [16], the authors discuss UAV detection using RF-transmitted signals between UAVs and their remote controllers. Power spectrum cancellation and multi-hop autocorrelation are developed to achieve RF passive detection of UAVs and controllers to detect emitted signals. The multi-hop autocorrelation method can detect the cross-correlation signal if the Signal-to-Noise (SNR) ratio is small by applying an emitted remote-control signal. A limitation of the multi-hop autocorrelation is the low accuracy in the case of fixed frequency in remote-control signals. The calculated parameters significantly depend on the autocorrelation function, leading to false positives. Hence, the study of [16] used the power spectrum cancellation technique to eliminate the effect of fixed frequency signals. Power spectrum cancellation works by first finding the differences between the control signal power spectrum and fixed frequency signals over time. Once the differences are identified, the fixed frequency signal is eliminated, and the remote control signal is applied to multi-hop auto-correlation to finalize the parameters for UAV detection.

Furthermore, [17] stated that the RF passive detection method has the advantage of low cost, license-free, long-range distance coverage, and early warning capability. They also illustrated an RF passive system architecture, which analyzes the electromagnetic RF spectrum emitted from exchanged signals between the UAV and its controller. The passive RF detection algorithms analyze these signals to sense alternations in the frequency and time domain RF spectrum.

Various studies have reported promising results utilizing different algorithms and techniques for RF-based UAV detection. However, the presence of noise affects the accuracy and detection range.

Table 2. Summary of reviewed RF-based techniques for UAV characterization.

Ref.	Operating Frequency	Functionalities			Performance
		Identification	Classification	Localization/ Tracking
[7]	2.4 GHz	√	√	-	Average of 97%
[8]	2.4 GHz	√	√	-	Greater than 96%
[9]	2.4 GHz	√	√	-	Average of 96.3%
[11]	20 MHz–6 GHz	√	-	√	-
[13]	1–6 GHz	√	√	-	Average of 99%
[10]	2.401–2.481 GHz	√	√	√	-
[16]	2.4 GHz and 5.8 GHz ISM bands	√	-	-	-
[12]	2.4 GHz	√	-	√	-
[18]	2.4 GHz ISM band	√	-	√	-

summarizes the reviewed papers and tabulates the features and accuracy of the undertaken methodology for RF-based UAV detection.

3.2. Radar

Radar signal processing is among the classical approach for aircraft and drone detection as it can be used in all weather conditions with 24/7 operation [18,21,52] as compared to acoustic and visual detection methods. In this approach, the received signal is characterized to detect echo, doppler signature, or radar cross-section (RCS) for detecting and tracking the target [21,28,29,53]. The conventional radar signal processing techniques have the limitation of accurate distinction of mini UAVs from birds due to their smaller RCSs. AI-based techniques are proposed [28,29,53,54] to process the extracted features from the radar signals to address this issue to some extent.

In radar-based detection, radio energy is used to detect the target and define its position [21,23,55]. Typically, a radar-based detection system has three main components: RF radar, data acquisition, and signal processing. In RF radar, the electromagnetic energy radiates into space and encounters the UAV’s body flying in the monitored area. The UAV’s reflected wave is returned and received by the system, measured, and processed in real-time (data acquisition and signal processing). Hence, the UAV is successfully located, and its flight path is tracked by the system [30,36,55,56].

Frequency-modulated continuous wave (FMCW) and continuous wave (CW) radars are preferred to be used in UAV detection and identification, especially for their continuous pulsing, effective cost, and performance [21]. The FMCW radar contains a transmitter and a receiver antenna. The oscillator and the control signal produce the transmitted signal. After the backscattering/reflected signal is received, it gets passed to the I/Q demodulator for filtering. Power is equally distributed into two signals with 90 degrees phase shift to be forwarded to the low pass filter (LPF). The intermediate frequency (IF) signal, resulting from in-phase and quadrature-phase components, is directed to the analog-to-digital converter (ADC) and the digital signal processing (DSP), as depicted in Figure 3. The distance and velocity of the target can be defined by using the time delay and phase information of both the transmitted and received signals [21].

The studies of [24,25,28,54,57,58] employed the principal component analysis (PCA) [24], convolutional neural networks (CNN) [23,28,51,54], long short-term memory (LSTM) [28], and support vector machines (SVM) [57,58] techniques for the processing of extracted features from radar signals such as micro-doppler spectrogram [23,28,54,57,58] and range-doppler signature [24] for the classification of drones. Recently authors in [13] used the hierarchical learning approach for the detection of the presence, type, and flight trajectory of a UAV. Due to the smaller size of most UAVs, wideband, high frequency, expensive radars are required for the accurate detection and tracking of mini UAVs [23,24,52,54], which increases the overall cost of the detection and localization system.

In [55], the authors proposed research and experiments for evaluating the data acquisition and signal processing algorithm in a CW radar system that supports C and X frequency bands operations. The radar system uses the micro-Doppler principle. The extracted signatures in the frequency and time domains are used in UAV classification for calculating the propeller blades’ length and determining the rotation propellers’ speed. For performance evaluation, the number of UAV propellers varies during the experiments while fixing the propellers’ rotational speed and a maximum distance of 25 m between the radar and UAV. The classification and measurement of UAVs become complex with the increase in propellers.

In another work [56], simulation and analysis of continuous wave radar’s echo signals are studied and presented in different conditions at an operating frequency of 35 GHz. Mainly UAV detection is based on the time-frequency characteristics of the Micro Doppler signal produced by the rotor rotation using singular value decomposition. Discrete wavelet transform is also used to remove environmental clutter from the radar echo signal, whereas the support vector machine (SVM) is used as a classifier. The detection accuracy of the developed system achieved 85%.

Another type of radar-based UAV detection mechanism, cylindrical phased array radar, was discussed in [27]. The system performs better for UAV detection when comparing the omnidirectional scanning to planner array radar due to the flexibility of changing the direction of the beam and illumination time to the target after the phased array was used. As for the operational norms, the system’s hardware structure and signal processing flow are designed to get a strong clutter suppression specified in the investigation, and the result of the experiment shows potential for UAV detection. Authors in [27] developed a cylindrical phased array radar system and explored signal optimization by specifying signal processing flow with the moving target detection (MTD) based on the maximum signal-to-clutter ratio (SCR) criterion.

Tang et al. [24] explained the type x-band, a small phased array radar based on AD9361, an RF Agile Transceiver. The AD9361 is a highly integrated RF module with a high-performance agile transceiver for 3G and 4G base station applications. The reported radar system consists of a control module controlling the antenna beam pointing through the transmitter/receiver (T/R) module. The signal processor also sends waveforms as transmitted RF signals to AD9361 within the timing sequence. Then the corresponding waveform is generated and established by AD9361. The radar simulation detects a drone with a radar cross section (RCS) of 0.01 m⁻² within the range of 5 km. For radar detection, enhanced reflected signals are necessary to minimize the effect of noise. An SNR value greater than 14 dB indicates a highly accurate detection.

In the case of a reliable RCS, the chosen wavelength should not reach half of the detected object’s dimension. It is critical to use a higher frequency while using Doppler-based detection. As illustrated in [22], radar is used to detect smaller drones; however, it has an ill-prepared standard for UAV detection based on low air-velocity aircraft and weak radar signature. During target detection, the radars would receive reflections from clutter-like objects, landscapes, and precipitation, posing a challenge in detection. A target can only be detected if system noise due to clutter is minimized. A 30 × 30 rectangular phase array used in [22] detects the presence of drones in monostatic radar. It would continuously scan the predefined surveillance region, with the limitation of a 90-degree azimuth sector, to achieve 360 azimuth coverage at a low cost. Doppler estimation discussed in [22] can be described as a spectrum estimation process.

The reference [26] illustrates the new method based on 5G millimeter waves with an end-to-end network. It further explains the detection method done using 5G millimeter-wave radar at rotors of UAVs. The high-resolution range profile (HRRP) can identify a UAV location, while micro-Doppler identifies the UAV. Moreover, the cepstrum method was used to extract any number and speed information of the detected UAV rotor. Multiple UAVs can be identified using the sinusoidal frequency modulation (SFM) parameter optimization method. The proposed method determines the following: the number of detected UAVs, the number of rotors, the rotation speed of all rotors, and the position of the UAVs. The proposed radar detection in [26] presents a UAV identification and detection study by providing a method for UAV tracking using the GPS-independent method, such as GPS signal failure, GPS signal interference, and satellite occlusion areas. HRRP technology and micro-Doppler provide a successful solution to detect and localize any rotating targets regardless of weather conditions. The presented simulated results showed high robustness and performance of the cepstrum method.

Authors in [59] presented a passive radio drone detection system that uses goodness-of-fit (GoF) based spectrum sensing and the MUSIC algorithm to detect the transmitted signal of a drone and its controller and estimate the DOA. Once a signal is detected, the DOA is estimated at the detected frequency. The MDL algorithm detects the number of targets and whether the source is a drone or controller. The detection system detected drones and controllers from different manufacturers with good sensitivity.

A challenge associated with UAV detection is the presence of aircraft and birds in the background [60,61,62]. Hence, clutter suppression and target detection algorithms are needed to overcome this complex issue, as stated in [26]. Rationally, object detection of possible UAVs comes first, followed by classification to separate UAVs from other detected objects. In addition, the purpose of these classifications and identifications can be used to extract many unique features of these UAVs [23]. As stated previously, the effects of Doppler radar are used to determine the velocity of a distant object more accurately. This is obtained from the radial component of a target velocity in relation to radar. Using stepped frequency waveform (SFW), an HRRP can be obtained. Due to HRRP and Doppler information from a wide-band Doppler radar, detected objects scanned using wide-band are identified and classified. Millimeter wave base stations and 5G network systems can be used as detection network channels for UAV detection using the data from the processing center of 5G base stations. The process includes extracting essential parameters from multipath locations through 5G bases.

Many factors must be considered during the development to enhance the radar systems’ performance, such as operating frequency, data acquisition, processing algorithms, classification techniques, and environmental clutter. The summary of reviewed studies of this technique is given in

Table 3. Summary of reviewed radar-based techniques for UAV characterization.

Ref.	Detection Technique	Specifications	Functionalities			Performance
			Identification	Classification	Localization/ Tracking
[21]	FMCW/CW radar	Doppler effect principle	√	-	-	NA
[55]	CW radar	C and X frequency bands, Micro Doppler principle	√	√	-	-
[56]	CW radar	Operating frequency: 35 GHz	√	√	-	Accuracy 85%
[27]	Cylindrical phased array radar	Operating frequency: C band	√	-	√	Performed well under a strong cluttered environment
[24]	Small phased array radar	Based on AD9361	√	-	√	Reliable and stabile
[22]	Rectangular phased array radar	Operating frequency: X band	√	-	-	Mixed up with birds
[26]	5G millimeter wave radar	Starting frequency is 25 GHz, which is in the 5G band	√	-	√	Detected at 300 m with a speed of 157.9 r/s & at 850.2 m with a speed of 88 r/s

.

3.3. Acoustic

Acoustic sensors, such as microphone arrays, capture the generated audio from the rotors and propellers of the drone and then compare the extracted features, including mel- frequency cepstral coefficients (MFCC) and short-time Fourier transform (STFT), with acoustic signature databases for the detection and classification of drones and UAVs using conventional and AI-based architectures. MFCC is a set of reflected human perception features of sounds, which is used in audio classification when paired with machine learning approaches. STFT is considered an intermediate feature compared to MFCC. MFCC compresses signals while representing them with coefficients set. On the other hand, STFT features contain more information and noise than MFCC, giving STFT an advantage. Deep learning models can easily adopt STFT and manage it given more complex data [37].

Authors proposed a machine learning framework in [38], shown in Figure 4, to detect and classify ADr sounds in a noisy environment, among other sounds. The required features are extracted from ADr sound using the feature extraction techniques of MFCC and linear predictive cepstral coefficients (LPCC). Following the feature extraction process, these sounds are then identified using SVMs. The results show that the SVM cubic kernel with MFCC outperforms the LPCC technique by detecting ADr sounds with 96.7% accuracy.

Acoustic-based technologies are effective for detecting UAVs since they are not affected by the UAV’s frequency range, weather fluctuations, e.g., fog, environmental disturbance, and noise. Hence, such technologies do not block the acoustic sensors’ earshot to detect the UAV’s acoustic signals. Acoustic signals produced by the engine and propeller blades of the UAV are collected and processed to classify the UAV and calculate its distance, direction, and location [33].

Authors in [39] proposed a CNN-based system to detect drones using acoustic signals received by a microphone. STFT magnitude is used as the two-dimensional feature in the study since drones’ harmonic properties differ from those of other devices that make a similar noise. The dataset comprised 68,931 and 41,958 frames of drone and non-drone sounds collected using DJI Phantom 3 and 4 drones flying outdoors. The proposed approach has a detection rate of 98.97% for the 100-epoch model and a false alarm rate of 1.28. Figure 5 illustrates the system overview of the proposed approach.

An acoustic-based detection system was designed and implemented in [30] to detect and locate the UAVs efficiently. The acoustic sensor array configuration comprises two tetrahedron-shaped microphones. The system uses multiple algorithms for data and features extraction from the collected acoustic signals: cepstral coefficients (CC) for extracting the harmonics’ features, SVM to classify and distinguish between the extracted features’ vectors related to UAV or background noise, and TDOA based on Bayesian framework. Signal processing is concluded with the temporal and dimensional features’ vectors calculations to acquire the accurate UAV classification and localization path. They also study the contribution of the SNR in detecting the UAVs against the detection rate.

In [36], multi-label UAV sound classification is examined using stacked bidirectional long short-term memory (BiLSTM), an advanced, recurrent neural network (RNN) capable of handling sequence or multiple classification tasks and avoiding long-term dependency issues. The proposed BiLSTM model is 94.02% successful in UAVs’ sound classification.

Several types of research and studies aim to investigate and evaluate different algorithms used in data acquisition, processing, and classification of the collected acoustic signals. In [31], the system’s performance level varies using different audio processing algorithms for characteristic feature vector extraction. The extracted features are inputted into concurrent neural network (CoNN) for classification. The results confirm better accuracy when integrating CoNN with the Wigner-Ville dictionary than MFCC and mean instantaneous frequency (MIF).

The authors in [35] gathered the acoustic data from a local suburban airport for the five samples of commercial multirotor UAVs to establish the performance based on passive acoustic detection. The study characterizes the emitted noise of UAVs of different levels in an anechoic chamber at the airborne time. The microphone array was arranged within two circular tiers, each 1-m in radius, and separated vertically by 1.6 m to collect data from the local airports. The generalized cross-correlation (GCC)-based algorithm is used to find direction by fusing the time difference of both arrivals and steered power response with phase transform (SRP-PHAT). The smallest UAV with a 294 m detection distance was tested and demonstrated. Differential Doppler is used to overcome the decorrelation effect for better accuracy, as stated in [35].

In [32], the authors used classical detection and direction-finding methods using an array of microphones. There had been a physical investigation of the UAVs through experiments on acoustic emission with two signal models presented in harmonic signal and broadband signal for open area and indoor environments, respectively. The spectral signs are used for detecting and recognizing the UAVs in a noisy environment by incorporating the effect of noises in urban transport, speech signals, and environment noises. The result gives the same quality as the MFCC method, where acoustic portraits are unnecessary. The cross-correlation function is efficient in the direction-finding of the UAV. The study of [32] concludes with the following points: (1) high-pass filters are effective in the processing stage of UAV acoustic emission; (2) taking a noisy environment as a background experiment while detecting and recognizing UAVs by spectral signs performs similarly to the MFCC method, excluding acoustic portraits; (3) it is suggested to improve the efficiency of the CCFM algorithm in acoustic signals to filter out low-frequency noise; (4) MFCC and CCFM can be used to create an effective counter-action system against UAVs.

Yang et al. [37] researched the utilization of acoustic nodes in the UAV detection system. The proposed system finds the best configuration of the node for deploying the UAV acoustic detection system using machine learning models. The study was designed to investigate the best combination of acoustic features, STFT and MFCC, machine learning algorithms, SVM and CNN, for node optimization. After integrating the sensing nodes in four different configurations among the test sets, the one that maximizes the detection range without blind spots is selected. A semi-circle by the STFT-SVM model with a 75-m distance between the protected area and node has the best performance for configuration optimization. Demonstrating machine learning in the audio signal domain with different learning algorithms was used for detection module development. The study [37] focused on event sound detection using binary classification with MFCC features in an urban area.

A drone acoustic detection system (DADS) is proposed and demonstrated experimentally to detect, classify, and track airborne objects in [33]. They used a Phantom 4 UAV for testing, which reached 350 m with four degrees as an average precision to track a maneuvering UAV with compact acoustic nodes. This test also implemented the classification algorithm to detect a multirotor UAV based on a specific sound inherent in the flight control mechanism. The Steven Institute of Technology has developed the DADS to detect, track, and classify anonymous UAVs by propeller noise. The proposed system has three or more microphone nodes in a tetrahedron configuration. The communication between the microphone nodes and the central computer is done through WiFi for processing. The orientation calibration for the DADS system is performed by emitting white noise from a speaker and tracking the GPS position for several minutes. Based on the difference between the detected direction and computed ones from the surveyed GPS, the orientation can easily be corrected in the case of detection and tracking. Establishing a tracking process can be predicted using collected data and parameters. Node placement, the direction-finding probability that depends on precision and range for a given target, and ambient conditions with the tracker association threshold are among the collected data.

Two main components affecting the system’s overall performance are (1) hardware specifications, including acoustic sensors and data acquisition tools, and (2) software tools and algorithms, including acoustic fingerprints and features extraction, classification, and localization.

Table 4. Summary of reviewed acoustic-based techniques for UAV characterization.

Ref.	Detection Technique	Functionalities			Performance
		Identification	Classification	Localization/ Tracking
[30]	Designed for Amateur Drones (200 Hz), SVM (Drone sound identification)	√	-	√	High accuracy
[36]	BiLSTM (UAV sound classification)	√	√	-	UAV sounds 94.02%
[31]	Concurrent Neural Networks	√	-	-	96.3%
[35]	TDoA, SRP-PHAT	√	-	√	SRP-PHAT outperform TDoA
[32]	-	√	-	√	-
[37]	MFCC, STFT, CNN, SVM	√	-	√	Noise affects the detection
[33]	SRP-PHAT	√	-	√	Drone classification algorithm to be improved according to distance
[34]	SRP-PHAT	√	-	√	-

summarizes the recent studies for acoustic-based UAV detection, classification, and localization.

Unlike radar and RF approaches, the acoustic solution does not require a line of sight (LOS). However, this solution has challenges of a short range, the need for an extensive large signature database, and vulnerability to ambient environmental noise and clutters, particularly in urban areas [14,30,31,40], and quiet operation of the drone [9,30,38]. The detection of the drone pilot could be very difficult, too, using acoustic sensors.

3.4. Electro-Optical

The electro-optic sensing system transmits, detects, and examines radiations in the optical spectrum, including visible light, infrared, and ultraviolet radiation. It can handle long-range imaging and has reliable results under different illumination levels. The components associated include optics, laser, detectors, camera, processing unit, etc. Such systems have been used for UAV detection, direction finding, and localization continuously and in all weather conditions.

In [46], the authors proposed using machine learning techniques to automatically detect and track small moving objects in the airfield from their motion patterns, i.e., the ways an object moves. The system utilized remote digital towers with high-resolution cameras covering the 360-degree view of airports to construct a video dataset comprising aircraft in an airfield and drones. Harris detection and convolutional neural network followed by optical flow we applied to the dataset to locate and track very small moving objects in the wide-area scene. The results showed that the system can detect objects with 15 × 15 pixels in 1080p images with a low miss rate. Motion-based features are extracted from their trajectories, after which a K-nearest neighbor classifier is applied to classify objects into drones or aircraft, with an accuracy of 93%.

The proposed approach in [42] performs the detection by integrating a 3D LADAR sensing system. The study employed voxel-based background subtraction and variable radially bounded nearest neighbor (V-RBNN) techniques to detect small UAVs up to 2 km. During the development phase, this integration is supported with augmented data set to enhance the model’s performance. The developed LADAR scanner can be rotated to cover a wide range of areas, e.g., 350 degrees for azimuth direction and 120 degrees for elevation direction. Furthermore, the used clustering algorithm, V-RBNN, has a good impact on the target UAV classification, which may increase the use of this proposed detection system in various applications.

In some electro-optic solutions, the detected data transferred to the analysis phase, including advanced processing, machine vision, or machine learning, are not accurate enough to track small UAVs effectively. Authors in [44] proposed an electro-optic system integrated with an all-sky camera system to get a wider view of the monitored area to improve the detection resolution. Multiple experiments were performed to evaluate the proposed solution and test its integration with other cues, i.e., acoustic. The combination of these three systems, electro-optic, all-sky camera, and acoustic cues, is also evaluated.

The study in [41] improved its outcomes’ reliability by using the electro-optical method for small UAV detection and tracking. The actual video stream was used in real-time, and a differential method was employed for analyzing and investigating UAV detection and tracking. The differential method finds the differences in sequenced frames in a video stream. In the case of hovering UAVs or some axially moving and revealing objects near the frames, the contrast was selected only for the displaced part of the object to process the video streams using the DIS algorithm.

The electro-optical detection method needs to consider the following factors: size and movement in 3D, speed of detected airborne objects, the maximal distance of detected objects from the camera position, optical lens descriptions, and linear object image resolution. Since these factors directly relate to image processing methods, detecting distance and outputs of detections get affected negatively if one of the aforementioned factors contains faulty or inaccurate information. Detecting moving objects at a maximal distance from a camera in real-time is the main objective of the method.

A single dynamic vision sensing (DVS) camera, a base station, UAV, and a blinking marker are used in [45] to detect and locate mobile UAVs. During video streaming, the differences among the captured frames are computed and filtered to detect UAVs in the background image using a temporal-filtering algorithm. The triangulation algorithm was also used to help capture UAVs or drones by extracting spatial localization parameters and providing details about the physical size of the detected object.

Similarly, in [43], Seidaliyeva et al. developed an algorithm to detect drones from a video stream. The system overview for the process is illustrated in Figure 6. The input frames are passed into a moving object detector algorithm. The authors relied on background subtraction followed by threshold filtering and morphological operations for detecting moving objects. The background subtraction method describes a model background image that is subtracted from all frames to extract the foreground. This method heavily depends on the background remaining static throughout the operation. A CNN-based classifier is used on the detections to distinguish drones from other objects such as birds.

The limited detection range can pose a challenge while employing the electro-optic technique. The detection performance can be enhanced by incorporating other supporting algorithms. The summary of reviewed studies of this method is depicted in

Table 5. Summary of reviewed electro-optical sensor-based techniques for UAV characterization.

Ref.	Detection Technique	Functionalities			Performance
		Identification	Classification	Localization/ Tracking
[42]	3D LADAR sensor, 3D background subtraction, V-RBNN	√	-	√	Detection Range 2 km
[44]	Combination of: EO/IR, All-sky, and acoustic cues	√	-	√	Line of sight limitation
[41]	Real stream detection, Differential method	√	-	√	-
[45]	DVS camera, Temporal filtering, Triangulation	√	-	√	Accurate Detection range 30 m
[43]	Background subtraction, CNN’s	√	-	-	Moving Background dependency

.

The performance of vision-based solutions becomes poor with no LOS (angle of camera), bad quality of lenses, in foggy, dark, and dusty environments (weather conditions), and background temperature [9,10,21,63]. The aforementioned limitations could be addressed to some extent by using an IR camera, i.e., detection based on drone component heat, but that increases the system cost significantly and limits the detection range and environment due to the sensibility of the sensors that measure the thermal difference between the drone and the background [14,63].

3.5. Hybrid Fusion Systems

The hybrid fusion of multiple cues, such as radio frequency, radar, acoustic, and visual sensors, improves the performance of detection, classification, and localization of both the drone and its controller. Jovanoska et al. [47] suggested an array of sensors to collect the detected drone’s data to be fed to a fusion engine for further analysis using the multiple hypothesis tracker (MHT) techniques, as illustrated in Figure 7. The RF signal is received and processed to compute the detected drone’s DOA for drone localization. The captured signals from the integrated acoustic sensors are filtered to remove unwanted noise. After identifying and detecting the drone signatures, the coherent broadband beamforming technique is used to recognize the drone bearing angle and reduce its error by the two-step filter. Finally, the extracted DOA and bearing angle are referred to by the fusion engine for localization purposes. Finally, the GSM passive radar [41] is used for UAV detection and localization, and its output is fed to the fusion engine of the overall system. Combining all these technologies improved the system performance and enhanced the localization accuracy.

In [40], the proposed UAV detection and localization system relies on time delay and beamforming of the collected acoustic signal from a set of microphones. Acoustic signal characteristics with such signal processing are used to find the DOA of the UAV’s detected recorded signal. Furthermore, Kalman filtering is used to improve the UAV’s trajectory. The system is designed to identify and track the RF signal emitted by portable RF devices [48]. The system consists of two parts: (1) RF signal acquisitions achieved by an antenna array followed by a Nyquist ADC converter and (2) signal processing. The RF signal from the first part is passed into FFT to measure the DOA. The DOA is passed into the digital bandpass filter to measure the TDOA, which is used together with the DOA to estimate the location. AOA calculated from the DOA, the location, and past tracking information are used for tracking the drone’s position [40].

3.6. Comparison of Detection Technologies

Earlier sections have discussed different techniques for detecting, identifying, and localizing UAVs. Each technique’s performance varies according to equipment complexity and cost, coverage range and distance, operation efficiency, accuracy and precision measurements, etc.

Table 6. Summary of all reviewed UAV detection techniques.

Detection Technique	Summary	Limitations	Ref.
Radio Frequency	Real-time analysis for the detected radio communication between UAV and its controller. However, it does not apply to autonomous UAV detection. Low cost and simple architecture and elements: Antennas, Processors, RF sensors. Power and sensitivity of each affect detection system performance and accuracy. Common frequency bands are around 2.4 and 5 GHz Covering a long detection range will perform more efficiently in the less congested RF zones. Referring to RF datasets and integrating with machine learning algorithms are advanced ways to enhance detection, localization, and precise classification.	The RF-based detection technique applies only if the UAV is remotely controlled.	[7,8,9,10,11,12,13,14,15,16,17,18,19,20]
Radar	Transmitting radio signals, then receiving and analyzing the reflection/backscattering/echo radar signals. UAV’s detection, tracking (Doppler-based), classification, and localization are based on the analysis of the reflected radio signal. Active sensor (Radar) and data processing modules with high-range detection and accurate localization. Machine learning algorithms and techniques’ integration for better performance and results. Less noise and applicable in different weather conditions (fog, dust, rain, etc.). UAVs with small radar cross-sections are difficult to be identified and classified.	UAVs generally have limited Radar Cross Sections similar to birds or pedestrians. The amount of false positives remains high and low-RCS limits the detection range of the radar, especially X-band Radars.	[21,22,23,24,25,26,27,28,29]
Acoustic	Analyze acoustic signals coming from UAV’s engine or propeller blades. Acoustic sensors/microphones arrays combined with data acquisition and signal processing modules Acoustic fingerprint analysis, features extraction, classification, and localization UAV’s identification and distinction from other objects Effective in a short distance, however, it’s affected by the nearby noise sources and weather. Acoustic dataset and Machine learning techniques integration for higher performance (detection and classification).	The detection of acoustic noise emitted by UAVs is low; thus, the acoustic technique requires a network of sensors deployed around sensitive places.	[30,31,32,33,34,35,36,37,38,39,40]
Electro-optic	Imaging and motion line of sight detection. High-cost equipment Ability to track autonomous UAVs. Controlling false alarms with advanced integration with other methods/algorithms/machine learning. Detection performance can vary with different environmental conditions and weather.	Using different electro-optics is required, and the fusion of video streams is required to cope with UAVs’ environment and type/size. This increases the cost of the solution.	[41,42,43,44,45,46]

summarizes the techniques cited in this study with their main features and affected factors. Combining the different techniques and integrating different sensors can increase the accuracy and reliability of the UAV detection systems, reduce the possibility of errors, and improve the system’s ability to adapt.

4. Drone Controller Detection and Localization

Once UAVs are detected, the detection and localization of the drone controller are implemented to monitor their communication and limit illegal use.

UAV detection systems differ according to the technologies and functions performed, such as identification, classification, tracking, localization, interdiction, destruction, and damage. Technology and functions are selected and implemented based on the main requirements of the UAV detection system. In this section, some detection systems that support localization functionality are reviewed.

The process of locating and positioning a UAV is mainly based on collecting the direct measurements of the detected UAV and its emitted signals. These direction measurements and other extracted features are calculated and utilized in the UAV detection system to estimate the geolocation of the UAV. The computed geolocation parameters for the direction-finding methods include angle of arrival (AOA) [40], time of arrival (TOA) [64], direction of arrival (DOA) [18,47], frequency difference of arrival (FDOA) [51], time difference of arrival (TDOA) [11,35], and received signal strength (RSS) [65].

The proposed system in [12] utilizes a low-cost passive RF-based UAV detection and localization method. The system computes AOA for the RF-based signal to determine whether the transmitted signals’ peaks correspond to the UAV or its controller. Then, it uses the triangulation technique to estimate the location of RF signal peak sources. The free-space path loss model and triangulation combination is reported in [50] to detect and localize a stationary drone controller. The proposed system contains two direction-finding systems for direction identification and localization for the drone and its controller. Each direction-finding system has an omnidirectional antenna for detecting drone signal occurrence and a mechanically agile directional antenna for directions identification and localization of RF signal peaks for UAV and/or its controller signals. The whole system is depicted in Figure 8.

The study [12] discussed that distinguishing between RF signals from the UAV and its controller from other RF signals in the surrounding area poses a challenge in drone controller localization. The reported direction-finding station consists of two modules: drone signal analysis to classify drone and remote controller (RC) signals and the direction-finding module. Each direction-finding station extracts acute parameters from detected RF signals and uses a mechanical steering antenna for identification and localization. The precision of the direction-finding function is dependent and affected by the antenna’s directivity and gain, drone velocity, scanning velocity, and beam width of the used directional antenna.

The reported detection system in [18] employs frequency hopping spread spectrum (FHSS) to detect and locate UAVs and RCs. The cyclostationarity analysis algorithm is used to identify the FHSS-type drone RC signals and differentiate them from other background signals operating in the same frequency band. After the successful classification of the drone RC signals, STFT and additional re-sampling processing are applied to enhance the detection accuracy of the reconstructed RC signal. Finally, the direction-finding phase is achieved by implementing the subspace algorithms to identify the AOA of the FHSS drone RC signal. The proposed system [49] utilizes a set of a uniform linear array of quasi-Yagi antennas in the experimental setup to enhance the precision of the direction-finding function.

5. Conclusions

This study has reviewed the most recent techniques for UAV/drone and its controller detection, classification, and localization. Cost-effectiveness, precision, accuracy, reliability, and real-time processing are among the factors considered while developing UAV detection systems. After discussing the high-level architecture of UAVs and security concerns, a comprehensive review of radio frequency, radar, acoustic, electro-optic, and hybrid systems for UAV detection is presented. The UAV detection systems employ different algorithms and techniques depending on the applications for detecting, classifying, locating, tracking, and alerting. To address the challenges, meet market needs, and improve reliability, employing a hybrid fusion of multiple cues, such as radio frequency, radar, acoustic, and visual sensors, can enhance detection performance.