Submit to Special Issue Submit Abstract to Special Issue Review for Sensors Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Intelligent Sensors".

Deadline for manuscript submissions: 30 November 2024 | Viewed by 12878

Share This Special Issue

Special Issue Editors

Dr. Boyang Li

E-Mail Website
Guest Editor

Department of Aeronautical and Aviation Engineering,The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
Interests: unmanned aerial vehicle; flight dynamics and control; aerial robotics; SLAM
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Chihyung Wen

E-Mail Website
Guest Editor

Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong
Interests: UAV/MAV technology research and development; aerothermodynamics; experimental fluid mechanics and CFD

Special Issue Information

Dear Colleagues,

Combining autonomous unmanned aerial vehicles (UAVs) and AI-based object detection and tracking could significantly improve efficiency, reduce cost, and lower risks for various applications. With fast developments in UAV platform design, cameras, micro-computers, and image-processing algorithms, autonomous UAVs have become a promising sensing platform for various applications such as environment monitoring and infrastructure inspection. These systems can reduce the necessity of traditional manual inspection in risky working environments and avoid the cost of using piloted fixed-wing aircraft or helicopters to conduct large-scale sensing tasks.

New aerial-based sensors with machine learning, object detection, and tracking capabilities provide both opportunities and challenges that allow the research community to provide novel solutions. The key aim of this Special Issue is to bring together innovative research that uses off-the-shelf or custom-made platforms to extend autonomous aerial sensing capabilities. Contributions from all fields related to UAVs and aerial-image processing techniques are of interest, particularly including, but not limited to, the following topics:

Unmanned aerial vehicle (UAV) system;
Machine learning;
AI-based data processing;
Object detection;
Object tracking;
Localization and mapping;
Path planning;
Obstacle avoidance;
Multi-agent collaboration.

Dr. Boyang Li
Prof. Dr. Chihyung Wen
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (7 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

17 pages, 10284 KiB

Open AccessArticle

Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage

by Joachim Rüter, Theresa Maienschein, Sebastian Schirmer, Simon Schopferer and Christoph Torens

Sensors 2024, 24(4), 1144; https://doi.org/10.3390/s24041144 - 09 Feb 2024

Viewed by 601

Abstract

A key necessity for the safe and autonomous flight of Unmanned Aircraft Systems (UAS) is their reliable perception of the environment, for example, to assess the safety of a landing site. For visual perception, Machine Learning (ML) provides state-of-the-art results in terms of performance, but the path to aviation certification has yet to be determined as current regulation and standard documents are not applicable to ML-based components due to their data-defined properties. However, the European Union Aviation Safety Agency (EASA) published the first usable guidance documents that take ML-specific challenges, such as data management and learning assurance, into account. In this paper, an important concept in this context is addressed, namely the Operational Design Domain (ODD) that defines the limitations under which a given ML-based system is designed to operate and function correctly. We investigated whether synthetic data can be used to complement a real-world training dataset which does not cover the whole ODD of an ML-based system component for visual object detection. The use-case in focus is the detection of humans on the ground to assess the safety of landing sites. Synthetic data are generated using the methods proposed in the EASA documents, namely augmentations, stitching and simulation environments. These data are used to augment a real-world dataset to increase ODD coverage during the training of Faster R-CNN object detection models. Our results give insights into the generation techniques and usefulness of synthetic data in the context of increasing ODD coverage. They indicate that the different types of synthetic images vary in their suitability but that augmentations seem to be particularly promising when there is not enough real-world data to cover the whole ODD. By doing so, our results contribute towards the adoption of ML technology in aviation and the reduction of data requirements for ML perception systems. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

34 pages, 1871 KiB

Open AccessArticle

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

by Raúl Arranz, David Carramiñana, Gonzalo de Miguel, Juan A. Besada and Ana M. Bernardos

Sensors 2023, 23(21), 8766; https://doi.org/10.3390/s23218766 - 27 Oct 2023

Viewed by 2202

Abstract

This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

16 pages, 15333 KiB

Open AccessArticle

YOLOv5 Drone Detection Using Multimodal Data Registered by the Vicon System

by Wojciech Lindenheim-Locher, Adam Świtoński, Tomasz Krzeszowski, Grzegorz Paleta, Piotr Hasiec, Henryk Josiński, Marcin Paszkuta, Konrad Wojciechowski and Jakub Rosner

Sensors 2023, 23(14), 6396; https://doi.org/10.3390/s23146396 - 14 Jul 2023

Cited by 5 | Viewed by 1597

Abstract

This work is focused on the preliminary stage of the 3D drone tracking challenge, namely the precise detection of drones on images obtained from a synchronized multi-camera system. The YOLOv5 deep network with different input resolutions is trained and tested on the basis of real, multimodal data containing synchronized video sequences and precise motion capture data as a ground truth reference. The bounding boxes are determined based on the 3D position and orientation of an asymmetric cross attached to the top of the tracked object with known translation to the object’s center. The arms of the cross are identified by the markers registered by motion capture acquisition. Besides the classical mean average precision (mAP), a measure more adequate in the evaluation of detection performance in 3D tracking is proposed, namely the average distance between the centroids of matched references and detected drones, including false positive and false negative ratios. Moreover, the videos generated in the AirSim simulation platform were taken into account in both the training and testing stages. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

22 pages, 27783 KiB

Open AccessArticle

Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine

by Zhongze Qiu, Hangfu Guo, Jun Hu, Hejun Jiang and Chaopeng Luo

Sensors 2023, 23(12), 5693; https://doi.org/10.3390/s23125693 - 18 Jun 2023

Cited by 2 | Viewed by 1976

Abstract

Compared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruise platform, we establish a multispectral dataset of scatterable mines, with mine-spreading areas of the ground vegetation considered. In order to achieve the robust detection of occluded landmines, first, we employ an active learning strategy to refine the labeling of the multispectral dataset. Then, we propose an image fusion architecture driven by detection, in which we use YOLOv5 for the detection part, to improve the detection performance instructively while enhancing the quality of the fused image. Specifically, a simple and lightweight fusion network is designed to sufficiently aggregate texture details and semantic information of the source images and obtain a higher fusion speed. Moreover, we leverage detection loss as well as a joint-training algorithm to allow the semantic information to dynamically flow back into the fusion network. Extensive qualitative and quantitative experiments demonstrate that the detection-driven fusion (DDF) that we propose can effectively increase the recall rate, especially for occluded landmines, and verify the feasibility of multispectral data through reasonable processing. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

15 pages, 968 KiB

Open AccessArticle

Dynamic Weighting Network for Person Re-Identification

by Guang Li, Peng Liu, Xiaofan Cao and Chunguang Liu

Sensors 2023, 23(12), 5579; https://doi.org/10.3390/s23125579 - 14 Jun 2023

Cited by 1 | Viewed by 969

Abstract

Recently, hybrid Convolution-Transformer architectures have become popular due to their ability to capture both local and global image features and the advantage of lower computational cost over pure Transformer models. However, directly embedding a Transformer can result in the loss of convolution-based features, particularly fine-grained features. Therefore, using these architectures as the backbone of a re-identification task is not an effective approach. To address this challenge, we propose a feature fusion gate unit that dynamically adjusts the ratio of local and global features. The feature fusion gate unit fuses the convolution and self-attentive branches of the network with dynamic parameters based on the input information. This unit can be integrated into different layers or multiple residual blocks, which will have varying effects on the accuracy of the model. Using feature fusion gate units, we propose a simple and portable model called the dynamic weighting network or DWNet, which supports two backbones, ResNet and OSNet, called DWNet-R and DWNet-O, respectively. DWNet significantly improves re-identification performance over the original baseline, while maintaining reasonable computational consumption and number of parameters. Finally, our DWNet-R achieves an mAP of 87.53%, 79.18%, 50.03%, on the Market1501, DukeMTMC-reID, and MSMT17 datasets. Our DWNet-O achieves an mAP of 86.83%, 78.68%, 55.66%, on the Market1501, DukeMTMC-reID, and MSMT17 datasets. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

14 pages, 3355 KiB

Open AccessArticle

Research of Maritime Object Detection Method in Foggy Environment Based on Improved Model SRC-YOLO

by Yihong Zhang, Hang Ge, Qin Lin, Ming Zhang and Qiantao Sun

Sensors 2022, 22(20), 7786; https://doi.org/10.3390/s22207786 - 13 Oct 2022

Cited by 4 | Viewed by 1708

Abstract

An improved maritime object detection algorithm, SRC-YOLO, based on the YOLOv4-tiny, is proposed in the foggy environment to address the issues of false detection, missed detection, and low detection accuracy in complicated situations. To confirm the model’s validity, an ocean dataset containing various concentrations of haze, target angles, and sizes was produced for the research. Firstly, the Single Scale Retinex (SSR) algorithm was applied to preprocess the dataset to reduce the interference of the complex scenes on the ocean. Secondly, in order to increase the model’s receptive field, we employed a modified Receptive Field Block (RFB) module in place of the standard convolution in the Neck part of the model. Finally, the Convolutional Block Attention Module (CBAM), which integrates channel and spatial information, was introduced to raise detection performance by expanding the network model’s attention to the context information in the feature map and the object location points. The experimental results demonstrate that the improved SRC-YOLO model effectively detects marine targets in foggy scenes by increasing the mean Average Precision (mAP) of detection results from 79.56% to 86.15%. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Figure 1

12 pages, 2840 KiB

Open AccessArticle

Towards Efficient Detection for Small Objects via Attention-Guided Detection Network and Data Augmentation

by Xiaobin Wang, Dekang Zhu and Ye Yan

Sensors 2022, 22(19), 7663; https://doi.org/10.3390/s22197663 - 09 Oct 2022

Cited by 11 | Viewed by 2701

Abstract

Small object detection has always been a difficult direction in the field of object detection, especially the detection of small objects in UAV aerial images. The images captured by UAVs have the characteristics of small objects and dense objects. In order to solve these two problems, this paper improves the performance of object detection from the aspects of data and network structure. In terms of data, the data augmentation strategy and image pyramid mechanism are mainly used. The data augmentation strategy adopts the method of image division, which can greatly increase the number of small objects, making it easier for the algorithm to be fully trained during the training process. Since the object is denser, the image pyramid mechanism is used. During the training process, the divided images are up-sampled into three different sizes, and then sent to three different detectors respectively. Finally, the detection results of the three detectors are fused to obtain the final detection results. The small object itself has few pixels and few features. In order to improve the detection performance, it is necessary to use context. This paper adds attention mechanism to the yolov5 network structure, while adding a detection head to the underlying feature map to make the network structure pay more attention to small objects. By using data augmentation and improved network structure, the detection performance of small objects can be significantly improved. The experiment in this paper is carried out on the Visdrone2019 dataset and DOTA dataset. Through experimental verification, our proposed method can significantly improve the performance of small object detection. Full article

(This article belongs to the Special Issue AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions)

► Show Figures

Journal Menu

Journal Browser

AI-Based Object Detection and Tracking in UAVs: Challenges and Research Directions

Share This Special Issue

Special Issue Editors

Special Issue Information

Published Papers (7 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI