Research

17 pages, 5734 KiB

Open AccessArticle

RU-SLAM: A Robust Deep-Learning Visual Simultaneous Localization and Mapping (SLAM) System for Weakly Textured Underwater Environments

by Zhuo Wang, Qin Cheng and Xiaokai Mu

Sensors 2024, 24(6), 1937; https://doi.org/10.3390/s24061937 - 18 Mar 2024

Viewed by 520

Abstract

Accurate and robust simultaneous localization and mapping (SLAM) systems are crucial for autonomous underwater vehicles (AUVs) to perform missions in unknown environments. However, directly applying deep learning-based SLAM methods to underwater environments poses challenges due to weak textures, image degradation, and the inability [...] Read more.

Accurate and robust simultaneous localization and mapping (SLAM) systems are crucial for autonomous underwater vehicles (AUVs) to perform missions in unknown environments. However, directly applying deep learning-based SLAM methods to underwater environments poses challenges due to weak textures, image degradation, and the inability to accurately annotate keypoints. In this paper, a robust deep-learning visual SLAM system is proposed. First, a feature generator named UWNet is designed to address weak texture and image degradation problems and extract more accurate keypoint features and their descriptors. Further, the idea of knowledge distillation is introduced based on an improved underwater imaging physical model to train the network in a self-supervised manner. Finally, UWNet is integrated into the ORB-SLAM3 to replace the traditional feature extractor. The extracted local and global features are respectively utilized in the feature tracking and closed-loop detection modules. Experimental results on public datasets and self-collected pool datasets verify that the proposed system maintains high accuracy and robustness in complex scenarios. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

15 pages, 5469 KiB

Open AccessArticle

Underwater Rescue Target Detection Based on Acoustic Images

by Sufeng Hu and Tao Liu

Sensors 2024, 24(6), 1780; https://doi.org/10.3390/s24061780 - 10 Mar 2024

Viewed by 570

Abstract

In order to effectively respond to floods and water emergencies that result in the drowning of missing persons, timely and effective search and rescue is a very critical step in underwater rescue. Due to the complex underwater environment and low visibility, unmanned underwater [...] Read more.

In order to effectively respond to floods and water emergencies that result in the drowning of missing persons, timely and effective search and rescue is a very critical step in underwater rescue. Due to the complex underwater environment and low visibility, unmanned underwater vehicles (UUVs) with sonar are more efficient than traditional manual search and rescue methods to conduct active searches using deep learning algorithms. In this paper, we constructed a sound-based rescue target dataset that encompasses both the source and target domains using deep transfer learning techniques. For the underwater acoustic rescue target detection of small targets, which lack image feature accuracy, this paper proposes a two-branch convolution module and improves the YOLOv5s algorithm model to design an acoustic rescue small target detection algorithm model. For an underwater rescue target dataset based on acoustic images with a small sample acoustic dataset, a direct fine-tuning using optical image pre-training lacks cross-domain adaptability due to the different statistical properties of optical and acoustic images. This paper therefore proposes a heterogeneous information hierarchical migration learning method. For the false detection of acoustic rescue targets in a complex underwater background, the network layer is frozen during the hierarchical migration of heterogeneous information to improve the detection accuracy. In addition, in order to be more applicable to the embedded devices carried by underwater UAVs, an underwater acoustic rescue target detection algorithm based on ShuffleNetv2 is proposed to improve the two-branch convolutional module and the backbone network of YOLOv5s algorithm, and to create a lightweight model based on hierarchical migration of heterogeneous information. Through extensive comparative experiments conducted on various acoustic images, we have thoroughly validated the feasibility and effectiveness of our method. Our approach has demonstrated state-of-the-art performance in underwater search and rescue target detection tasks. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

14 pages, 2133 KiB

Open AccessArticle

Trajectory Planning of Autonomous Underwater Vehicles Based on Gauss Pseudospectral Method

by Wenyang Gan, Lixia Su and Zhenzhong Chu

Sensors 2023, 23(4), 2350; https://doi.org/10.3390/s23042350 - 20 Feb 2023

Cited by 1 | Viewed by 1554

Abstract

This paper aims to address the obstacle avoidance problem of autonomous underwater vehicles (AUVs) in complex environments by proposing a trajectory planning method based on the Gauss pseudospectral method (GPM). According to the kinematics and dynamics constraints, and the obstacle avoidance requirement in [...] Read more.

This paper aims to address the obstacle avoidance problem of autonomous underwater vehicles (AUVs) in complex environments by proposing a trajectory planning method based on the Gauss pseudospectral method (GPM). According to the kinematics and dynamics constraints, and the obstacle avoidance requirement in AUV navigation, a multi-constraint trajectory planning model is established. The model takes energy consumption and sailing time as optimization objectives. The optimal control problem is transformed into a nonlinear programming problem by the GPM. The trajectory satisfying the optimization objective can be obtained by solving the problem with a sequential quadratic programming (SQP) algorithm. For the optimization of calculation parameters, the cubic spline interpolation method is proposed to generate initial value. Finally, through comparison with the linear fitting method, the rapidity of the solution of the cubic spline interpolation method is verified. The simulation results show that the cubic spline interpolation method improves the operation performance by 49.35% compared with the linear fitting method, which verifies the effectiveness of the cubic spline interpolation method in solving the optimal control problem. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

13 pages, 1130 KiB

Open AccessArticle

An Underwater Human–Robot Interaction Using a Visual–Textual Model for Autonomous Underwater Vehicles

by Yongji Zhang, Yu Jiang, Hong Qi, Minghao Zhao, Yuehang Wang, Kai Wang and Fenglin Wei

Sensors 2023, 23(1), 197; https://doi.org/10.3390/s23010197 - 24 Dec 2022

Cited by 2 | Viewed by 1779

Abstract

The marine environment presents a unique set of challenges for human–robot interaction. Communicating with gestures is a common way for interacting between the diver and autonomous underwater vehicles (AUVs). However, underwater gesture recognition is a challenging visual task for AUVs due to light [...] Read more.

The marine environment presents a unique set of challenges for human–robot interaction. Communicating with gestures is a common way for interacting between the diver and autonomous underwater vehicles (AUVs). However, underwater gesture recognition is a challenging visual task for AUVs due to light refraction and wavelength color attenuation issues. Current gesture recognition methods classify the whole image directly or locate the hand position first and then classify the hand features. Among these purely visual approaches, textual information is largely ignored. This paper proposes a visual–textual model for underwater hand gesture recognition (VT-UHGR). The VT-UHGR model encodes the underwater diver’s image as visual features, the category text as textual features, and generates visual–textual features through multimodal interactions. We guide AUVs to use image–text matching for learning and inference. The proposed method achieves better performance than most existing purely visual methods on the dataset CADDY, demonstrating the effectiveness of using textual patterns for underwater gesture recognition. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

21 pages, 3888 KiB

Open AccessArticle

A Learning-Based Vehicle-Cloud Collaboration Approach for Joint Estimation of State-of-Energy and State-of-Health

by Peng Mei, Hamid Reza Karimi, Fei Chen, Shichun Yang, Cong Huang and Song Qiu

Sensors 2022, 22(23), 9474; https://doi.org/10.3390/s22239474 - 04 Dec 2022

Cited by 6 | Viewed by 1472

Abstract

The state-of-energy (SOE) and state-of-health (SOH) are two crucial quotas in the battery management systems, whose accurate estimation is facing challenges by electric vehicles’ (EVs) complexity and changeable external environment. Although the machine learning algorithm can significantly improve the accuracy of battery estimation, [...] Read more.

The state-of-energy (SOE) and state-of-health (SOH) are two crucial quotas in the battery management systems, whose accurate estimation is facing challenges by electric vehicles’ (EVs) complexity and changeable external environment. Although the machine learning algorithm can significantly improve the accuracy of battery estimation, it cannot be performed on the vehicle control unit as it requires a large amount of data and computing power. This paper proposes a joint SOE and SOH prediction algorithm, which combines long short-term memory (LSTM), Bi-directional LSTM (Bi-LSTM), and convolutional neural networks (CNNs) for EVs based on vehicle-cloud collaboration. Firstly, the indicator of battery performance degradation is extracted for SOH prediction according to the historical data; the Bayesian optimization approach is applied to the SOH prediction combined with Bi-LSTM. Then, the CNN-LSTM is implemented to provide direct and nonlinear mapping models for SOE. These direct mapping models avoid parameter identification and updating, which are applicable in cases with complex operating conditions. Finally, the SOH correction in SOE estimation achieves the joint estimation with different time scales. With the validation of the National Aeronautics and Space Administration battery data set, as well as the established battery platform, the error of the proposed method is kept within 3%. The proposed vehicle-cloud approach performs high-precision joint estimation of battery SOE and SOH. It can not only use the battery historical data of the cloud platform to predict the SOH but also correct the SOE according to the predicted value of the SOH. The feasibility of vehicle-cloud collaboration is promising in future battery management systems. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

12 pages, 2762 KiB

Open AccessArticle

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

by Guangcheng Wang, Fenglin Wei, Yu Jiang, Minghao Zhao, Kai Wang and Hong Qi

Sensors 2022, 22(21), 8562; https://doi.org/10.3390/s22218562 - 07 Nov 2022

Cited by 15 | Viewed by 2269

Abstract

Target search for moving and invisible objects has always been considered a challenge, as the floating objects drift with the flows. This study focuses on target search by multiple autonomous underwater vehicles (AUV) and investigates a multi-agent target search method (MATSMI) for moving [...] Read more.

Target search for moving and invisible objects has always been considered a challenge, as the floating objects drift with the flows. This study focuses on target search by multiple autonomous underwater vehicles (AUV) and investigates a multi-agent target search method (MATSMI) for moving and invisible objects. In the MATSMI algorithm, based on the multi-agent deep deterministic policy gradient (MADDPG) method, we add spatial and temporal information to the reinforcement learning state and set up specialized rewards in conjunction with a maritime target search scenario. Additionally, we construct a simulation environment to simulate a multi-AUV search for the floating object. The simulation results show that the MATSMI method has about 20% higher search success rate and about 70 steps shorter search time than the traditional search method. In addition, the MATSMI method converges faster than the MADDPG method. This paper provides a novel and effective method for solving the maritime target search problem. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

14 pages, 3132 KiB

Open AccessArticle

Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning

by Jiayi Wen, Shaoman Liu and Yejin Lin

Sensors 2022, 22(18), 6942; https://doi.org/10.3390/s22186942 - 14 Sep 2022

Cited by 3 | Viewed by 1864

Abstract

The unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic [...] Read more.

The unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic environment to perform multiple tasks simultaneously. Thus, a multi-USV cooperative approach can be adopted to obtain the desired success rate in the presence of multi-mission objectives. In this paper, we propose a cooperative navigating approach by enabling multiple USVs to automatically avoid dynamic obstacles and allocate target areas. To be specific, we propose a multi-agent deep reinforcement learning (MADRL) approach, i.e., a multi-agent deep deterministic policy gradient (MADDPG), to maximize the autonomy level by jointly optimizing the trajectory of USVs, as well as obstacle avoidance and coordination, which is a complex optimization problem usually solved separately. In contrast to other works, we combined dynamic navigation and area assignment to design a task management system based on the MADDPG learning framework. Finally, the experiments were carried out on the Gym platform to verify the effectiveness of the proposed method. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

15 pages, 483 KiB

Open AccessArticle

RBF Neural Network Sliding Mode Control for Passification of Nonlinear Time-Varying Delay Systems with Application to Offshore Cranes

by Baoping Jiang, Dongyu Liu, Hamid Reza Karimi and Bo Li

Sensors 2022, 22(14), 5253; https://doi.org/10.3390/s22145253 - 13 Jul 2022

Cited by 6 | Viewed by 1472

Abstract

This paper is devoted to studying the passivity-based sliding mode control for nonlinear systems and its application to dock cranes through an adaptive neural network approach, where the system suffers from time-varying delay, external disturbance and unknown nonlinearity. First, relying on the generalized [...] Read more.

This paper is devoted to studying the passivity-based sliding mode control for nonlinear systems and its application to dock cranes through an adaptive neural network approach, where the system suffers from time-varying delay, external disturbance and unknown nonlinearity. First, relying on the generalized Lagrange formula, the mathematical model for the crane system is established. Second, by virtue of an integral-type sliding surface function and the equivalent control theory, a sliding mode dynamic system can be obtained with a satisfactory dynamic property. Third, based on the RBF neural network approach, an adaptive control law is designed to ensure the finite-time existence of sliding motion in the face of unknown nonlinearity. Fourth, feasible easy-checking linear matrix inequality conditions are developed to analyze passification performance of the resulting sliding motion. Finally, a simulation study is provided to confirm the validity of the proposed method. Full article

(This article belongs to the Special Issue Sensors, Modeling and Control for Intelligent Marine Robots)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Sensors, Modeling and Control for Intelligent Marine Robots

Share This Special Issue

Special Issue Editors

Special Issue Information

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI