Research

13 pages, 2085 KiB

Open AccessArticle

Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks

by Oladayo S. Ajani, Sung-ho Hur and Rammohan Mallipeddi

Mathematics 2023, 11(23), 4744; https://doi.org/10.3390/math11234744 - 23 Nov 2023

Viewed by 840

Domain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has been studied in [...] Read more.

Domain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has been studied in terms of varying the operational characters of the associated systems or physical dynamics rather than their environmental characteristics. This is counter-intuitive as it is unrealistic to alter the mechanical dynamics of a system in operation. Furthermore, most works were based on cherry-picked environments within different classes of RL tasks. Therefore, in this work, we investigated domain randomization by varying only the properties or parameters of the environment rather than varying the mechanical dynamics of the featured systems. Furthermore, the analysis conducted was based on all six RL locomotion tasks. In terms of training the RL agents, we employed two proven RL algorithms (SAC and TD3) and evaluated the generalization capabilities of the resulting agents on several train–test scenarios that involve both in-distribution and out-distribution evaluations as well as scenarios applicable in the real world. The results demonstrate that, although domain randomization favors generalization, some tasks only require randomization from low-dimensional distributions while others require randomization from high-dimensional randomization. Hence the question of what level of randomization is optimal for any given task becomes very important. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

24 pages, 1558 KiB

Open AccessArticle

Mixture Basis Function Approximation and Neural Network Embedding Control for Nonlinear Uncertain Systems with Disturbances

by Le Ma, Qiaoyu Zhang, Tianmiao Wang, Xiaofeng Wu, Jie Liu and Wenjuan Jiang

Mathematics 2023, 11(13), 2823; https://doi.org/10.3390/math11132823 - 23 Jun 2023

Viewed by 769

Abstract

A neural network embedding learning control scheme is proposed in this paper, which addresses the performance optimization problem of a class of nonlinear system with unknown dynamics and disturbance by combining with a novel nonlinear function approximator and an improved disturbance observer (DOB). [...] Read more.

A neural network embedding learning control scheme is proposed in this paper, which addresses the performance optimization problem of a class of nonlinear system with unknown dynamics and disturbance by combining with a novel nonlinear function approximator and an improved disturbance observer (DOB). We investigated a mixture basic function (MBF) to approximate the unknown nonlinear dynamics of the system, which allows an approximation in a global scope, replacing the traditional radial basis function (RBF) neural networks technique that only works locally and could be invalid beyond some scope. The classical disturbance observer is improved, and some constraint conditions thus are no longer needed. A neural network embedding learning control scheme is exploited. An arbitrary type of neural network can be embedded into a base controller, and the new controller is capable of optimizing the control performance by tuning the parameters of neural network and satisfying the Lyapunov stability simultaneously. Simulation results verify the effectiveness and advantage of our proposed methods. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

13 pages, 11783 KiB

Open AccessArticle

Fixed-Time Controller for Altitude/Yaw Control of Mini-Drones: Real-Time Implementation with Uncertainties

by Moussa Labbadi, Chakib Chatri, Sahbi Boubaker and Souad Kamel

Mathematics 2023, 11(12), 2703; https://doi.org/10.3390/math11122703 - 14 Jun 2023

Cited by 1 | Viewed by 1043

Abstract

Gradually, it has become easier to use aerial transportation systems in practical applications. However, due to the fixed-length wire, recent studies on load-suspended transportation systems have revealed some practical constraints, especially when using quadrotor unmanned aerial vehicles (UAVs). By actively adjusting the distance [...] Read more.

Gradually, it has become easier to use aerial transportation systems in practical applications. However, due to the fixed-length wire, recent studies on load-suspended transportation systems have revealed some practical constraints, especially when using quadrotor unmanned aerial vehicles (UAVs). By actively adjusting the distance between the quadrotor and the payload, it becomes possible to carry out a variety of challenging tasks, including traversing confined spaces, collecting samples from offshore locations, and even landing a payload on a movable platform. Thus, mass variable aerial transportation systems should be equipped with trajectory tracking control mechanisms to accomplish these tasks. Due to the above-mentioned reasons, the present paper addresses the problem of the altitude/yaw tracking control of a mini-quadrotor subject to mass uncertainties. The main objective of this paper is to design a fixed-time stable controller for the perturbed altitude/yaw motions, based on recent results using the fixed-time stability approach. For comparison reasons, other quadrotor motion controllers such as dual proportional integral derivative (PID) loops were considered. To show its effectiveness, the proposed fixed-time controller was validated on a real mini-quadrotor under different scenarios and has shown good performance in terms of stability and trajectory tracking. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

17 pages, 1222 KiB

Open AccessArticle

Adaptive Finite/Fixed Time Control Design for a Class of Nonholonomic Systems with Disturbances

by Moussa Labbadi, Sahbi Boubaker, Souad Kamel and Faisal S. Alsubaei

Mathematics 2023, 11(10), 2287; https://doi.org/10.3390/math11102287 - 14 May 2023

Viewed by 835

Abstract

This paper addresses the fixed-time stability analysis of a mobile unicycle-like system (UTMS) with chained shape dynamics (CFD) and subjected to unknown matched uncertainties. To achieve fixed-time stabilization of a nonholonomic (NS) system in CFD, an adaptive nonsingular fast terminal sliding mode control [...] Read more.

This paper addresses the fixed-time stability analysis of a mobile unicycle-like system (UTMS) with chained shape dynamics (CFD) and subjected to unknown matched uncertainties. To achieve fixed-time stabilization of a nonholonomic (NS) system in CFD, an adaptive nonsingular fast terminal sliding mode control scheme (ANFTSMC) is proposed. To determine the upper bounds of the disturbances, only velocity and position measurements are required. In addition, the control rule uses the Lyapunov theory, which guarantees the stability of the closed-loop system. To emphasize/evaluate the efficacy of the proposed method, simulations are performed in different disturbance situations. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

21 pages, 1596 KiB

Open AccessArticle

Improvement of an Adaptive Robot Control by Particle Swarm Optimization-Based Model Identification

by Hazem Issa and József K. Tar

Mathematics 2022, 10(19), 3609; https://doi.org/10.3390/math10193609 - 02 Oct 2022

Cited by 4 | Viewed by 1086

Abstract

Model-based controllers suffer from the effects of modeling imprecisions. The analytical form of the available model often contains only approximate parameters and can be physically incomplete. The consequences of these effects can be compensated by adaptive techniques and by the improvement of the [...] Read more.

Model-based controllers suffer from the effects of modeling imprecisions. The analytical form of the available model often contains only approximate parameters and can be physically incomplete. The consequences of these effects can be compensated by adaptive techniques and by the improvement of the available model. Lyapunov function-based classic methods, which assume exact analytical model forms, guarantee asymptotic stability by cautious and slow parameter tuning. Fixed point iteration-based adaptive controllers can work without the exact model form but immediately yield precise trajectory tracking. They neither identify nor improve the parameters of the available model. However, any amendment of the model can improve the controller’s operation by affecting its range and speed of convergence. It is shown that even very primitive, fast, and simple versions of evolutionary computation-based methods can produce considerable improvement in their operation. Particle swarm optimization (PSO) is an attractive, efficient, and simple tool for model improvement. In this paper, a PSO-based model approximation technique was investigated for use in the control of a three degrees of freedom PUMA-type robot arm via numerical simulations. A fixed point iteration (FPI)-based adaptive controller was used for tracking a nominal trajectory while the PSO attempted to refine the model. It was found that the refined model still had few errors, the effects of which could not be completely neglected in the model-based control. The best practical solution seems to be the application of the same adaptive control with the use of the more precise, PSO-improved model. Apart from a preliminary study, the first attempt to combine PSO with FPI is presented here. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

17 pages, 5956 KiB

Open AccessArticle

New Result for the Analysis of Katugampola Fractional-Order Systems—Application to Identification Problems

by Omar Kahouli, Assaad Jmal, Omar Naifar, Abdelhameed M. Nagy and Abdellatif Ben Makhlouf

Mathematics 2022, 10(11), 1814; https://doi.org/10.3390/math10111814 - 25 May 2022

Cited by 4 | Viewed by 1574

Abstract

In the last few years, a new class of fractional-order (FO) systems, known as Katugampola FO systems, has been introduced. This class is noteworthy to investigate, as it presents a generalization of the well-known Caputo fractional-order systems. In this paper, a novel lemma [...] Read more.

In the last few years, a new class of fractional-order (FO) systems, known as Katugampola FO systems, has been introduced. This class is noteworthy to investigate, as it presents a generalization of the well-known Caputo fractional-order systems. In this paper, a novel lemma for the analysis of a function with a bounded Katugampola fractional integral is presented and proven. The Caputo–Katugampola fractional derivative concept, which involves two parameters 0 < α < 1 and ρ > 0, was used. Then, using the demonstrated barbalat-like lemma, two identification problems, namely, the “Fractional Error Model 1” and the “Fractional Error Model 1 with parameter constraints”, were studied and solved. Numerical simulations were carried out to validate our theoretical results. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

23 pages, 7095 KiB

Open AccessFeature PaperArticle

Adaptive Intelligent Sliding Mode Control of a Dynamic System with a Long Short-Term Memory Structure

by Lunhaojie Liu, Wen Fu, Xingao Bian and Juntao Fei

Mathematics 2022, 10(7), 1197; https://doi.org/10.3390/math10071197 - 06 Apr 2022

Viewed by 1565

Abstract

In this work, a novel fuzzy neural network (NFNN) with a long short-term memory (LSTM) structure was derived and an adaptive sliding mode controller, using NFNN (ASMC-NFNN), was developed for a class of nonlinear systems. Aimed at the unknown uncertainties in nonlinear systems, [...] Read more.

In this work, a novel fuzzy neural network (NFNN) with a long short-term memory (LSTM) structure was derived and an adaptive sliding mode controller, using NFNN (ASMC-NFNN), was developed for a class of nonlinear systems. Aimed at the unknown uncertainties in nonlinear systems, an NFNN was designed to estimate unknown uncertainties, which combined the advantages of fuzzy systems and neural networks, and also introduced a special LSTM recursive structure. The special three gating units in the LSTM structure enabled it to have selective forgetting and memory mechanisms, which could make full use of historical information, and have a stronger ability to learn and estimate unknown uncertainties than general recurrent neural networks. The Lyapunov stability rule guaranteed the parameter convergence of the neural network and system stability. Finally, research into a simulation of an active power filter system showed that the proposed new algorithm had better static and dynamic properties and robustness compared with a sliding controller that uses a recurrent fuzzy neural network (RFNN). Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

18 pages, 4281 KiB

Open AccessEditor’s ChoiceArticle

Adaptive Fuzzy Neural Network Harmonic Control with a Super-Twisting Sliding Mode Approach

by Qi Pan, Xiangguo Li and Juntao Fei

Mathematics 2022, 10(7), 1063; https://doi.org/10.3390/math10071063 - 25 Mar 2022

Cited by 4 | Viewed by 1905

Abstract

This paper designed an adaptive super-twisting sliding mode control (STSMC) scheme based on an output feedback fuzzy neural network (OFFNN) for an active power filter (APF), aiming at tracking compensation current quickly and precisely, and solving the harmonic current problem in the electrical [...] Read more.

This paper designed an adaptive super-twisting sliding mode control (STSMC) scheme based on an output feedback fuzzy neural network (OFFNN) for an active power filter (APF), aiming at tracking compensation current quickly and precisely, and solving the harmonic current problem in the electrical grid. With the use of OFFNN approximator, the proposed controller has the characteristic of full regulation and high approximation accuracy, where the parameters of OFFNN can be adjusted to the optimal values adaptively, thereby increasing the versatility of the control method. Moreover, due to an added signal feedback loop, the controller can obtain more information to track the state variable faster and more correctly. Simulations studies are given to demonstrate the performance of the proposed controller in the harmonic suppression, and verify its better steady-state and dynamic performance. Full article

(This article belongs to the Special Issue Advances in Intelligent Control)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Advances in Intelligent Control

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI