Trajectories Generation for Unmanned Aerial Vehicles Based on Obstacle Avoidance Located by a Visual Sensing System

Abstract

In this work, vectorial trajectories for unmanned aerial vehicles are completed based on a new algorithm named trajectory generation based on object avoidance (TGBOA), which is presented using a UAV camera as a visual sensor to define collision-free trajectories in scenarios with randomly distributed objects. The location information of the objects is collected by the visual sensor and processed in real-time. This proposal has two advantages. First, this system improves efficiency by focusing the algorithm on object detection and drone position, thus reducing computational complexity. Second, online trajectory references are generated and updated in real-time. To define a collision-free trajectory and avoid a collision between the UAV and the detected object, a reference is generated and shown by the vector, symmetrical, and parametric equations. Such vectors are used as a reference in a PI-like controller based on the Newton–Euler mathematical model. Experimentally, the TGBOA algorithm is corroborated by developing three experiments where the F-450 quadcopter, MATLAB^® 2022ª, PI-like controller, and Wi-Fi communication are applied. The TGBOA algorithm and the PI-like controller show functionality because the controller always follows the vector generated due to the obstacle avoidance.

1. Introduction

The study of unmanned aerial vehicles (UAVs) has grown in recent decades due to their features and the large field of applications that have been found in agronomy, surveillance, and tracking, among others [1,2]. Researchers’ work on new methods for creating autonomous navigation on systems are focused on three stages, mainly the following [3]: obtaining information about the environment using different sensors and methodologies; [4,5] processing data to obtain a reference trajectory using a localization, mapping, or optimization algorithm [6,7]; and implementing flight controllers to keep the drone altitude and track the signal reference [8,9,10]. The first depends on a great variety of sensors such as cameras, light detection and ranging (LiDAR), or inertial measurement units (IMU). The second stage uses algorithms for object detection, and collision avoidance, as well as route optimization to find the best trajectory. The third stage consists of developing the flight controller, which is focused on approaching the reference to the position and posture desired for the UAV by means of a mathematical model and some control technique.

One way to obtain the reference is based on generating a vector, whose origin point is the position where the UAV starts its route, and the endpoint is the target to be reached, as shown in reference [11]: here, a vector is generated to move the UAV among separate trees by means of a photometric technique. However, various factors influence the drone path, such as changes in the environment and collisions; therefore, it becomes necessary to implement methods for obstacle detection and avoidance while the UAV trajectory is updated [5]. For this case, two methods are considered: manual and automatic. In the first, an operator sends the signal from a remote control to enable a flight mode [12,13]. In the second, a trajectory generator algorithm and a flight controller are combined [14,15]. The generator can be executed from the UAV or with a remote workstation.

To solve the trajectory generation problem, two main methods are considered: offline and online [16]. In the offline method, an algorithm analyzes and calculates a trajectory using a pre-load scenario that consists of a static image or model of the place with constrains already known by the system before the UAV take-off and calculates a trajectory for the controller. The environment must not change, since if it does (if some object appears), the technique loses its functionality due to the existence of a high possibility of a collision between the UAV and the object. On the other hand, the online method uses an algorithm based on a dynamic analysis of its sensors, detecting the object’s movement to generate a customized trajectory [17,18]. Both methods are applicable; however, each method has its advantages and disadvantages. For example, the offline method is effective if the scenario is static because the UAV trajectory is calculated before its movement, but for a changing scenario, this method should not be considered because the collision between the UAV and an object is very possible. If the scenario is changing because of an object’s movement or the UAV movement, the trajectory creation online is recommended [19] because the UAV trajectory is updated based on the scenario analysis. The algorithm’s effectiveness, however, depends on the complexity of the data and the execution time. The literature has proposed different methods and devices to counter these drawbacks. Some works report waypoints with the GPS, as in [20], where a strategic route planning for a multi-UAV routing problem is shown, using a multi-step plan which includes automated definition for GNSS based on a georeferenced three-dimensional domain model, derivation of obstacle-free candidate routes, assignment of waypoints, and definition of time-stamped trajectories for all UAVs. Their aim is to ensure that the designed trajectories are reliable with the given positioning accuracy using propagated covariance as a metric. On the other hand, the authors of [21] improve the computationally fast trajectory generation algorithm using linear interpolation, which gives a fast calculation response, numerical simulation results are shown, and a nonlinear flight controller is also designed to track the generated trajectories. In addition, different works use artificial intelligence and optimization methods to obtain the best path in predetermined scenarios, as in [22], where a vision-based controller for a quadcopter is presented, to enable tracking of a moving target by a neural network which obtains image features.

For online trajectory cases, the most complex situation occurs when both the UAV and the objects are moving at the same time. Here, the algorithm must develop dynamic trajectories at a high speed according to the data analysis. To solve the problems mentioned above, several authors have proposed different methodologies. For example, in [23], the authors use a convolutional neural network to recognize images, path planning, and its implementation on an FPGA. In reference [24], the authors use the so-called simultaneous localization and mapping (SLAM) algorithm that allows the map building or updating of an unknown environment while tracking the drone location. A route planner is developed by updating itself as it explores various areas of the surrounding environment. Such methods can be functional in unknown scenarios but not dynamic. The processing tasks are hard to process due to the constant updating of the map. In [19], the authors propose a direct visual serving system for UAVs, using an onboard high-speed monocular camera with an integrated GPU. This technology is characterized by environment recognition through artificial vision and by sending signals to the controller, which reduces latency. However, the equipment used for this technique is expensive and requires prior knowledge or estimation of the objects in its spatial environment. In [25], a vision-guided autonomous flight system for quadrotor navigation is presented. Using a real-time obstacle avoidance technique based on optical flow, the frontal obstacles can be detected and guide the quadrotor with a proper direction. Nevertheless, optical flow is difficult to achieve in the real-time performance on the low-cost hardware. On the other hand, implementing the generated trajectories is carried out by flight controllers that allow the drone to move into space by manipulating the speed of its rotors. The flight control system is the fundamental aspect of the quadrotor, becoming an important platform for UAV research and development, as in [25], where different control techniques are reported. Furthermore, Newton–Euler, Euler–Lagrange, or quaternions [26,27,28,29,30,31] are examples of mathematical models applied in controllers whose aim is to follow the reference by reducing the tracking error by adjusting the UAV posture and its rotational and translational movement.

Finally,

Table 1. Comparation between different UAV trajectories’ generation methods.

	Relevant Aspect	Disadvantage
[19]	High image sampling rates to improve the environment recognition. The position control signals are transmitted to the flight controller directly.	Expensive hardware and requires prior knowledge or estimation of the objects in its spatial environment.
[23]	Convolutional neural networks used to possible path recognition. ARM and FPGA implementation.	Large amount of computational resources used for CNN. Three vision sensors needed.
[24]	Monocular-inertial SLAM in the loop of navigation in a previously unknown environment.	Updating environment requires a lot of time and computer resources. Dynamic scenarios are not considered.
[25]	Path planning and real-time obstacle avoidance techniques. Frontal static obstacles can be detected and guide the quadrotor with a proper direction.	The presence of moving obstacles is not considered. Optical flow is not sufficient to achieve the real-time performance on the low-cost hardware.

shows a comparation between some of the related work mentioned above.

In this article, a solution with low computational resources is presented for obstacle detection and avoidance, trajectory generation, and autonomous drone navigation using only a camera and without specialized high-performance hardware. It focuses on implementing a novel algorithm called trajectory generation based on object avoidance (TGBOA), which works in a real-time analysis through video frames taken from the camera placed on the UAV and communication with a workstation. Furthermore, a PI-like controller is developed and simulated, using as a reference the vector generated by the TGBOA algorithm. The Newton–Euler mathematical model is considered by the controller to represent the equations of the rigid body of the drone.

2. Materials and Methods

2.1. Materials

Figure 1 shows the proposed system for trajectory generation based on a video analysis and the evasion of localized objects through a vision system. The required material consists of a model OV5647 camera (Raspberry PI REES52, Taiwan) that works as a sensor with a Raspberry Pi 4B (Raspberry Pi Foundation, Raspberry Pi Foundation Maurice Wilkes Building St. John’s Innovation Park Cambridge, UK) responsible for transmitting the video, a model F-450 quadcopter (INVENTO, India) (shown in Figure 2), a flight controller card (Pixhawk 3 (Bombay Electronics, Mumbai, India)) installed, and a laptop as a workstation. The camera technical features are M12 × 0.5 lenses, a 5MPixels sensor, a resolution of 640 × 480 pixels, 90 frames/s, and a size of 2.5 cm × 2.5 cm × 2.6 cm. The Raspberry features are a 1.5 GHz quad-core 64-bit ARM Cortex processor, 2 GB of LPDDR4 SDRAM, and Dual-band 802.11 ac wireless networking. The drone has four brushless direct current motors (BLDC) and a 3DR-PIXHAWK controller card. The features of the laptop are 8Gb RAM and Intel i7 CPU. Finally, the software used is MATLAB^® 2022a executing on the Windows 11 Operative System.

The drone camera observes the front part where the UAV moves and generates a video during the flight. The video is transmitted in real-time to the workstation via Wi-Fi using the Raspberry Pi, and the transmission rate depends on the bandwidth Wi-Fi signal [32] and the adaptor of the workstation. In this case, the connection is made by wireless transmission at 2.4 GHz with a usable speed of 50–70 Mbps. Subsequently, the workstation analyzes the video frames considering the following objectives: object detection on the stage, collision-free zone calculation, and generation of free-collision trajectories for the flight controller.

2.2. Object Detection and Assignation of Collision-Free Zones

The generated video is 90 frames/second with 640 × 480 pixels for each frame and is sent from the UAV to the workstation. For the analysis, a frame is taken from the video, obtaining a digital RGB image as shown in Figure 3a. The image is represented through the discrete function $f\left(m,n\right)$ where $m=1,2,\dots ,M \mathrm{and} n=1,2,\dots ,N$ are the coordinates of the image pixels. Subsequently, the RGB image is transformed into an 8-bit gray-scale image and converted by the Otsu algorithm into a binary image [33]. In the binary image, the detected objects are white, and the background is marked in black, as illustrated in Figure 3b. Finally, the size, position, and centroid are obtained for each object.

The next step is calculating the collision-free regions, so the image $f\left(m,n\right)$ is divided into thirteen regions: $R_k\left(k=1,2,\dots ,13\right)$. The first region $R_1$ has been assigned concerning the UAV size and is in the image center, and the image resolution limits the remaining thirteen zones, as observed in Figure 3c. If objects in the scenario are in one or more regions, then a correlation $“A”$ between the detected objects and the region exists, and the following condition is true:

$$R_k\to \{\begin{array}{cc}{R}_{k,\alpha }& iff A=0\\ R_{k,\beta }& iff A\ne 0\end{array},$$

(1)

where $R_{k,\alpha }$ indicates the $\alpha -th$ collision-free region and $R_{k,\beta }$ indicates the $\beta -th$ region with possible collision. Now, based on Equation (1), a region $R_{k,\alpha }$ is collision-free if, and only if, the correlation area $A$ is equal to 0. On the other hand, region $R_{k,\beta }$ has a possible collision if, and only if, the correlation area $A$ is different from zero. To illustrate the above, Figure 2b is considered, where the objects are in the regions $R_4$, $R_5$, $R_{9–13}$ (marked as red), and consequently, collision-free regions are $R_{1–3}$ y $R_{6–8}$ (marked in blue).

2.3. Trajectory Generation

Once the collision-free regions have been defined, it is possible to generate flight trajectories for the UAV controller to avoid a collision and improve the success of missions.

Figure 4 shows the geometry for generating the trajectories of the controller. Based on the figure mentioned above, the origin is the drone position whose coordinates are $P_c\left(0,0,0\right)$, and the endpoint of the trajectory is the center of the collision-free region. In this case, the coordinates will be $C_k\left(x_k, y_k,z_k\right):$ $x_k$ is the distance from the UAV to the scenario, $y_k$ represents the length, and $z_k$ is the altitude. To generate the trajectory, the vector equation is used:

$${\overrightarrow{\mathit{f}}}_{\mathit{k}}={\overrightarrow{\mathit{P}}}_{\mathit{c}}+t_i{\overrightarrow{\mathit{v}}}_{\mathit{k}} i \in \mathbb{Z},$$

(2)

where ${\overrightarrow{\mathit{P}}}_{\mathit{c}}$ is a vector such that ${\overrightarrow{\mathit{v}}}_{\mathit{k}}=\overrightarrow{{\mathit{P}}_{\mathit{c}}{\mathit{C}}_{\mathit{k}}}$ vector calculated from the UAV to the center of the k-th region $R_k$, ${\overrightarrow{\mathit{f}}}_{\mathit{k}}$ is the collision-free trajectory, and $t$ are instants of time. From here, we go from a 2D problem to 3D as illustrated in Figure 4.

Taking into consideration the geometry of the problem, Equation (2) can be rewritten as follows:

$${\overrightarrow{\mathit{f}}}_{\mathit{k}}=\left(0, 0, 0\right)+t\left(x_k,y_k,z_{k }\right),$$

(3)

From Equation (3), the following parametric equations are obtained,

$$\begin{array}{c}x=x_{k}t\\ y=y_{k}t\\ z=z_{k}t\end{array},$$

(4)

Finally, the vector equation is represented in the form of symmetric equations,

$$\frac{x}{x_k}=\frac{y}{y_k}=\frac{z}{z_k},$$

(5)

Equations (4)–(6) can be applied as a reference in flight controllers for the UAV. Figure 4 shows the graphical representation for Equations (3)–(5) (possible trajectory vectors), ${\overrightarrow{\mathit{f}}}_{\mathit{k}} \left(k=1,2,3,6–8\right)$ where free regions are $R_k \left(1,2,3,6–8\right)$ and the objects are detected in regions $R_k \left(4,5,9–13\right).$ Notice that vectors ${\overrightarrow{\mathit{f}}}_{\mathit{k}} \left(k=1,2,3,6–8\right)$ are free trajectories and the drone does not have a collision due to the object evasion.

2.4. Controller Design

In this work, a controller is designed to verify the functionality of the trajectories generated in Section 2.3. To achieve this, the Newton–Euler mathematical model is used, which is presented below.

2.4.1. Mathematical Model

The UAV considered in this work is based on a quadcopter shown in Figure 5, which consists of a rigid body with four rotors. The movement of the drone is generated from the interaction of the forces resulting from each motor, which are represented as $F_i=b{\omega }_{p.i}^2$, where b is the thrust coefficient calculated by the physical and dynamic characteristics of the propellers while ${\omega }_{pi},i=1,2,3,4$ are the angular velocities of each motor. Propellers 1 and 3 rotate clockwise, and propellers 2 and 4 rotate counterclockwise.

For the mathematical model, two frameworks are considered: the one that remains fixed with respect to the Earth that we will call $E^E\left(O_E,{ϵ}_{E1},{ϵ}_{E2},{ϵ}_{E3}\right)$ and the frame of reference of the drone located at its center of mass $E^D\left(O_D,{ϵ}_{D1},{ϵ}_{D2},{ϵ}_{D3}\right)$. The absolute position of the UAV in terms of $E^E$ is expressed by $\rho ={\left(x,y,z\right)}^T$ and its attitude is described by Euler angles $\mathsf{\Theta }={\left(x,y,z\right)}^T$: roll $\left(\varphi \right)$ rotates on the x-axis, pitch ($\theta$) works around the y-axis, and the z-turn is called yaw ($\psi$). All angles exist in the range $\left(-\pi /2 ,\pi /2\right).$ This allows the drone to have six degrees of freedom, according to the framework $E^D$, three to translation $v_D={\left(v_x,v_y,v_z\right)}^T$ and three to rotation $\mathsf{\Omega }={\left({\omega }_1,{\omega }_2,{\omega }_3\right)}^T$, which are linear and angular velocities, respectively.

To continue with the Newton–Euler model, the dynamics of the rigid body are obtained by the forces applied to the center of mass of the quadcopter, and the equation of motion for the quadcopter can be described by means of Figure 4 and the translation dynamics, expressed in the following system of equations.

$$\begin{gathered} \dot{\rho }=v_D \\ {\dot{v}}_D=\frac{1}{m}{R}_{DE}{F}_{prop}-\frac{1}{m}{F}_{grav} \\ \dot{\mathsf{\Theta }}={\dot{R}}_{DE}{S}_{\omega } \\ J\dot{\mathsf{\Omega }}=-S_{\omega }J\mathsf{\Omega }+{\tau }_{prop}-{\tau }_{gyro} \end{gathered}$$

(6)

where $m$ is the quadcopter mass, $J=diag\left(I_x,I_y,I_z\right)$ is the inertia matrix of the quadcopter expressed in $E^E$, and $F_{prop}$ and ${\tau }_{prop}$ are forces and moments produced by the propellers. Moreover, $F_{grav}$ are forces and moments applied to the body of the helicopter, consisting of its own weight, and ${\tau }_{gyro}$ is defined as the gyroscopic effects resulting from helix rotations. The forces and torques due to the external disturbances are here assumed to be negligible.

In Equation (6), $S_{\omega }$ is the antisymmetric matrix defined as:

$$S_{\omega }=\left(\begin{array}{ccc}0& -{\omega }_3& {\omega }_2\\ {\omega }_3& 0& -{\omega }_1\\ -{\omega }_2& {\omega }_1& 0\end{array}\right),$$

(7)

Equation (7) in known as the dyadic representation of $\mathsf{\Omega }$.

Matrix $R_{DE}$ describes the spatial orientation of the drone framework $E^D$ to the Earth framework $E^E$. The sequence $R_{z,\psi }{R}_{y,\theta }{R}_{x,\varphi }$ is used [34], and it is obtained by applying the product of three rotation matrices for each axis, so that

$$R_{x,\varphi }=\left(\begin{array}{ccc}1& 0& 0\\ 0& c_{\varphi }& s_{\varphi }\\ 0& -s_{\varphi }& c_{\varphi }\end{array}\right); R_{y,\theta }=\left(\begin{array}{ccc}{c}_{\theta }& 0& -s_{\theta }\\ 0& 1& 0\\ s_{\theta }& 0& c_{\theta }\end{array}\right);R_{z,\psi }=\left(\begin{array}{ccc}{c}_{\psi }& s_{\psi }& 0\\ -s_{\psi }& c_{\psi }& 0\\ 0& 0& 1\end{array}\right)$$

$$R_{DE}=R_{z,\psi }{R}_{y,\theta }{R}_{x,\varphi }=\left(\begin{array}{ccc}{c}_{\theta }{c}_{\psi }& s_{\varphi }{s}_{\theta }{c}_{\psi }-c_{\varphi }{s}_{\psi }& c_{\varphi }{s}_{\theta }{c}_{\psi }+s_{\psi }{s}_{\varphi }\\ c_{\theta }{s}_{\psi }& c_{\varphi }{c}_{\psi }+s_{\varphi }{s}_{\theta }{s}_{\psi }& c_{\varphi }{s}_{\theta }{s}_{\psi }-c_{\psi }{s}_{\varphi } \\ -s_{\theta }& s_{\varphi }{c}_{\theta }& c_{\varphi }{c}_{\theta }\end{array}\right),$$

(8)

with $c_{\gamma }=\cos \left(\gamma \right)$ and $s_{\gamma }=\sin \left(\gamma \right)$. $\gamma =\varphi ,\theta ,\psi$.

The dynamics of angular velocities are expressed in matrix form:

$${\dot{R}}_{DE}=\left(\begin{array}{ccc}1& s_{\varphi }t{g}_{\theta }& c_{\varphi }t{g}_{\theta }\\ 0& c_{\varphi }& -s_{\varphi }\\ 0& s_{\varphi }s{c}_{\theta }& c_{\varphi }s{c}_{\theta }\end{array}\right),$$

(9)

with $t{g}_{\gamma }=\tan \left(\gamma \right)$ and $s{c}_{\gamma }=\sec \left(\gamma \right)$. Moreover, using small angles, Equation (9) reduces to an identity matrix $I_{3x3}$. This assumption is justified by the fact that the movements of the quadcopter are slow and soft [35].

Furthermore, the input forces and moments are the torque applied to the roll angle ${\tau }_1$, produced by the forces of the motors $F_4$ and $F_2$, respectively; the torque applied to pitch ${\tau }_3$ is produced by the forces $F_1$ and $F_3$, respectively; due to Newton’s third law, the propellers produce a yawing torque ${\tau }_2$ on the body of the quadrotor, in the opposite direction of the propeller rotation. The helices’ drag produces a torque in the angle of rotation in the body of the quadcopter and is represented by $c$. The distance between the quadcopter’s center of mass and rotor shaft is represented by $l$. All these variables are defined as:

$$\begin{gathered} F_{prop}=\left(\begin{array}{c}0\\ 0\\ \mathsf{\Sigma }{F}_i\end{array}\right) \\ {F}_{grav}=\left( \begin{array}{c}0\\ 0\\ mg\end{array}\right) \\ {\tau }_{prop}=\left(\begin{array}{c}{\tau }_1\\ {\tau }_3\\ {\tau }_2\end{array}\right)=\left(\begin{array}{c}l\left(F_4-F_2\right)\\ l\left(F_1-F_3\right)\\ c\left(-F_1+F_2-F_3+F_4\right)\end{array}\right)=\left(\begin{array}{c}l{u}_3\\ l{u}_2\\ u_4\end{array}\right). \end{gathered}$$

(10)

Next, to the dynamic model described in (6), it is possible to include the gyroscopic term ${\tau }_{gyro}$, caused by the combination of the rotations of the airframe and the four rotors, i.e., due to the fact that the pair of rotors 1–3 rotate in the opposite direction of the pair 2–4 [36]:

$${\tau }_{gyro}={{\displaystyle \sum }}_{i=1}^4{\left(-1\right)}^i{I}_R{\omega }_{R}S{ϵ}_3,$$

(11)

where ${ϵ}_3={\left(0 0 1\right)}^T$ is the vector along the axis $z$ in $E^E$, $J_R$ is the moment of inertia of the propeller with respect to its axis of rotation, and ${\omega }_R={\omega }_{R,1}-{\omega }_{R,2}+{\omega }_{R,3}-{\omega }_{R,4}$ is the so-called relative rotor speed.

Finally, using Equations (6) and (10) and under the small angle assumption, the quadcopter model is

$$\begin{array}{c}\ddot{x}=\frac{1}{m}\left(c_{\varphi }{s}_{\theta }{c}_{\psi }+s_{\varphi }{s}_{\psi } \right)u_1\hfill \\ \ddot{y}=\frac{1}{m}\left(c_{\varphi }{s}_{\theta }{s}_{\varphi }-s_{\varphi }{c}_{\psi } \right)u_1\hfill \\ \ddot{z}=-g+\frac{1}{m}{c}_{\varphi }{c}_{\theta }{u}_1\hfill \\ \ddot{\varphi }=\left(\frac{J_y-J_z}{J_x}\right)\dot{\theta }\dot{\psi }-\frac{J_R}{J_x}\dot{\theta }{\omega }_R+\frac{l}{J_x}{u}_3\hfill \\ \ddot{\theta }=\left(\frac{J_z-J_x}{J_y}\right)\dot{\varphi }\dot{\psi }+\frac{J_R}{J_y}\dot{\varphi }{\omega }_R+\frac{l}{J_y}{u}_2\hfill \\ \ddot{\psi }=\left(\frac{J_x-J_y}{J_z}\right)\dot{\varphi }\dot{\theta }+\frac{1}{J_z}{u}_4\hfill \end{array}$$

(12)

where the control variable is $u_i, i=1, 2, 3, 4$ are defined as in (10), and the parameter values used in (12) are shown in

Table 2. Coefficients and variables of the quadcopter.

Parameter	Variable	Value
Mass	$m$	1.3 kg
Frame length	l	0.233 m
Inertia	$J_x$ $J_y$ $J_z$	8.825 × 10−3 kg m2 8.825 × 10−3 kg m2 14.39 × 10−3 kg m2
Propeller inertia	$J_R$	104 × 10−6 kg m 2
Gravity	g	9.81 m/s2
Thrust	b	52.5 × 10−6
Drag	c	1.15 × 10−6 N s2 rad−2

.

Subsequently, the control problem consists of ensuring the convergence of the set of variables $X=\left(x,y,z,\psi \right)$ to references $X_{ref}=\left(x_{ref},y_{ref},z_{ref},{\psi }_{ref}\right)$. To achieve this, control subsystems composed of the rotational and translational dynamics of the system are considered, which relate the control inputs $u_i$ with spatial coordinates $\left(x,y,z\right)$ by means of Euler angles $\left(\varphi , \theta , \psi \right).$ The following coordinate change is considered: $\varphi ={\varphi }_1, {\dot{\varphi }}_1={\varphi }_2;$ $\theta ={\theta }_1, {\dot{\theta }}_1={\theta }_2;$ $\psi ={\psi }_1, {\dot{\psi }}_1={\psi }_2$ and the system of Equation (12) became:

$$\begin{array}{c}{\dot{x}}_1=x_2\hfill \\ x_2=\frac{1}{m}\left(c_{{\varphi }_1}{s}_{{\theta }_1}{c}_{{\psi }_1}+s_{{\varphi }_1}{s}_{{\psi }_1} \right)u_1\hfill \\ {\dot{y}}_1=y_2\hfill \\ y_2=\frac{1}{m}\left(c_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}-s_{{\varphi }_1}{c}_{{\psi }_1} \right)u_1\hfill \\ {\dot{z}}_1=z_2\hfill \\ \ddot{z}=-g+\frac{1}{m}{c}_{{\varphi }_1}{c}_{{\theta }_1}{u}_1\hfill \\ \dot{{\varphi }_1}={\varphi }_2\hfill \\ {\dot{\varphi }}_2=\left(\frac{J_y-J_z}{J_x}\right){\theta }_2{\psi }_2-\frac{J_R}{J_x}{\theta }_2{\omega }_R+\frac{l}{J_x}{u}_3\hfill \\ {\dot{\theta }}_1={\theta }_2\hfill \\ {\dot{\theta }}_2=\left(\frac{J_z-J_x}{J_y}\right){\varphi }_2{\psi }_2+\frac{J_R}{J_y}{\varphi }_2{\omega }_R+\frac{l}{J_y}{u}_2\hfill \\ {\dot{\psi }}_1={\psi }_2\hfill \\ {\dot{\psi }}_2=\left(\frac{J_x-J_y}{J_z}\right){\varphi }_2{\theta }_2+\frac{1}{J_z}{u}_4 \hfill \end{array}$$

(13)

Therefore, the following four subsystems shall be considered: the altitude subsystem $S_1$, the latitudinal subsystem $S_2$, the longitudinal subsystem $S_3$, and the yaw $S_4$, and then the set of Equation (13) becomes:

$$\begin{array}{c}{S}_1=\{\begin{array}{c}{\dot{z}}_1=z_2\hfill \\ {\dot{z}}_2=-g+\frac{1}{m}{c}_{{\varphi }_1}{c}_{{\theta }_1}{u}_1\hfill \end{array}\hfill \\ S_2=\{\begin{array}{c}{\dot{x}}_1=x_2\hfill \\ {\dot{x}}_2=\frac{1}{m}(c_{{\varphi }_1}{s}_{{\theta }_1}{c}_{{\psi }_1}+s_{{\varphi }_1}{s}_{{\psi }_1})u_1\hfill \\ {\dot{\theta }}_1={\theta }_2\hfill \\ {\dot{\theta }}_2=\left(\frac{J_z-J_x}{J_y}\right){\varphi }_2{\psi }_2+\frac{J_R}{J_y}{\varphi }_2{\omega }_R+\frac{l}{J_y}{u}_3\hfill \end{array}\hfill \\ S_3=\{\begin{array}{c}{\dot{y}}_1=y_2\hfill \\ {\dot{y}}_2=\frac{1}{m}(c_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}-s_{{\varphi }_1}{c}_{{\psi }_1})u_1\hfill \\ {\dot{\varphi }}_1={\varphi }_2\hfill \\ {\dot{\varphi }}_2=\left(\frac{J_y-J_z}{J_x}\right){\theta }_2{\psi }_2-\frac{J_R}{J_x}{\theta }_2{\omega }_R+\frac{l}{J_x}{u}_2\hfill \end{array}\hfill \\ S_4=\{\begin{array}{c}{\dot{\psi }}_1={\psi }_2\hfill \\ {\dot{\psi }}_2=\left(\frac{J_x-J_y}{J_z}\right){\varphi }_2{\theta }_2+\frac{1}{J_z}{u}_4\hfill \end{array}\hfill \end{array}$$

(14)

Following this, each of the subsystems is described.

2.4.2. Altitude Control

As mentioned, in order to verify the applicability of the references for the UAV flight controller, an altitude control is proposed in this section. The altitude control subsystem $S_1$ is defined by the following:

$$\begin{array}{c}{\dot{z}}_1=z_2\hfill \\ {\dot{z}}_2=-g+\frac{1}{m}{c}_{{\varphi }_1}{c}_{{\theta }_1}{u}_1 \hfill \end{array}$$

(15)

Since the aim is to keep the UAV at a constant altitude according to the desired reference $z_{ref}$, with $z_{2,ref}={\dot{z}}_{1,ref}$, then the tracking error and its dynamics are set for altitude control via the following:

$$\begin{gathered} e_{z_1}=z_1-z_{1,ref} \\ {\dot{e}}_{z_1}=z_2-{\dot{z}}_{1,ref} \end{gathered}$$

(16)

For virtual control gains $k$ are used such that

$$-k_{z_1}{e}_{z_1}-k_{0z_1}I{e}_{z_1}=z_{2,d}-{\dot{z}}_{1,ref}$$

where an integrative term is added in $e_{z_1}$, in such a way that, $\dot{I}{e}_{z_1}=e_{z_1}$, and therefore,

$$z_{2,d}={\dot{z}}_{1,ref}-k_z{}_1{e}_{z_1}-k_{0z_1}I{e}_{z_1},$$

(17)

The derivative of (17) with respect to time will be:

$$\begin{array}{c}{\dot{z}}_{2,d}={\ddot{z}}_{1,ref}-k_z{}_1{\dot{e}}_{z_1}-k_{0z_1}\dot{I}{e}_{z_1}\hfill \\ {\dot{z}}_{2,d}={\ddot{z}}_{1,ref}-k_z{}_1\left(z_2-{\dot{z}}_{1,ref}\right)-k_{0z_1}{e}_{z_1},\hfill \end{array}$$

(18)

Now, the tracking error $e_{z2}$ is obtained by the following:

$$e_{z_2}=z_2-z_{2d};$$

and its dynamics results are

$${\dot{e}}_{z_2}=-g+\frac{1}{m}{c}_{\varphi 1}{c}_{\theta 1}{u}_1-{\dot{z}}_{2,d} ,$$

(19)

then, the control input of altitude u₁ proposed is

$$u_1=\frac{m}{c_{\varphi 1}{c}_{\theta 1}}\left({\dot{z}}_{2,d}+g-k_{z_2}{e}_{z_2}-k_{0z_2}I{e}_{z_2}\right),$$

(20)

with $c_{\varphi }\ne 0,c_{\theta }\ne 0$.

Considering [37], the closed-dynamics considering (17) and (20) become

$$\begin{gathered} \left(\begin{array}{c}\dot{I{e}_{z_1}}\\ {\dot{e}}_{z,1}\end{array}\right)=A_{z,1}\left(\begin{array}{c}I{e}_{z_1}\\ e_{z,1}\end{array}\right), A_{z,1}=\left(\begin{array}{cc}0& 1\\ -k_{0z_1}& -k_{z_1}\end{array}\right) \\ \left(\begin{array}{c}\dot{I{e}_{z_2}}\\ {\dot{e}}_{z,2}\end{array}\right)=A_{z,2}\left(\begin{array}{c}I{e}_{z_2}\\ e_{z,2}\end{array}\right), A_{z,2}=\left(\begin{array}{cc}0& 1\\ -k_{0z_2}& -k_{z_2}\end{array}\right) \end{gathered}$$

(21)

where the integral gains $k_{0z_1}, k_{0z_2}$ and proportional gains $k_{z_1},k_{z_2}$ are fixed so that the error dynamics are exponentially stable.

2.4.3. Longitudinal Motion Control

In this subsection, making use of the subsystem $S_2$ and applying the technique of virtual control, longitudinal motion is studied, which is described by the following system of equations.

$$\begin{array}{c}{\dot{x}}_1=x_2\hfill \\ {\dot{x}}_2=\frac{1}{m}(c_{{\varphi }_1}{s}_{{\theta }_1}{c}_{{\psi }_1}+s_{{\varphi }_1}{s}_{{\psi }_1})u_1\hfill \\ {\dot{\theta }}_1={\theta }_2\hfill \\ {\dot{\theta }}_2=\left(\frac{J_z-J_x}{J_y}\right){\varphi }_2{\psi }_2+\frac{J_R}{J_y}{\varphi }_2{\omega }_R+\frac{l}{J_y}{u}_3\hfill \end{array}$$

(22)

The tracking error $e_{x_1}=x_1-x_{1,ref}$ is considered, and the dynamics of error results:

$${\dot{e}}_{x_1}=x_2-{\dot{x}}_{1,ref}.$$

(23)

To eliminate disturbances, an integrative term $\dot{I}{e}_{x_1}=e_{x_1}$ is added to the controller, and the virtual control is proposed,

$$x_{2,d}={\dot{x}}_{1,ref}-k_{x_1}{e}_{x_1}-k_{0x_1}I{e}_{x_1}.$$

(24)

Differentiation $x_{2,d}$ with respect to the time,

$$\begin{array}{c}{\dot{x}}_{2,d}={\ddot{x}}_{1,ref}-k_{x_1}{\dot{e}}_{x_1}-k_{0x_1}\dot{I}{e}_{x_1}\hfill \\ {\dot{x}}_{2,d}={\ddot{x}}_{1,ref}-k_{x_1}\left(x_2-{\dot{x}}_{1,ref}\right)-k_{0x_1}{e}_x{}_1.\hfill \end{array}$$

(25)

The error is $e_{x_2}=x_2-x_{2d}$ and its dynamics results are:

$$\begin{array}{c}{\dot{e}}_{x_2}={\dot{x}}_2-{\dot{x}}_{2,d}\hfill \\ {\dot{e}}_{x_2}=\frac{1}{m}\left(c_{{\varphi }_1}{s}_{{\theta }_1}{c}_{{\psi }_1}+s_{{\varphi }_1}{s}_{{\psi }_1} \right)u_1-{\dot{x}}_{2,d};\hfill \end{array}$$

(26)

for stabilization of $e_{x_2}$, the value for $s_{{\theta }_1}$ can be fixed by solving (26) as follows:

$$s_{{\theta }_{1,d}}=\frac{m}{c_{{\varphi }_1}{c}_{{\psi }_1}{u}_1}\left({\dot{x}}_{2,d}-\frac{1}{m}{s}_{{\varphi }_1}{s}_{{\psi }_1}{u}_1-k_{x_2}{e}_{x_2}-k_{0x_2}I{e}_{x_2}\right).$$

(27)

To impose the reference, $\theta$ is considered the tracking error $e_{{\theta }_1}=s_{{\theta }_1}-s_{{\theta }_{1,d}}$ and its derivative

$$\begin{array}{c}{\dot{e}}_{{\theta }_1}={\dot{s}}_{{\theta }_1}-{\dot{s}}_{{\theta }_{1,d} }\hfill \\ {\dot{e}}_{{\theta }_1}=c_{{\theta }_1}{\theta }_2-{\dot{s}}_{{\theta }_{1,d}},\hfill \end{array}$$

(28)

For virtual control, ${\theta }_{2,d}$ is used

$$\begin{array}{c}-k_{{\theta }_1}{e}_{{\theta }_1}-k_{0{\theta }_1}I{e}_{{\theta }_1}=c_{{\theta }_1}{\theta }_{2,d}-{\dot{s}}_{{\theta }_{1,d}};\hfill \\ {\theta }_{2,d}=\frac{1}{c_{\theta }}\left({\dot{s}}_{{\theta }_{1,d}}-k_{{\theta }_1}{e}_{{\theta }_1}-k_{0{\theta }_1}I{e}_{{\theta }_1}\right),\hfill \end{array}$$

(29)

Defining $e_{{\theta }_2}$

$$\begin{array}{c}{e}_{{\theta }_2}={\theta }_2-{\theta }_{2,d};\hfill \\ {\dot{e}}_{{\theta }_2}={\dot{\theta }}_2-{\dot{\theta }}_{2,d},\hfill \end{array}$$

(30)

the dynamics of the error results

$${\dot{e}}_{{\theta }_2 }=\frac{J_z-J_x}{J_y}{\varphi }_2{\psi }_2+\frac{J_R}{J_y}{\varphi }_2{\omega }_R+\frac{l}{J_y}{u}_3-{\dot{\theta }}_{2,d},$$

(31)

and finally, the control input $u_3$ is obtained as

$$u_3=\frac{J_y}{d}\left({\dot{\theta }}_{2,d}-k_{{\theta }_2}{e}_{{\theta }_2}-k_{0{\theta }_2}I{e}_{{\theta }_2}-\frac{J_z-J_x}{J_y}{\varphi }_2{\psi }_2-\frac{J_R}{J_y}{\varphi }_2{\omega }_R\right),$$

(32)

2.4.4. Lateral Movement Control

A lateral movement controller is designed in this section and its operation is along the y-axis and the UAV. Using a procedure similar to that described in Section 2.4.2, the subsystem $S_3$ is used:

$$\begin{array}{c}{\dot{y}}_1=y_2\hfill \\ {\dot{y}}_2=\frac{1}{m}(c_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}-s_{{\varphi }_1}{c}_{{\psi }_1})u_1\hfill \\ {\dot{\varphi }}_1={\varphi }_2\hfill \\ {\dot{\varphi }}_2=\left(\frac{J_y-J_z}{J_x}\right){\theta }_2{\psi }_2-\frac{J_R}{J_x}{\theta }_2{\omega }_R+\frac{l}{J_x}{u}_2\hfill \end{array}$$

(33)

the tracking error and dynamic error are defined by,

$$\begin{array}{c}{e}_{y1}=y-y_{1,ref} \hfill \\ {\dot{e}}_{y1}=y_2-{\dot{y}}_{1,ref},\hfill \end{array}$$

(34)

adding an integrative term to the controller $\dot{I}{e}_{y_1}=e_{y_1}$, the virtual control for $y_{2,d}$ is obtained

$$\begin{array}{c}-k_{y_1}{e}_{y_1}-k_{0y_1}I{e}_{y_1}=y_{2,d}-{\dot{y}}_{1,ref}\hfill \\ y_{2,d}={\dot{y}}_{1,ref}-k_{y_1}{e}_{y_1}-k_{0y_1}I{e}_{y_1},\hfill \end{array}$$

(35)

differentiation with respect to the time of Equation (35) yields

$$\begin{array}{c}{\dot{y}}_{2,d}={\ddot{y}}_{1,ref}-k_{y_1}{\dot{e}}_{y_1}-k_{0y_1}\dot{I}{e}_{y_1}\hfill \\ {\dot{y}}_{2,d}={\ddot{y}}_{1,ref}-k_{y_1}\left(y_2-{\dot{y}}_{1,ref}\right)-k_{0y_1}{e}_y{}_1.\hfill \end{array}$$

(36)

Subsequently, the tracking $e_{y_2}=y_2-y_{2,d}$ is defined, and the dynamic is as follows:

$$\begin{array}{c}{\dot{e}}_{y2}={\dot{y}}_2-{\dot{y}}_{2,d}; \hfill \\ {\dot{e}}_{y_2}=\frac{1}{m}\left(c_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}-s_{{\varphi }_1}{c}_{{\psi }_1}\right)u_1-{\dot{y}}_{2,d}.\hfill \end{array}$$

(37)

Now, it clears $s_{{\varphi }_1}$ to obtain

$$\begin{array}{c}-k_{y_2}{e}_{y_2}-k_{0y_2}I{e}_{y_2}=\frac{1}{m}{c}_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}{u}_1-\frac{1}{m}{s}_{{\varphi }_1}{c}_{{\psi }_1}{u}_1-{\dot{y}}_{2,d}\hfill \\ s_{{\varphi }_{1,d}}=\frac{m}{c_{{\psi }_1}{u}_1}\left(\frac{1}{m}{c}_{{\varphi }_1}{s}_{{\theta }_1}{s}_{{\psi }_1}{u}_1-{\dot{y}}_{2,d}+k_{y_2}{e}_{y_2}+k_{0y_2}I{e}_{y_2}\right).\hfill \end{array}$$

(38)

Considering the angular error $e_{{\varphi }_1}=s_{{\varphi }_1}-s_{{\varphi }_{1,d}}$, its dynamics can be expressed as

$$\begin{array}{c}{\dot{e}}_{{\varphi }_1}={\dot{s}}_{{\varphi }_1}-{\dot{s}}_{{\varphi }_{1,d}}\hfill \\ {\dot{e}}_{{\varphi }_1}=c_{{\varphi }_1}{\varphi }_2-{\dot{s}}_{{\varphi }_{1,d}}.\hfill \end{array}$$

(39)

On the other hand, for virtual control, ${\varphi }_{2,d}$ is used

$$\begin{array}{c}-k_{{\varphi }_1}{e}_{{\varphi }_1}-k_{0{\varphi }_1}I{e}_{{\varphi }_1}=c_{{\varphi }_1}{\varphi }_{2,d}-{\dot{s}}_{{\varphi }_{1,d}}\hfill \\ {\varphi }_{2,d}=\frac{1}{c_{\varphi }}\left({\dot{s}}_{{\varphi }_{1,d}}-k_{{\varphi }_1}{e}_{{\varphi }_1}-k_{0{\varphi }_1}I{e}_{{\varphi }_1}\right).\hfill \end{array}$$

(40)

Furthermore, with $e_{{\varphi }_2}={\varphi }_2-{\varphi }_{2,d}$, the dynamic is defined by

$${\dot{e}}_{{\varphi }_2}={\dot{\varphi }}_2-{\dot{\varphi }}_{2,d}.$$

(41)

Now, using (33) in (41), the angular velocity error dynamic is expressed as

$${\dot{e}}_{{\varphi }_2 }=\frac{J_y-J_z}{J_x}{\theta }_2{\psi }_2-\frac{J_R}{J_x}{\theta }_2{\omega }_R+\frac{l}{J_x}{u}_2-{\dot{\varphi }}_{2,d}.$$

(42)

Finally, the control input $u_2$ proposed is

$$\begin{array}{c}{u}_2=\frac{J_x}{l}\left({\dot{e}}_{{\varphi }_2}-\frac{J_y-J_z}{J_x}{\theta }_2{\psi }_2+\frac{J_R}{J_x}{\theta }_2{\omega }_R+{\dot{\varphi }}_{2,d}\right);\hfill \\ u_2=\frac{J_x}{l}\left({\dot{\varphi }}_{2,d}-k_{{\varphi }_2}{e}_{{\varphi }_2}-k_{0{\varphi }_2}I{e}_{{\varphi }_2}-\frac{J_y-J_z}{J_x}{\theta }_2{\psi }_2+\frac{J_R}{J_x}{\theta }_2{\omega }_R\right).\hfill \end{array}$$

(43)

2.4.5. Yaw Motion Control

For rotation control, the subsystem $S_4$ is considered

$$\begin{array}{c}{\dot{\psi }}_1={\psi }_2;\hfill \\ {\dot{\psi }}_2=\left(\frac{J_x-J_y}{J_z}\right){\varphi }_2{\theta }_2+\frac{1}{J_z}{u}_4, \hfill \end{array}$$

(44)

where the error system is $e_{{\psi }_1}={\psi }_1-{\psi }_{1,ref}$, and its dynamic results are

$${\dot{e}}_{{\psi }_1}={\psi }_2-{\dot{\psi }}_{1,ref}.$$

(45)

Defining the virtual control as in the previous sections, ${\psi }_{2,d}$ is:

$${\psi }_{2,d}={\dot{\psi }}_{1,ref}-k_{\psi }{}_1{e}_{{\psi }_1}-k_{0{\psi }_1}I{e}_{{\psi }_1}.$$

(46)

The differentiation of (46) with respect to time is obtained as

$${\dot{\psi }}_{2,d}={\ddot{\psi }}_{1,ref}-k_{{\psi }_1}{\dot{e}}_{{\psi }_1}-k_{0{\psi }_1}\dot{I}{e}_{{\psi }_1}.$$

(47)

Finally, with the error $e_{\psi 2}={\psi }_2-{\psi }_{2,d} ,$ its dynamic is defined as

$$\begin{array}{c} {\dot{e}}_{\psi 2}={\dot{\psi }}_2-{\dot{\psi }}_{2,d}\hfill \\ {\dot{e}}_{{\psi }_2}=\left(\frac{J_x-J_y}{J_z}\right){\varphi }_2{\theta }_2+\frac{1}{J_z}{u}_4-{\dot{\psi }}_{2,d}, \hfill \end{array}$$

(48)

then the controller input $u_4$ is defined by

$$u_4=J_z\left({\dot{\psi }}_{2,d}-k_{{\psi }_2}{e}_{{\psi }_2}-k_{0{\psi }_2}I{e}_{{\psi }_2}-\left(\frac{J_x-J_y}{J_z}\right){\varphi }_2{\theta }_2\right).$$

(49)

As in Section 2.4.2., the exponential stability can be inferred from the yaw error dynamics, i.e.:

$$\begin{gathered} \left(\begin{array}{c}\dot{I{e}_{{\psi }_1}}\\ {\dot{e}}_{{\psi }_1}\end{array}\right)=A_{\psi ,1}\left(\begin{array}{c}I{e}_{{\psi }_1}\\ e_{\psi ,1}\end{array}\right), A_{\psi ,1}=\left(\begin{array}{cc}0& 1\\ -k_{0{\psi }_1}& -k_{\psi }{}_1\end{array}\right) \\ \left(\begin{array}{c}\dot{I{e}_{{\psi }_2}}\\ {\dot{e}}_{\psi ,2}\end{array}\right)=A_{\psi ,2}\left(\begin{array}{c}I{e}_{{\psi }_2}\\ e_{\psi ,2}\end{array}\right), A_{\psi ,2}=\left(\begin{array}{cc}0& 1\\ -k_{0{\psi }_2}& -k_{{\psi }_2}\end{array}\right) \end{gathered}$$

(50)

where the integral gains $k_{0{\psi }_1}, k_{0{\psi }_2}$ and proportional gains $k_{{\psi }_1},k_{{\psi }_2}$ are fixed so that the error dynamics are exponentially stable.

3. Results

Three experiments are documented to show the functionality of our proposal. For each case, the collision-free trajectories are calculated with one frame by the TGBOA algorithm in real-time. Subsequently, a trajectory is selected and employed as a reference for the PI-like controller. MATLAB 2022a is used for the controller simulation because of its compatibility with other platforms and its wide variety of tools and different uses in the industry, allowing the use of the algorithm in later works.

3.1. Experiment 1

Figure 6a shows the video frame transmitted from the UAV to the workstation used for the first experiment. In the digital image, thirteen regions are defined as $R_{1–13}$, and objects are detected in regions $R_1$ and $R_{9–13}$ whose centers are marked in red. Consequently, collision-free regions are detected from region $R_2$ to region $R_8$ (centers are marked in blue). Subsequently, using the procedure described in Section 2.3, the collision-free trajectories between the object and UAV are calculated. The time required for the TGBOA algorithm is 26 ms in this frame. These trajectories are expressed using their vector equations, as seen in

Table 3. Collision-free trajectories generated by object detection for frame shown in Figure 6a.

Region	Vector Equation	Parametric Equations	Symmetric Equations
$R_k$	${\overrightarrow{f}}_k=\left(0,0,0\right)+t\left(x,y_k,z_k\right) \left(\mathrm{m}\right)$	$\begin{array}{c}x=x_{k}t\\ y=y_{k}t\\ z=z_{k}t\end{array}\left(\mathrm{m}\right)$	$\frac{x}{x_k}=\frac{y}{y_k}=\frac{z}{z_k} \left(\mathrm{m}\right)$
$R_2$	${\overrightarrow{f}}_2=\left(0,0,0\right)+t\left(10,-2.4 , 0.613\right)$	$\begin{array}{c}{x}_2=10t \\ y_2=-2.4t \\ z_2=1.613t\end{array}$	$\frac{x_2}{10}=\frac{y_2}{-2.4}=\frac{z_2}{0.613}$
$R_3$	${\overrightarrow{f}}_3=\left(0,0,0\right)+t\left(10,-2.4 , 1.838\right)$	$\begin{array}{c}{x}_3=10t \\ y_3=-2.4t \\ z_3=1.838t\end{array}$	$\frac{x_3}{10}=\frac{y_3}{-2.4}=\frac{z_3}{1.838}$
$R_4$	${\overrightarrow{f}}_4=\left(0,0,0\right)+t\left(10,-0.8 , 1.838\right)$	$\begin{array}{c}{x}_4=10t \\ y_4=-0.8t \\ z_4=1.838t\end{array}$	$\frac{x_4}{10}=\frac{y_4}{-0.8}=\frac{z_4}{1.838}$
$R_5$	${\overrightarrow{f}}_5=\left(0,0,0\right)+t\left(10, 0.8 , 1.838\right)$	$\begin{array}{c}{x}_5=10t \\ y_5=0.8t \\ z_5=1.838t\end{array}$	$\frac{x_5}{10}=\frac{y_5}{0.8}=\frac{z_5}{1.838}$
$R_6$	${\overrightarrow{f}}_6=\left(0,0,0\right)+t\left(10, 2.4 , 1.838\right)$	$\begin{array}{c}{x}_6=10t \\ y_6=2.4t \\ z_6=1.838t\end{array}$	$\frac{x_6}{10}=\frac{y_6}{2.4}=\frac{z_6}{1.838}$
$R_7$	* ${\overrightarrow{f}}_7=\left(0,0,0\right)+t\left(10, 2.4 , 0.613\right)$	$\begin{array}{c}{x}_7=10t \\ y_7=2.4t \\ z_7=0.613t\end{array}$	$\frac{x_7}{10}=\frac{y_7}{2.4}=\frac{z_7}{0.613}$

* The vector ${\overrightarrow{f}}_7$ is marked in blue because this vector is used as a reference in the UAV controller.

. Then, the trajectory ${\overrightarrow{f}}_7$ is selected, and it is marked in blue. Figure 6b illustrates the reference vector for the controller described in Section 2.4.

On the other hand, the simulation controller results are shown by a graphic in Figure 7: Figure 7a is the result for the position control, including horizontal, frontal, and altitude control; Figure 7b presents the results for the attitude control; Figure 7c shows the control input signal controller; and finally Figure 7d shows a 3D representation of the simulation. In each case, the generated reference is taken and discretized at five points, starting from the origin of the center of the selected free zone determined by the video analysis (see Figure 6a). The interval between each reference point is 5 s. Based on Figure 7, at all points, the PI-like controller reaches its stability in 4 s, and then 25 s is the total time required to complete the reference vector.

3.2. Experiment 2

For this case, the video frame acquired from the UAV camera is shown in Figure 8a. As observed, there are six regions $(R_2$, $R_{9–13})$ with detected objects whose centers are marked with red dots, and the free regions found are $R_1, R_{3–8}$ (center marked with a blue dot). Using the UAV position coordinates and the central coordinates of the collision-free regions, trajectories are generated and represented through vector, parametric, and symmetric equations. These equations can be seen in

Table 4. Collision-free trajectories generated by the TGBOA algorithm using the frame shown in Figure 8a.

Region	Vector Equation	Parametric Equations	Symmetric Equations
$R_k$	${\overrightarrow{f}}_k=\left(0,0,0\right)+t\left(x,y_k,z_k\right) \left(\mathrm{m}\right)$	$\begin{array}{c}x=x_{k}t\\ y=y_{k}t\\ z=z_{k}t\end{array}\left(\mathrm{m}\right)$	$\frac{x}{x_k}=\frac{y}{y_k}=\frac{z}{z_k} \left(\mathrm{m}\right)$
$R_1$	* ${\overrightarrow{f}}_1=\left(0,0,0\right)+t\left(10, 0 , 0\right)$	$\begin{array}{c}{x}_1=10t \\ y_1=0t \\ z_1=0t\end{array}$	$\frac{x_1}{10}=\frac{y_1}{0}=\frac{z_1}{0}$
$R_3$	${\overrightarrow{f}}_3=\left(0,0,0\right)+t\left(10,-2.4 , 1.838\right)$	$\begin{array}{c}{x}_3=10t \\ y_3=-2.4t \\ z_3=1.838t\end{array}$	$\frac{x_3}{10}=\frac{y_3}{-2.4}=\frac{z_3}{1.838}$
$R_4$	${\overrightarrow{f}}_4=\left(0,0,0\right)+t\left(10,-0.8 , 1.838\right)$	$\begin{array}{c}{x}_4=10t \\ y_4=-0.8t \\ z_4=1.838t\end{array}$	$\frac{x_4}{10}=\frac{y_4}{-0.8}=\frac{z_4}{1.838}$
$R_5$	${\overrightarrow{f}}_5=\left(0,0,0\right)+t\left(10, 0.8 , 1.838\right)$	$\begin{array}{c}{x}_5=10t \\ y_5=0.8t \\ z_5=1.838t\end{array}$	$\frac{x_5}{10}=\frac{y_5}{0.8}=\frac{z_5}{1.838}$
$R_6$	${\overrightarrow{f}}_6=\left(0,0,0\right)+t\left(10, 2.4 , 1.838\right)$	$\begin{array}{c}{x}_6=10t \\ y_6=2.4t \\ z_6=1.838t\end{array}$	$\frac{x_6}{10}=\frac{y_6}{2.4}=\frac{z_6}{1.838}$
$R_7$	${\overrightarrow{f}}_7=\left(0,0,0\right)+t\left(10, 2.4 , 0.613\right)$	$\begin{array}{c}{x}_7=10t \\ y_7=2.4t \\ z_7=0.613t\end{array}$	$\frac{x_7}{10}=\frac{y_7}{2.4}=\frac{z_7}{0.613}$
$R_8$	${\overrightarrow{f}}_8=\left(0,0,0\right)+t\left(10, 2.4 ,-\mathrm{0..613}\right)$	$\begin{array}{c}{x}_8=10t \\ y_8=2.4t \\ z_8=0.613t\end{array}$	$\frac{x_8}{10}=\frac{y_8}{2.4}=\frac{z_8}{0.613}$

* The vector ${\overrightarrow{f}}_1$ is marked in blue due to this vector being used as a reference in the UAV controller.

. Next, the vector ${\overrightarrow{f}}_1$ is used as a reference for the UAV controller, and its graph is displayed in Figure 8b. In this case, the TGBOA algorithm displays the trajectory in 21 ms. On the other hand, Figure 9 shows the results obtained for the second experiment when the vector ${\overrightarrow{f}}_1$ is applied as a reference in the controller.

For this experiment, five points are taken from the chosen trajectory with a duration of five seconds between them. Each reference point is reached in 4 s as observed in Figure 9a. The longitudinal movement retains the same behavior as in Experiment 1, while the lateral movement remains constant as the UAV follows a line forward. In addition, the altitude is maintained until the final point is reached. Again, the last point stabilizes after 24 s.

3.3. Experiment 3

This experiment uses the same methodology as the first by moving objects and taking a new frame for the TGBOA algorithm, detecting the position, finding collision-free zones, and generating the optimal trajectory. In Figure 10a, we can see the video frame for Experiment 3, where the objects are in the regions $R_{1 }$ and $R_8$ with the centers marked in red. Therefore, the collision-free regions are $R_{2–7}$ and $R_{9–13}$, whose centers are marked in blue. Following the procedure described in Section 2.3, the possible trajectories for the UAV are calculated, and each trajectory is represented through a vector, parametric, and symmetric equation. Such equations are observable in

Table 5. Collision-free trajectories generated by the TGBOA algorithm using the frame shown in Figure 10a.

Region	Vector Equation	Parametric Equations	Symmetric Equations
$R_k$	${\overrightarrow{f}}_k=\left(0,0,0\right)+t\left(x,y_k,z_k\right) \left(\mathrm{m}\right)$	$\begin{array}{c}x=x_{k}t\\ y=y_{k}t\\ z=z_{k}t\end{array}\left(\mathrm{m}\right)$	$\frac{x}{x_k}=\frac{y}{y_k}=\frac{z}{z_k} \left(\mathrm{m}\right)$
$R_2$	${\overrightarrow{f}}_2=\left(0,0,0\right)+t\left(10,-2.4 , 0.613\right)$	$\begin{array}{c}{x}_2=10t \\ y_2=-2.4t \\ z_2=0.613t\end{array}$	$\frac{x_2}{10}=\frac{y_2}{-2.4}=\frac{z_2}{0.613}$
$R_3$	${\overrightarrow{f}}_3=\left(0,0,0\right)+t\left(10,-2.4 , 1.838\right)$	$\begin{array}{c}{x}_3=10t \\ y_3=-2.4t \\ z_3=1.838t\end{array}$	$\frac{x_3}{10}=\frac{y_3}{-2.4}=\frac{z_3}{1.838}$
$R_4$	${\overrightarrow{f}}_4=\left(0,0,0\right)+t\left(10,-0.8 , 1.838\right)$	$\begin{array}{c}{x}_4=10t \\ y_4=-0.8t \\ z_4=1.838t\end{array}$	$\frac{x_4}{10}=\frac{y_4}{-0.8}=\frac{z_4}{1.838}$
$R_5$	${\overrightarrow{f}}_5=\left(0,0,0\right)+t\left(10, 0.8 , 1.838\right)$	$\begin{array}{c}{x}_5=10t \\ y_5=0.8t \\ z_5=1.838t\end{array}$	$\frac{x_5}{10}=\frac{y_5}{0.8}=\frac{z_5}{1.838}$
$R_6$	${\overrightarrow{f}}_6=\left(0,0,0\right)+t\left(10, 2.4 , 1.838\right)$	$\begin{array}{c}{x}_6=10t \\ y_6=2.4t \\ z_6=1.838t\end{array}$	$\frac{x_6}{10}=\frac{y_6}{2.4}=\frac{z_6}{1.838}$
$R_7$	${\overrightarrow{f}}_7=\left(0,0,0\right)+t\left(10, 2.4 , 0.613\right)$	$\begin{array}{c}{x}_7=10t \\ y_7=2.4t \\ z_7=0.613t\end{array}$	$\frac{x_7}{10}=\frac{y_7}{2.4}=\frac{z_7}{0.613}$
$R_9$	${\overrightarrow{f}}_9=\left(0,0,0\right)+t\left(10 , 2.4,-1.838\right)$	$\begin{array}{c}{x}_9=10t \\ y_9=2.4t \\ z_9=-1.838t\end{array}$	$\frac{x_9}{10}=\frac{y_9}{2.4}=\frac{z_9}{-1.838}$
$R_{10}$	${\overrightarrow{f}}_{10}=\left(0,0,0\right)+t\left(10, 0.8,-1.838\right)$	$\begin{array}{c}{x}_{10}=10t \\ y_{10}=0.8t \\ z_{10}=-1.838t\end{array}$	$\frac{x_{10}}{10}=\frac{y_{10}}{0.8}=\frac{z_{10}}{-1.838}$
$R_{11}$	${\overrightarrow{f}}_{11}=\left(0,0,0\right)+t\left(10,-0.8,-1.838\right)$	$\begin{array}{c}x=10t \\ y_2=-0.8t \\ z_{11}=-1.838t\end{array}$	$\frac{x_{11}}{10}=\frac{y_{12}}{-0.8}=\frac{z_{12}}{-1.838}$
$R_{12}$	* ${\overrightarrow{f}}_{12}=\left(0,0,0\right)+t\left(10,-2.4,-1.838\right)$	$\begin{array}{c}{x}_{12}=10t \\ y_{12}=-2.4t \\ z_{12}=-1.838\end{array}$	$\frac{x_{12}}{10}=\frac{y_{12}}{-2.4}=\frac{z_{12}}{-1.838}$
$R_{13}$	${\overrightarrow{f}}_{13}=\left(0,0,0\right)+t\left(10,-2.4,-0.613\right)$	$\begin{array}{c}{x}_{13}=10t \\ y_{13}=-2.4t \\ z_{13}=-0.613t\end{array}$	$\frac{x_{13}}{10}=\frac{y_{13}}{-2.4}=\frac{z_{13}}{-0.613}$

* The vector ${\overrightarrow{f}}_{12}$ is marked in blue due to this vector being used as a reference in the UAV controller.

. The references are obtained in 28 ms by the TGBOA algorithm. Subsequently, the trajectory ${\overrightarrow{f}}_{12}$ is used as a reference in the controller (see Figure 10b), and then the numerical results can be seen in Figure 11.

For the last experiment presented in this paper, the trajectory pointing towards zone number 12 is used, and the simulation controller results are shown in Figure 11, where the dotted line represents the reference, and the line represents the numerical results. It is observed that the PI-like controller is stable since it reaches each point of the trajectory in approximately 4 s and it is maintained until the next point, reaching the center of the collision-free zone in 25 s.

3.4. Object Avoidance

With the results obtained in the three experiments (Section 3.1–Section 3.3), the behavior graph of object avoidance is shown in Figure 12.

Analyzing Figure 12 reveals the UAV travels in an initial charged trajectory in the controller, until an object is detected with the vision sensing system using the TGBOA algorithm to create the new trajectory. Then, the new trajectory is added to the PI-like controller. Note that the new trajectory ${\overrightarrow{f}}_7$ (Experiment 1) evades the object located during the flight, as observed in Figure 6. This new trajectory ${\overrightarrow{f}}_7$ is followed until the visual sensing systems detect other objects in the UAV trajectory, and as a consequence, the TGBOA algorithm creates a new trajectory again ${\overrightarrow{f}}_1$(Experiment 2), and then is added to the PI-like controller, updating the UAV with the new objects in the scenario. The ${\overrightarrow{f}}_1$ makes the drone to evade in accordance with the new values added to the algorithm. Notice that the UAV follows the ${\overrightarrow{f}}_1$ until the vision sensing system detects objects in the scenario, and then it is necessary to create a new trajectory with the TGBOA algorithm; in this case, the trajectory vector ${\overrightarrow{f}}_{12}$ (Experiment 3) is created. Please refer to Figure 11 and Table 5. As in the other cases, ${\overrightarrow{f}}_{12}$ allows the drone to evade objects thanks to a change in the direction or trajectory.

Therefore, considering the results shown in Figure 12, the TGBOA algorithm creates new trajectories in real-time using the evading of objects. These are detected with the visual sensing system. The trajectory vectors are then added to the controller, and with this new information the drone moves to a new position.

4. Discussion

In this work, a vision system was applied for object detection to generate real-time trajectories for drones and was probed as references in a newly designed PI-like controller. A new algorithm called trajectory generation based on object avoidance (TGBOA) was reported. In the TGBOA algorithm, the trajectories were created based on object evasion and using the drone position and the central coordinates of collision-free regions. Its mathematical representation was calculated and shown on vectors, parametric, and symmetric equations. In each experiment, one vector was selected and used as a reference in the PI-like controller (see Section 2.4) by a simulation software. The numerical results obtained verify the operation of our proposal since the drone always reached the reference generated by the TGBOA algorithm. Based on the results, the following points can be highlighted:

• The new trajectory generation based on object avoidance (TGBOA) algorithm is proposed to generate real-time trajectories for drones. The TGBOA algorithm is based on evading detected objects using a vision sensing system.
• The TGBOA algorithm is experimentally checked since the trajectory vectors were generated from a video analysis. Then, they are applied as references in the designated PI-like controller.
• Our new proposal generates new trajectories in real-time for drone controllers, taking into consideration changing scenarios because of its movement or random natural scenarios.
• The TGBOA algorithm is more efficient in the creation of new drones’ trajectories.
• The TGBOA algorithm is easy to implement because it is based on the detection of objects in a changing scenario, and it is not based on the optic flow concept [35]. As a consequence, the number of computational operations and the mathematical complexity are reduced.
• With the objects evading model from the TGBOA algorithm, the drone can be more autonomous when it travels from an initial point $f_A$ to a final point $f_B$, even though this is a changing scenario.
• The TGBOA algorithm can be implemented in specialized development boards.

However, due to the applied software, the data transmission, and the controller, time limitations take place in the execution such as:

• For each frame, the TGBOA algorithm creates a new trajectory in 30 ms.
• Including the transmission time between the UAV and the workstation, the total time to produce and update a trajectory is up to 390 ms, due to the delay in the Wi-Fi network. Then, the data transmission time is 360 ms.
• Experimentally, the execution time is long since the drone’s controller requires up to 25 s to reach the center of the chosen free-collision zone.
• The total execution time is experimentally 25.360 s.
• MATLAB 2022a is used due to its compatibility with other platforms and its wide variety of tools and different uses in the industry, allowing the use of the algorithm in future projects.

Comparing our trajectory generation proposal (TGBOA algorithm) with those reported in the references, the TGBOA algorithm is functional for random and dynamic scenarios, is based on object evasion, operates in real-time, and can work online. However, in the reported techniques, the surrounding environment cannot change, the controller references must be preloaded, and the trajectory works offline. Based on the above, the TGBOA algorithm offers advantages compared to the previously described techniques, due to how it operates in changing and random situations.

Comparing the TGBOA algorithm with the proposals reported in the references [7,19,24,25], the TGBOA algorithm reduces complexity because our work is based on the detection of moving objects, while the indicated references are based on the calculation of optical flow, which is calculated through the frequency components of the image or optimization algorithms.

Table 6. Comparison between related work and TGBOA algorithm.

	Relevant Aspect	Our Proposal
[19]	High image sampling and position control signal rate. Expensive hardware and requires prior knowledge or estimation of the objects in its spatial environment.	Only needs a vision sensor, and online trajectory references are generated and updated in real-time.
[23]	Large amount of computational resources used for CNN. Three vision sensors needed. High performance boards used to improve the performance.	Improves efficiency by focusing the algorithm on object detection and drone position, thus reducing computational complexity.
[24]	Updating environment requires a lot of time and computer resources. Dynamic scenarios are not considered.	Focuses on the position of moving objects and dynamic scenarios without relying on full scenario or feature extraction.
[25]	The optical flow constructed from the image sequence makes it difficult to achieve the real-time performance on the low-cost hardware. The presence of moving obstacles is not considered.	Easy to implement because it is based on the detection of objects in a changing scenario and not based on the optic flow concept with Fourier transform [38].

shows the comparison between the TGBOA algorithm and the references where the trajectory of the drone is determined by applying the optical flow technique.

Based on Table 3, Table 4 and Table 5 and Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12, the TGBOA algorithm works correctly in the generation of trajectories in real-time only if the vision system is correctly applied in the evasion of objects. However, the runtime is high due to the software used in our experiments. Therefore, our future work has the following directions: the first is to reduce the execution time using electronic development cards, the second is to improve the transmission data rate, and the third is to apply the TGBOA algorithm when the UAV will move through random and changing scenarios.

5. Conclusions

The new algorithm called trajectory generation based on object avoidance (TGBOA) is capable of flight planning in real-time from object detection by a vision sensor and with low computational resources by processing a video transmitted from a UAV to a workstation. The algorithm can generate collision-free trajectories that can be used as a reference in the UAV controller to avoid objects during the flight. However, the optimal trajectory will depend on the detected objects, UAV positions, and the endpoint. Thirteen regions were defined in the image for the video analysis. This number can be increased to have more trajectories generated in different directions; however, as a consequence, the execution time increases due to the number of computational operations required. This problem can be solved by programming the TGBOA algorithm on electronic cards with greater computational power and using different platforms dedicated to robotic systems. A PI-like controller was developed for a quadcopter position and yaw control. The dynamics of the UAV have been divided into four subsystems with a control input and an output variable each. Using the references obtained from the TGBOA algorithm, the flight controller led to satisfactory results in the experiments, due to where the objects were located and evaded in different areas and choosing the appropriate trajectory for each case. The numerical simulations of the proposed controllers have been implemented using the mathematical software MATLAB 2022a, and the results show a good performance of the proposed control law, reaching the proposed references in 4 s. Future work will concern the implementation of the algorithm and different controllers using other simulation platforms and development boards.