Digital Twin 3D System for Power Maintenance Vehicles Based on UWB and Deep Learning

Abstract

To address the issue of the insufficient safety monitoring of power maintenance vehicles during power operations, this study proposes a vehicle monitoring scheme based on ultra wideband (UWB) and deep learning. The UWB localization algorithm employs Chaotic Particle Swarm Optimization (CSPO) to optimize the Time Difference of Arrival (TDOA)/Angle of Arrival (AOA) locating scheme in order to overcome the adverse effects of the non-visual distance and multipath effects in substations and significantly improve the positioning accuracy of vehicles. To solve the problem of the a large aspect ratio and the angle in the process of power maintenance vehicle operation situational awareness in the mechanical arm of the maintenance vehicle, the arm recognition network is based on the You Only Look Once version 5 (YOLOv5) and modified by Convolutional Block Attention Module (CBAM). The long-edge definition method with circular smoothing label, SIoU loss function, and HardSwish activation function enhance the precision and processing speed for the arm state. The experimental results show that the proposed CPSO-TDOA/AOA outperforms other algorithms in localization accuracy and effectively attenuates the non-visual distance and multipath effects. The recognition accuracy of the YOLOv5-CSL-CBAM network is substantially improved; the mAP value of the vehicles arm reaches 85.04%. The detection speed meets the real-time requirement, and the digital twin of the maintenance vehicle is effectively realized in the 3D substation model.

1. Introduction

Irregular operations and a lack of safety awareness are the primary causes of safety accidents. With the rapid development of artificial intelligence, machine vision and wireless positioning technology have been effectively applied to the monitoring system, enabling object recognition, tracking, and safety warning.

Deep learning object recognition networks are currently a popular research topic and are widely employed in intelligent monitoring systems. Recognition networks can generally be classified into two categories: two-stage and single-stage targets. The two-stage [1] network achieves object detection via region box selection and position regression, which obtains high accuracy through tedious calculations and time consumption. For instance, Li et al. [2] used fast R-CNN to improve the detection of pedestrians and He et al. [3] used Mask R-CNN to enhance the detection of rail transit obstacles. Single-stage [4] networks directly extract the features by regression strategies and determine the location of the target. The representative algorithms are YOLO [5,6,7,8], RFBNet [9], and SSD [10,11,12,13]. In application, Lu [14] presented a method for detecting pedestrians using multiscale convolutional features and a three-layer pyramidal network to enhance pedestrian-target detection accuracy. Meanwhile, Lin [15] introduced a multi-scale feature cross-layer to improve YOLOv5 and enable the accurate identification of ultra-small targets in remote sensing images. Lin [16] proposed a traffic sign detection approach that utilized a lightweight multiscale feature fusion network, which significantly enhanced the detection performance of small targets and delivered better real-time results. Yang [17] proposed a YOLO network target tracking algorithm based on multi-feature fusion to track and localize operators’ helmets. Recently, Huang [18] utilized Alphapose with ResNet to achieve dressing detection for power operators, which can play a vital role in regulating their attire.

Ultra Wide Band (UWB) positioning technology [19] utilizes high-frequency radio pulses for triangulation positioning. It has the advantages of high accuracy, strong anti-interference performance, stable performance, and low energy consumption. Therefore, it is a popular choice for object positioning, indoor navigation, tracking, and surveillance applications. Lin [20] proposed a drift-free visual SLAM technique for mobile robot localization by integrating UWB, which resulted in a significant reduction in the overall drift error of robot navigation. Li [21] proposed a pseudo-GPS positioning system for underground coal mines consisting of noisy UWB ranging to achieve robust and accurate positioning estimation for CMR applications. Lee [22] proposed a marker-based hybrid indoor positioning system (HIPS) that performs hybrid positioning by using marker images and inertial measurement unit data from smartphones, enabling accurate navigation in subways.

The positioning of power maintenance vehicles and the state of the crank arm are the main causes of safety accidents. Vehicle supervision relies on manual monitoring, and the application of intelligent technology in vehicle monitoring is insufficient. Therefore, improving vehicle monitoring by utilizing deep learning techniques and wireless positioning technologies is an urgent problem.

This paper utilizes UWB to renew the location information of the power maintenance vehicle in a three-dimensional model of the substation and to determine whether they are within a prohibited area. Additionally, deep learning is employed to evaluate the arm status of the vehicle in a safe area, thus creating a digital twin of the vehicle in the three-dimensional substation model and facilitating safety monitoring. The innovative work of this paper is as follows:

• A chaotic particle swarm optimization TDOA/AOA algorithm is proposed to improve the TDOA/AOA method in order to find the optimal method and improve positioning accuracy with less UWB stations and antennas.
• An improved YOLOv5 state recognition network for vehicle arms has been designed. We used a long-edge definition method (LDM) and a circular smoothing labeling (CSL) complex model to achieve state recognition of rotating arms. Additionally, we introduced a CBAM attention mechanism to enhance feature extraction of the network, while employing the SIoU loss function to reduce loss value and enhance the nonlinear segmentation ability of the network. Comparative experimental results demonstrate the superiority of our method in achieving state-of-the-art performance.
• A three-dimensional digital twin monitoring system is designed; the location of the vehicle and status of the arm are live updated in the twin monitoring system.

2. Digital Twinning Route

The three-dimensional model of a substation and vehicle is modeled and depicted in Figure 1. The UWB and deep learning methods are employed separately to locate the vehicle in the operational setting and evaluate the status of its arm. Virtual vehicle real-time update via the location and status information in the 3D model. The overall route of the system is shown in Figure 2.

2.1. CPSO + TDOA/AOA Algorithm

In UWB, the TDOA/AOA algorithm can improve the localization accuracy with less base stations. It measures AOA parameters at the base station and TDOA parameters at the mobile target to estimate target localization. Although positioning results can be obtained by using only two base stations in the unobstructed environment, actual environments are affected by non-line-of-sight propagation, multipath, and geometric accuracy, which can cause location errors. To improve the positioning accuracy, this paper adopts chaotic particle swarm algorithm to increase precision of TDOA/AOA.

Chaotic Particle Swarm Optimization (CPSO) is a combination of chaotic optimization algorithm (COA) and particle swarm algorithm (PSA). CPSO can enhance the search ability of particles and avoids falling into local optimal solutions. The proposed composite scheme TDOA/AOA with CPSO optimizing is depicted in Figure 3.

When the particles of the traditional particle swarm algorithm search in a complex environment, the flight directions all point to the global optimal solution. When one of the particles finds a local optimal solution during the flight, the search speed of the remaining particles will largely slow down to zero, causing the particles to fall into the local optimal solution, i.e., premature defects. The Chaotic Particle Swarm Optimization (CPSO) algorithm is a combination of chaotic optimization and particle swarm algorithm. Chaotic optimization has the characteristics of randomness and convenience, which can enhance the search ability of particles for targets at any position in space and avoid the algorithm optimization process to fall into local optimal solutions.

There are various chaos models, mainly Logistic mapping model, the Henon mapping model, and the Lorenz mapping model. Among them, the Logistic mapping model has a simple structure and better ergodicity compared to other mapping models, and the Logistic mapping model is used as the chaos model in this paper. The logistic mapping model is

$$Z^{i+1}=\mu Z^i\left(1-Z^i\right)\begin{array}{cc}& i=0,1,2,\dots .\end{array}$$

(1)

where $\mu \in (2,4]$ is the control parameter and the value of $\mu$ is proportional to the chaotic occupancy ratio. $Z^i\in (0,1)$ is the chaotic domain, capable of generating chaotic sequences $Z^1$, $Z^2$,…, $Z^n$.

The iterative processes of the particle swarm algorithm to find the optimal solution and the global optimal solution are

$$V_i^k=\omega V_i^{k-1}+c_1{r}_1(P_{b_i}-X_i^{k-1})+c_2{r}_2(G_{b_i}-X_i^{k-1}),$$

(2)

$$X_i^k=X_i^{k-1}+V_i^{k-1}.$$

(3)

In Equations (2) and (3), $i=1,2,\dots ,N$, N is the number of particles in the particle swarm. As the value of N increases, the optimization ability of the algorithm gradually improves, but when the value of N exceeds a certain threshold, the optimization ability no longer improves and consumes more time. $k$ is the number of current update iterations. $\omega$ is the inertia weight coefficient, and the value of $\omega$ is isotropically correlated with the global search ability of the particle and anisotropically correlated with the local search ability, which is usually performed by the dynamic $\omega$ method. $c_1$ and $c_2$ are learning factors, highlighting the proportion of “self-cognition” and “social experience” of particles. Usually, $c_1=c_2\in [0,4]$. When $c_1=0$ the group diversity of the algorithm disappears and the algorithm will fall into the local optimal solution. When $c_2=0$, there is no information exchange between particles in the algorithm, and the convergence rate of the algorithm decreases. $r_1$, $r_2$ are random numbers in the range $[0,1]$. $P_{b_i}$ is the individual optimal position of the i-th particle, $G_{b_i}$ is the global optimal position of the particle population at the k − 1th iteration, and $V_i^k$ and $X_i^k$ are the velocity and position of particle i at the k-th iteration, respectively.

The iterative process of the CPSO is based on the particle swarm algorithm and is implemented as follows:

Firstly, related parameters are initialized and processed. $r_1$ and $r_2$ of Equation (2) are set random values. The initial velocity and direction of each particle are irregular, some positions will be missed in the search process, which cannot ensure ergodicity and diversity. CPSO performs a chaotic mapping of the velocities and positions of each particle in the initial stage, replacing $r_1$ and $r_2$ with the chaotic sequence formed by Equation (1) to enhance to steadily search for the global optimal solution.

Secondly, update the particle parameters. According to Equation (3), the velocity and position vector of each particle are updated iteratively, and the range of velocity is $\left[V_{\min },V_{\max }\right]$, and the positions are $\left[x_{\min },x_{\max }\right]$ and $\left[y_{\min },y_{\max }\right]$. The inertia weighting factor $\omega$ is set dynamically. It is

$$\omega ={\omega }_{\max }-\frac{k({\omega }_{\max }-{\omega }_{\min })}{k_{\max }}.$$

(4)

where ${\omega }_{\max }$ and ${\omega }_{\min }$ represent the maximum and minimum weight coefficients, respectively, and $k$ and $k_{\max }$ represent the current and maximum number of update iterations, respectively.

Thirdly, the fitness of each particle is calculated. The CPSO provides the search direction for the particles by fitness function, and the value of the fitness is anisotropically related to the particles to the function. In this paper, the fitness function is designed using the target coordinates to be measured, and its expressions is

$$Fitness\left(x^{\prime },y^{\prime }\right)=[{\left(d_{i1}-d_i+d_1\right)}^T\left(d_{i1}-d_i+d_1\right)+\frac{{\sigma }_{\epsilon }^2}{n_{\beta }^2}{\left(\beta -\arctan \left(\frac{y-y_1}{x-x_1}\right)\right)}^2]$$

(5)

where $i=2,3,4\dots N$, $d_{i1}$ denotes the distance difference between the target MS $(x,y)$ to be located and the base station $B{S}_i$ and $B{S}_1$, $d_i$ denotes the positioning distance error, $n_{\beta }$ denotes the AOA measurement noise, $\beta$ is the observation angle between $B{S}_1$ and MS, ${\sigma }_{\epsilon }^2$ is the variance of the AOA view measurement error.

Fourthly, the value of the historical fitness is updated, and it is judged whether the particle with updated fitness is in stagnation. If the particle is in stagnation, its chaotic perturbation is performed using Equation (1).

Finally, when the number of iterations reaches the maximum, the global optimal position $G_b$ corresponding to the smallest value of the fitness function is determined as the optimal solution optimized by the algorithm. Otherwise, return to the second step and continue the iterations.

2.2. YOLOv5-CSL for Vehicle Arm Recognition

A novel YOLOv5-based network for vehicle arms recognition is proposed. The network uses long-edge definition method (LDM) and circular smooth label (CSL) to reduce cross loss. HardSwish is implanted in the convolutional layer to improve the feature extraction capability and CBAM built to improve recognition accuracy of the network.

2.2.1. YOLOv5-CSL with Attention Mechanism

The proposed network adopts R-YOLOv5 as backbone network to detect vehicle arm and calculate arm angle. The structure of modified R-YOLOv5 is presented in Figure 4.

Where BottleneckCSP1_X: CSP1_X structure, BottleneckCSP2_X: CSP2_X structure, SPPF: fast spatial pyramid pooling module, Upsample: upsampling module, Concat: connection module, Conv: convolution module, Backbone: backbone network, Neck: bottleneck network, Prediction: prediction module, CBAM: attention mechanism module.

The Backbone consists of the backbone network CSPDarkNet and the spatial pyramid pooling SPPF for feature extraction. CSP1_X is applied to CSPDarkNet to enhance the feature extraction ability of images. Compared with SPP, SPPF adds two CBS modules to enhance the training efficiency of the network. The structures are presented in Figure 5 and Figure 6.

Where Conv: convolution module, BN: Batch Normalization structure, SiLU: activation function, Resunit: residual module; add: tensor summation, Concat: tensor stitching, CBS: consists of a two-dimensional convolution layer + a Bn layer + a SiLU activation function.

Where MaxPool: maximum pooling, Cancat: tensor stitching, CBS: consists of a 2D convolutional layer + a Bn layer + a SiLU activation function.

The Neck part consists of a feature pyramid network and a discriminator. cSP2_X can enhance the feature fusion capability and make the network extract more detailed features, and the structure is shown in Figure 7. The prediction part implements the object detection function for three scales: large, medium, and small. The YOLOv5 network adds 180 angle classification channels in the prediction part to accomplish prediction of object rotation angle.

Where CBS: composed of a 2D convolutional layer + a BN layer + a SiLU activation function, Concat: tensor stitching.

The hybrid attention mechanism (CBAM) is a hybrid attention mechanism that combines both channel attention and spatial attention, which is typically represented by the convolutional block attention module [23]; the network structure is shown in Figure 8.

The channel attention module (CAM) performs pooling operation and compression in spatial dimensions on the feature map F₁ extracted from the backbone network to obtain two dimensional 1 × 1 × C feature matrices, and then inputs them sequentially into the Multi-Layer Perceptron [24] (MLP) network, which is processed by the MLP network, and then inputs the Sigmoid activation function to acquire the channel attention module associated feature parameter Mc. Finally, Mc is dotted with the feature map F₁ to output the feature map Fc of the CAM.

The spatial attention module (SAM) takes Fc as the new input feature map, pools it and obtains two feature matrices with same channel, then splices them in channel order to receive a new feature matrix, convolves them and inputs them into the Sigmoid activation function to obtain the relevant feature parameters Ms of the SAM. Finally, Ms is then combined with the feature map F₂ and outputs the feature map of the whole CBAM by performing the corresponding operation.

The CBAM integrates the advantages of CAM and SAM, focuses on both channel features and spatial features, enhances the attention to important channels and focal regions of images, and improves the feature expression capability of the network. CAM and SAM in CBAM are both lightweight modules with fewer internal convolution operations, which reduce the computational effort and improve the performance of the network with a small increase in the number of network parameters.

2.2.2. Long-Edge Definition Method with Circular Smoothing Label

The vehicle arm has a large aspect ratio and multiple rotation angle. Data labeling with the rotating method can reduce the redundant information, improve the detection accuracy, and increase training efficiency of the network; however, Exchangability of Edges (EoE) [25] and Periodicity of Angle (PoA) [26] problems occur during network training to reduce recognition accuracy.

In this study, we adopted a combination of LDM and CSL to solve the boundary problem of θ. Where LDM tackles the edge variation problem, and CSL settles the angle period problem.

LDM is a five-parameter labeling method, which is a novel angle definition method and avoids the edge exchangeability. LDM describes target as ([x, y, w, h, θ]), (x, y) is the rectangular center coordinate of the rotated box, w and h are the width and length of the rectangular box, respectively. θ is the angle between the length and the x-axis, where θ ∈ [−90°,90°). The LDM is demonstrated in Figure 9:

LDM can eliminate EoE. CSL transforms the regression problem of θ into a classification problem, divides angles in different ranges and categories, and discretizes the continuous problem to avoid PoA. However, the discretization process inevitably generates an accuracy loss. To evaluate the loss, the maximum loss and the average loss of accuracy (obeying uniform distribution) are calculated by the following formula:

$$Max(loss)=\omega /2,$$

(6)

$$E(loss)={\displaystyle {\int }_a^{b}x\times \frac{1}{b-a}dx=}{\displaystyle {\int }_0^{\omega /2}x\times \frac{1}{\omega /2-0}dx=}\omega /4.$$

(7)

where $\omega$ is the width of the rectangular box and the values of a, b are in the interval $[-\frac{\pi }{2},\frac{\pi }{2}]$.

The minimum precision of angle range is 1°, and the maximum loss and expected loss were separately set to 0.50 and 0.25. When two rotating rectangular frames with a 1:9 aspect ratio were used for the test, the intersection ratio of the two rotating rectangular frames decreased by 0.05 and 0.02, the accuracy loss of the method can be acceptable.

In order to make the classification, loss can be used to predict the distance between the result and the angle label, a One-hot coding method was designed, assuming that the real angle label is 0°, and the accuracy loss values were the same when the angle alter 1° to 90°. The One-hot coding method is in Figure 10. Based on One-hot label, CSL was introduced, and the CSL is presented in Figure 11.

The expressions for CSL are

$$CSL(x)=\{\begin{array}{cc}g(x)& \theta -r<x<\theta +r\\ 0& otherwise\end{array},$$

(8)

$$\begin{array}{l}s.t.\\ \{\begin{array}{c}g(x)=g(x+kT),k\in N\\ 0\le g(\theta +\epsilon )=g(\theta -\epsilon )\le 1,\left|\epsilon \right|<r\\ 0\le g(\theta \pm \epsilon )\le g(\theta \pm \varsigma )\le 1,\left|\varsigma \right|<\left|\epsilon \right|<r\\ g(\theta )=1\end{array}\end{array}.$$

(9)

where g(x) is the window function with periodicity, monotonicity and symmetry. The radius r determines the size of the window. In this study, the Gaussian function is used as the window function with a radius of 6. The functional expression of g(x) is as following:

$$g(x)=a{e}^{-\frac{{(x-b)}^2}{2c^2}}.$$

(10)

where a, b, and c are constants, and in this paper, a is set to 1, b to 0, c to 4, and x is the angle parameter.

2.2.3. HardSwish Convolution Module

In YOLOv5 network, Leaky ReLU and SiLU are frequently used activation functions. Leaky ReLU is updated form of Rectified Linear Unit (ReLU), which introduces a fixed slope to solve the problem of fixed parameters caused by Dead ReLU, but its performance is unstable. Sigmoid-weighted Linear Unit (SiLU) and HardSwish are other forms of Swish activation function. Swish function has no maximum value but a minimum value with smoothness and non-monotonicity. Its functions are

$$Swish(x)=x\cdot Sigmoid(\beta x),$$

(11)

$$Sigmoid(\beta x)=\frac{1}{1+\exp (-\beta x)}.$$

(12)

When the β is 1, the Swish function becomes the SiLU function, and it has better performance and effect than the Leaky ReLU.

HardSwish uses a strong nonlinear function and improves the accuracy of Swish. It is

$$HardSwish(x)=\{\begin{array}{cc}0& x\le -3\\ x& x\ge 3\\ \frac{x(x+3)}{6}& others\end{array}.$$

(13)

HardSwish has a stronger nonlinear capability. The SiLU of R-YOLOv5 is replaced by the HardSwish, and the improved network structure is shown in Figure 12.

3. Maintenance Vehicle State Identification and Three-Dimensional Reproduction

3.1. CPSO + TDOA/AOA Positioning Experiment

Taylor [27], Chan [28], TDOA/AOA [29], and PSO + TDOA/AOA [30] algorithms were utilized to conduct experimental comparisons. As presented in Figure 13, the experimental and computational results were compared in different environments, including various stations, communication radii, and AOA measurement errors.

The results of using the positioning algorithm designed in this paper to locate the power maintenance vehicle in the three-dimensional model of the substation with UWB positioning equipment are shown in Figure 14. Figure 14a,b represent the positioning results of the power maintenance vehicle at different operating positions, and it can be seen from the figures that the algorithm designed in this paper can accurately locate the maintenance vehicle.

3.2. Experiment of Crank Arm State Recognition

3.2.1. Experimental Environment and Evaluation Criteria

The experimental platform is PyCharm and Microsoft visual studio 2017, the computer operating system is Windows 10, the graphics card model is a NVIDIA TITAN XP with 12 G of video memory, and the deep learning framework is Pytorch.

Objective evaluation index the average precision (AP) of a single category, the mean average precision (mAP), Frames Per Second (FPS), and error detection rate (EDR) are used to evaluate metrics for model evaluation.

3.2.2. Experimental Data and Data Processing

We did not find any publicly available data sets related to the power maintenance vehicle after reviewing the relevant literature, so this paper uses a homemade dataset approach for the experiments. Firstly, the robotic arms of the power maintenance vehicle are calibrated in categories, and the upper and lower robotic arms are calibrated as arma and armb, respectively, as shown in Figure 15.

Since the rotating target detection algorithm used in this paper refers to the target detection algorithm in the field of remote sensing, the homemade dataset format refers to the annotation format of the remote sensing target detection dataset DOTA, and the RoLableImg annotation software is used to annotate the mechanical arm of the power maintenance vehicle in the dataset, and the annotation process is described in Figure 16.

The labeled results are saved and an .xml file is generated, which contains information about the position of the rotated rectangular box, converting the .xml file into a .txt file in the Dataset for Object Detection (DOTA) dataset.

In this paper, the homemade dataset has a total of 1200 images of curved-arm power maintenance vehicles, and the training set, validation set, and test set are set according to the ratio of 4:1:1. In the process of training a convolutional neural network, if the number of samples in the training set is small, the model obtained from the network training is largely poorly generalized. Therefore, although the sample numbers of category arma and armb in the dataset are basically in equilibrium, in order to enhance the diversity of the dataset and prevent the overfitting problem caused by too little data, we augmented the training and validation sets in the dataset by enlarging, cropping, and adjusting the contrast of the original images, thereby increasing the diversity of the dataset. The numbers of training set and validation set images before enhancement are 800 and 200, respectively, and the numbers of training set and validation set images after enhancement are 2979 and 762, respectively, and the enhanced images are presented in Figure 17.

3.2.3. Experimental Pretreatment

During training, the size of the input image was set to 608 × 608, the training period was 300 epochs, the initial value of the learning rate was 0.001, the optimizer selects Adam, the number of images per batch iteration batch size was setup to 16, the angle loss parameter angle loss was 0.8, angle BCELoss positive weight was set to 1.0. The confidence value threshold was 0.55 for all the inferred images for the experimental algorithm, and the IoU threshold was 0.45 for the NMS operation.

3.2.4. Experimental Comparison

YOLOv5-CSL-CBAM to perform the recognition. The trained network models are used for recognizing arms, and the resulting experimental data results are shown in

Table 1. Horizontal comparison experiment results.

Network Model	AP/%		mAP/%	Parameters/MB	FPS	Perror/%
	arma	armb
R-Faster-RCNN	78.62	79.47	79.18	314.0	8.6	38.0
R-Reppoints	87.70	66.60	75.65	280.0	14.1	74.4
RoI Transformer	81.12	80.76	80.94	421.0	6.2	59.2
R-YOLOv5-based	80.55	79.47	80.01	34.5	33.2	21.2
R-YOLOv7-based	88.78	80.25	84.01	42.5	30.5	12.9
YOLOv5-CSL-CBAM	89.88	80.20	85.04	35.2	32.8	13.6

.

In experiments, the upper and lower vehicle arms were calibrated as arma and armb. Table 1 shows that YOLOv5-CSL-CBAM has higher AP values for target arma and armb, with 89.88% and 80.20%, respectively, its mAP value is higher than R-Faster-RCNN, R-Reppoints, RoI Transformer, R-YOLOv5-Based, and R-YOLOv7-based by 5.86%, 9.39%, 4.10%, 5.03%, and 1.03%. This suggests that YOLOv5-CSL-CBAM has the best recognition performance for vehicle arm. By examining the parameter quantities of each network in Table 1, it becomes clear that YOLOv5-CSL-CBAM’s parameter quantity is 35.2 MB. Compared to R-YOLOv5-Based network, there is a slight increase in the parameter quantities, yet its complexity is lower than R-YOLOv7-based network. The network’s detection accuracy has improved, and the inference speed has reached 32.8 FPS, which is sufficient for real-time detection.

The error detection rate of YOLOv5-CSL-CBAM is 13.6%, whereas the compared networks have an error detection rate of more than 50%. Therefore, based on the above data, it is evident that the proposed YOLOv5 vehicle arm state recognition network can accurately recognize vehicle arms in substations and fulfill the demands of real-time detection.

3.2.5. Ablation Experiments

To further validate the efficiency of our proposed network, we performed ablation experiments to analyze the longitudinal performance. We pruned and modified the model using HardSwish, resulting in R-YOLOv5-HardSwish, employed SIoU loss function to produce R-YOLOv5-SIoU and integrated CBAM attention mechanism to create R-YOLOv5-CBAM.

Table 2. Ablation experiment results.

Network Model	HardSwish	SIoU	CBAM	AP/%		mAP/%
				arma	armb
R-YOLOv5-Based	×	×	×	80.55	79.47	80.01
R-YOLOv5-HardSwish	√	×	×	89.30	79.01	84.16
R-YOLOv5-SIoU	×	√	×	89.50	80.41	84.96
R-YOLOv5-CBAM	×	×	√	89.79	79.98	84.88
YOLOv5-CSL-CBAM	√	√	√	89.88	80.20	85.04

presents the findings of these ablation experiments.

It shows that the R-YOLOv5-HardSwish network improved by 4.15% compared to the original network, indicating that the HardSwish can enhance the network’s nonlinearity. SIoU loss function can lessen the network training loss values and improve network performance, resulting in mAP of R-YOLOv5-SIoU increased by 4.85%. By introducing CBAM into the original network, the AP values of the arms in the R-YOLOv5-CBAM network, respectively, increased by 9.24% and 0.51%, while the mAP increased by 4.87%, indicating that CBAM can effectively extract image feature information and upgrade the network’s feature extraction capability. Compared to the original network, the mAP value of YOLOv5-CSL-CBAM increased by 5.03%. Thus, we can conclude that the YOLOv5-CSL-CBAM network designed in this paper can accurately detect vehicle arms.

3.3. Vehicle Arm Angle Measurement

To further test the recognition accuracy of vehicle arm angle, the RoLableImg annotation software was used to annotate the vehicle arm, and the vehicle arm angles are predicted by different network models. One of the test pictures is shown in Figure 18 and prediction results are presented in

Table 3. Prediction results from the perspective of each model.

Network Model	$\mathit{\theta }$ Predicted Value/o		$\mathit{\theta }$ Prediction Error/o		Average Prediction Error/o
	${\mathit{\theta }}_{\mathit{a}\mathit{r}\mathit{m}\mathit{a}}$	${\mathit{\theta }}_{\mathit{a}\mathit{r}\mathit{m}\mathbf{b}}$	$\mathbf{\Delta }{\mathit{\theta }}_{\mathit{a}\mathit{r}\mathit{m}\mathit{a}}$	$\mathbf{\Delta }{\mathit{\theta }}_{\mathit{a}\mathit{r}\mathit{m}\mathit{b}}$
R-Faster-RCNN	5	28	5	9	7.0
R-Reppoints	12	53	2	16	9.0
RoI Transformer	8	45	2	8	5.0
R-YOLOv5-Based	10	35	0	2	1.0
R-YOLOv7-based	9	36	1	1	1.0
YOLOv5-CSL-CBAM	10	38	0	1	0.5

.

It is starkly reflected in Table 3, where the average error of the vehicle arm angle is 0.5 predicted by YOLOv5-CSL-CBAM. Compared with other networks, the angle prediction error is reduced by 0.5, 0.5, 4.5, 8.5, and 6.5, respectively. These findings demonstrate that the proposed network achieves the highest prediction accuracy.

3.4. Three-Dimensional Twin Implementation of the Vehicle

The vehicle safety operation monitoring and twin system consists of server, cameras, UWB base stations, and tags. In Figure 19, the cameras are applied to obtain images of the vehicle operation. The UWB achieved the location of the vehicle. In the 3D scene, the server completes the real-time presentation of the location and arm state of vehicle.

In the 3D model of the substation, the results of the positioning of the CPSO + TDOA/AOA method are in Figure 20. From the figure, it can be seen that the positioning algorithm designed in this paper achieves the real-time and accurate presentation of the position information of the vehicle. Figure 20a shows the result of the initial position positioning of the power maintenance vehicle, and Figure 20b shows the positioning result map after the vehicle position is changed and updated in real time in the 3D twin system.

In order to verify the reconfiguration of the power maintenance vehicle in the substation 3D model, the updated results of the operation status of the power maintenance vehicle in the substation 3D model in the actual power operation scenario are shown in Figure 21. Figure 21a shows the actual scene diagram of the operation process of the maintenance vehicle, and Figure 21b shows the updated results of the operation status of the maintenance vehicle in the 3D model.

As can be seen from Figure 21, the power maintenance vehicle in the actual operation power operation scene can realize the operation state update in the substation 3D model, and the operation state of the maintenance vehicle in the actual operation power operation scene and the substation 3D model is more matching. Therefore, the algorithm proposed in this paper successfully realizes the real-time twinning of vehicles in the 3D system.

4. Conclusions

This paper introduces a safety monitoring and digital twin scheme for power maintenance vehicles. The scheme employs UWB technology to acquire vehicle position information and machine vision technology to recognize the arm state of the vehicle, then update the status of vehicles in a 3D scene with the acquired information. In the locating algorithm, CPSO was applied to optimize global search for the initial position of the target and eliminate interference problems in the TDOA/AOA algorithm and improve positioning accuracy. CSL, HardSwish, and CBAM models are applied to YOLOv5 network to increase the accuracy of vehicle arm status. In the substation three-dimensional model, the status of virtual vehicle is real-time update and safety monitored.

5. Discussion

Our designed positioning algorithm and robotic arm state recognition algorithm have better positioning effect and recognition effect. However, there are still some shortcomings in the process of monitoring the operation safety of the electric power maintenance vehicle. Although the detection of the mechanical arm state recognition network of the electric power maintenance vehicle is more accurate and the detection speed can meet the requirements of real-time detection, there is still a large space for improving the detection speed. In the future, the network can be considered for light weight processing to further improve the detection speed while maintaining the detection accuracy.