A Novel Monocular Vision Technique for the Detection of Electric Transmission Tower Tilting Trend

Abstract

Transmission lines are primarily deployed overhead, and the transmission tower, acting as the fulcrum, can be affected by the unbalanced force of the wire and extreme weather, resulting in the transmission tower tilt, deformation, or collapse. This can jeopardize the safe operation of the power grid and even cause widespread failures, resulting in significant economic losses. Given the limitations of current tower tilt detection methods, this paper proposes a tower tilt detection and analysis method based on monocular vision images. The monocular camera collects the profile and contour features of the tower, and the tower tilt model is combined to realize the calculation and analysis of the tower tilt. Through this improved monocular visual monitoring method, the perception accuracy of the tower tilt is improved by 7.5%, and the axial eccentricity is accurate to ±2 mm. The method provides real-time reliability and simple operation for detecting tower inclination, significantly reducing staff inspection intensity and ensuring the power system operates safely and efficiently.

1. Introduction

Transmission poles and towers play a crucial role in the safe operation of the power system by supporting overhead power transmission lines. However, due to their widespread distribution, these structures are often constructed in areas prone to landslides, subsidence, riverbeds, and hillsides. Extreme weather can cause transmission tower tilting, deformation, or collapse, leading to significant power grid failures. Over the years, various methods for monitoring the status of towers have been developed. Video inspection of the substation to detect tower inclination has been proposed [1,2,3]. Mridha and Maity proposed a method for the dynamic monitoring of the towering state [4]. In spite of its innovative approach, it lacks real-time analysis functions, and it is difficult to respond effectively to emergencies due to the limited computing power on edge computing servers. Further, using the existing power line corridor, some researchers removed the tower base and used a vertical stratification method and a convex shell algorithm to determine the tower inclination when the point cloud tower is scarce [5]. However, this method focuses on 3D data presentation and does not respond in real time. Ming et al. used an optical fiber sensing technology to monitor the stress deformation and tilting angle of the power tower, but the accuracy was not high [6]. Unmanned Aerial Vehicles (UAVs) equipped with video cameras were also used to detect tower tilting by processing the captured images [7]. Smart antennas have been used to detect the deformation of ultra-high transmission towers [8], where the detected deformation information can be directly used to assess the health status of the towers. In addition to the video camera and smart antennas, a tension and tilt-monitoring device was developed by Hu and Liu to detect the imbalance of the poles and towers [9]. The health status of towers can also be analyzed by installing an unbalanced tension sensor on the insulator [3] or using a PVDF piezoelectric sensor [4]. In addition to installing sensors on power systems, remote sensing technologies such as the Beidou Satellite have been used for online monitoring [9].

Hu and Liu [10] designed a transmission line pole, and tower imbalances and the current state of the tower were analyzed and detected [3]. The health status of a high-voltage transmission tower has also been studied [11]. Additionally, the Beidou satellite has been used to investigate an online monitoring system [9]. While these methods are effective, they face challenges in terms of low accuracy, poor real-time performance, or high cost of operation.

Inspired by previous research using a monocular camera to monitor power lines against tree intrusions, this paper proposes using monocular cameras are used to monitor the tilting status of the towers in this paper. Monocular cameras are inexpensive and, when paired with an edge-computing module to process and transmit the images, can provide online and real-time monitoring. The edge-computing module can also be connected to a data center for further analysis.

Currently, common methods for detecting the posture of towers include periodic manual measurement, unmanned aerial vehicle inspection, and video surveillance [12,13,14]. Although these methods can realize a certain level of monitoring of the transmission channel, they often have limitations, including low accuracy and poor real-time performance.

• Manual measurement requires on-site measurement with instruments, which is cumbersome and requires a lot of workforce and material resources. It is also complex and poses specific security risks due to the geographical conditions of transmission lines. UAV monitoring uses airborne laser radar to build three-dimensional point clouds, which are then projected onto two-dimensional planes and analyzed using correlation algorithms [15]. However, this method has a high cost and does not provide real-time monitoring. Video monitoring has a strong real-time capability and can offer the intuitive perception of external force intrusion and other field conditions. It can timely detect abnormal conditions of transmission channels, such as external force damage. However, it requires manual monitoring and manual analysis for multi-directional distance monitoring of transmission channels. Further, it cannot accurately obtain real-time data and timely collection of spatial status data of conductors [16].
• For distance monitoring of tree barriers, commonly used methods include ranging with an airborne laser scanning device, setting up radar or laser ranging device, processing aerial inspection video, building a tree line model to predict distance, and binocular vision image ranging [17]. These methods improve the efficiency of distance measurement of tree barriers but also have some problems [18]. For example, 3D modeling requires regular updating and reconstruction, and the workload is significant. The tree growth cycle model construction is complex, while the stereo matching of binocular vision ranging is complicated [19]. Moreover, these tree barrier ranging methods have high costs and cannot provide real-time dynamic data of transmission lines.
• The traditional multi-directional distance monitoring method for a transmission channel cannot provide real-time analysis and processing, abnormal alarm, and trend warning [20].

Therefore, this paper proposes a tower tilt analysis method based on a monocular vision image. It builds an edge computing layer linking the back-end cloud service platform and front-end monitoring sensor to address the real-time data problem, uses the tower to share monocular vision data, develops a ranging algorithm for non-fixed objects, selects the profile and features of the collected information, combines it with the pole tower tilt model to calculate and analyze the tilt, and finally determines the attitude of the transmission tower. This method has the advantages of real-time reliability and simple operation in tower tilt detection, significantly reducing the inspection intensity of staff and enabling timely understanding of the tower tilt degree to ensure the safe and stable operation of the transmission lines.

2. Contour Extraction of Transmission Tower

In order to analyze the posture of the tower using the monocular camera, the first step is to identify and extract the tower’s profile, which is divided into two stages. The first stage involves identifying the pole and tower in the image based on the characteristics of monocular vision and extracting the corresponding coordinate information of the tower’s height and the camera’s distance. The RGB images obtained by the camera are converted into grayscale images through image processing and denoised to facilitate accurate contour extraction. The first image obtained after the camera installation is a sample reference image, and subsequent images are used to compare the distance between the monitoring camera and the tower and the changes in the tower’s attitude. In the second stage, the distance between the camera and the tower is calculated using the camera imaging model and the tower imaging model to determine the three-dimensional position of the tower. The analysis process of tower tilt is shown in Figure 1.

2.1. Edge Detection of the Modified Canny Algorithm

The first step in rod-tower tilt monitoring is detecting and extracting rod towers from a complex background. Image segmentation is a technique and process of dividing an image into several specific and unique regions that closely mimic human perception and extract the target of interest. Choosing the appropriate color space is an important problem in color image segmentation. RGB (Red, Green, Blue) and HSV (Hue, Saturation, Value) are the two common color spaces, and the three indicators of HSV space are independent of each other, making it more intuitive for expressing light and shade, color, and bright color degree, and facilitating the contrast between colors. Therefore, this project is based on tower extraction based on HSV color space.

2.2. The RGB Converted to HSV Color Space

The HSV color space is similar to how humans perceive color, encapsulating color information. Intuitively, color (Hue) represents color; saturation (Saturation) represents the purity of color; the higher the saturation, the purer the color, but it becomes gradually grayer; lightness (Value) indicates the brightness of color, and the three together constitute the HSV color space.

Since the captured raw images are in RGB format, they need to be converted from the RGB color space to the HSV color space.

First, the range of the RGB is normalized from 0–255 to 0–1 according to Formula (1):

$${\mathrm{R}}^{\prime } =\frac{\mathrm{R}}{255}, {\mathrm{G}}^{\prime } =\frac{\mathrm{G}}{255}, {\mathrm{B}}^{\prime } =\frac{\mathrm{B}}{255},$$

(1)

R′, G′, and B′ are the normalized RGB color intervals.

To obtain the maximum C_max and minimum C_min from Formula (2) and Formula (3), respectively:

$${\mathrm{C}}_{\max }=\max \left\{ {\mathrm{R}}^{\prime } , {\mathrm{G}}^{\prime } , {\mathrm{B}}^{\prime } \right\},$$

(2)

$${\mathrm{C}}_{\min }=\min \left\{ {\mathrm{R}}^{\prime } , {\mathrm{G}}^{\prime } , {\mathrm{B}}^{\prime } \right\},$$

(3)

Calculate the maximum difference between the three according to Formula (4):

$$\Delta ={\mathrm{C}}_{\max }-{\mathrm{C}}_{\min },$$

(4)

Finally, the values of hue H, saturation S, and value V are calculated respectively according to Formulas (5)–(7):

$$\mathrm{H}=\{\begin{array}{cc}0& \Delta =0\\ 30\times \left(\frac{ {\mathrm{G}}^{\prime } - {\mathrm{B}}^{\prime } }{\Delta }\right)& { \mathrm{C}}_{\max }= {\mathrm{R}}^{\prime } \\ 30\times \left(\frac{ {\mathrm{B}}^{\prime } - {\mathrm{R}}^{\prime } }{\Delta }+2\right)& { \mathrm{C}}_{\max }= {\mathrm{G}}^{\prime } \\ 30\times \left(\frac{ {\mathrm{R}}^{\prime } - {\mathrm{G}}^{\prime } }{\Delta }+4\right)& { \mathrm{C}}_{\max }= {\mathrm{B}}^{\prime } \end{array}$$

(5)

$$\mathrm{S}=\{\begin{array}{cc}0& { \mathrm{C}}_{\max }=0\\ \frac{\Delta }{{\mathrm{C}}_{\max }}\times 255& {\mathrm{C}}_{\max }\ne 0\end{array}$$

(6)

$$\mathrm{V}={\mathrm{C}}_{\max }\times 255$$

(7)

In this paper, HSV images were extracted from the towers, and the results of RGB images converted to HSV images are shown in Figure 2.

In order to improve the effectiveness of extracting the pole and tower, the HSV image enhancement algorithm is used to enhance the converted image. The enhancement results are shown in Figure 3.

2.3. Tower Contour Extraction

In this paper, the value range of this top red is determined by taking the histogram of the components as the boundary standard for our segmentation. The histogram of HSV images is shown in Figure 4.

According to the HSV color space model, a large amount of data appears in the range of 90 to 110 on the histogram, so the upper and lower limits of the component are 90–110.3.

2.4. Construction and algorithm

The upper and lower limit of the red H component can be determined to be 170–180. As for the SV limit, an initial value of 30–255 can be given, which can be adjusted to determine the mask when it is extracted. The mask is generated by the red bound of HSV, and the post-treatment results are shown in Figure 5.

After determining the spatial range of HSV color, the subject changes the saturation and brightness by adjusting the HSV slide bar, as shown in Figure 6.

3. Establishment of Tower Tilt Calculation Model

3.1. Model Building

3.1.1. Camera Imaging Model

Usually, the camera uses a lens to focus the light reflected from the object’s surface onto the photosensitive plane of the camera [20]. This project analyzed the tower tilt by the camera and used monitoring devices to verify, as shown in Figure 7. It can be visualized how a point in the real world is projected onto the camera’s light screen.

This project simplifies the model by calculating the one-dimensional projection size, as shown in Figure 7b, and extending the results to a two-dimensional (2D) space. The similar relationship in Figure 7 can be simplified as Formula (8):

$$\Delta \mathrm{CAB}\cong \Delta \mathrm{CDE}\to \frac{\mathrm{DE}}{\mathrm{CE}}=\frac{\mathrm{AB}}{\mathrm{CB}}\to \frac{\mathrm{Poy}}{\mathrm{CE}}=\frac{\mathrm{Oy}}{\mathrm{CB}}$$

(8)

In the equation, O_y represents the object’s height, P_oy represents the pixel height where the object is imaged, CB represents the object distance, and CE represents the image distance. In this way, 3D coordinate points in space can be converted to 2D coordinates under the camera coordinate system, allowing for subsequent distance calculations to be performed using 2D coordinates.

3.1.2. Pole-Tower Imaging Model

The rod-tower model structure is shown in Figure 7c, which includes two groups of similar triangles existing in the imaging model of the tower:

$$\Delta \mathrm{OAB}\cong \Delta \mathrm{OCD},\hspace{1em}\hspace{1em}\Delta \mathrm{OAS}\cong \Delta \mathrm{OCI},$$

(9)

Based on the ratio of the three sides of a similar triangle, Formulas (9)–(11) are obtained:

$$\frac{\mathrm{AB}}{\mathrm{CD}}=\frac{\mathrm{OS}}{\mathrm{OI}}\to \frac{\mathrm{Py}}{\mathrm{CD}}=\frac{\mathrm{Pz}}{\mathrm{OI}}$$

(10)

$$\frac{\mathrm{AS}}{\mathrm{CI}}=\frac{\mathrm{OS}}{\mathrm{OI}}\to \frac{\left|\mathrm{Py}-\mathrm{Cy}\right|}{\mathrm{CI}}=\frac{\mathrm{PZ}}{\mathrm{OI}}$$

(11)

Among them, Py represents the known vertical height of the tower, Pz represents the known distance from the tower to the camera, Cy represents the known camera installation height, OI represents the image distance, and CD and CI are the pixel height of the tower Py and the height of the projection of the image at the top of the pixel height.

3.1.3. Tower Overlooking Imaging Model

In order to measure the horizontal distance (PM) between the camera and the pole tower, a pole-tower overlooking model is established to observe the simulated scene. The simulated scene and lens imaging relationship are shown in Figure 8:

Top view of the tree-line scene and the imaging relationship

$$\Delta \mathrm{OPQ}\cong \Delta \mathrm{OUV},\hspace{1em}\hspace{1em}\Delta \mathrm{ONE}\cong \Delta \mathrm{OIG}$$

(12)

Based on the scale of the three sides of a similar triangle in Formulas (12) and (13):

$$\frac{\mathrm{PQ}}{\mathrm{UV}}=\frac{\mathrm{OM}}{\mathrm{OI}}\to \frac{\mathrm{PQ}}{\mathrm{UV}}=\frac{\mathrm{Pz}}{\mathrm{OI}}$$

(13)

where, PQ and UV represent the known horizontal component on the tower and its horizontal pixel distance.

3.2. Establishment and Analysis Model of the Distance from the Center of the Tower

Once the tower tilt monitoring model and algorithm have been established, it is necessary to find out the tilt distance of the tower pixels in every location. The specific process is as follows:

(1)

According to the standard “Operation Regulations of Overhead territorial Lines (DS/CLC/TR 50412-1)”, set the tilt range of the pole and tower, and divide the dangerous image area;

(2)

Convert the pole and tower images to grayscale and binary image transformation;

(3)

Define the upper left corner of the binary image as the coordinate origin (0, 0), go through the fixed interval from left to right and from top to bottom, determine the pixel coordinates of the tower, and judge the top and bottom positions of the tower according to the y-axis pixel coordinates:

(4)

Perverse the outline of the tower, take any point on it as the high point and the bottom of the tower as the lowest point, and calculate the tower outline’s pixel height at any point from the bottom;

(5)

Calculate the actual height of the outline of the tower, the distance between the camera and the tower, and the distance from the known horizontal parts;

(6)

Compare the desired distance with the set distance threshold, determine the danger point, and mark it.

The tower traversal process is shown in Figure 9.

4. Experimental Results

Due to the systematic error and random error in the calculation of distance, systematic error is inevitable, while random error is caused by external environmental factors. In order to ensure the accuracy of the algorithm; this paper opted for an indoor experimental platform and conducted an algorithm stability test. The stability of the algorithm was tested by using the selected algorithm to process images of the pole tower.

4.1. Subjects and Apparatus

The indoor transmission line test environment, shown in Figure 10a, was built by simulating external transmission lines using a small tower, wire, and tower tilt-monitoring terminal.

According to the actual measurement, the camera installation height was 2.82 m, the tower height was 3.40 m, the tower width was 1.4 m, and the distance between the two towers was 16.22 m. During the test, the photo interval was set, and the tower was given a tilt angle after each shot. The pictures taken are shown in Figure 10b. The tree was used as a reference for the calibration of the pictures. The distance between the camera and the tower and the axial eccentricity distance between the camera and the tower components were calculated using the established tower tilt state model and algorithm.

The detailed steps are as follows:

(1)

Obtain the first picture and use it as the reference image to find the vertical distance between the camera and the tower (P1) and the horizontal distance between the camera and the tower components (P2) using a laser rangefinder.

(2)

Process the captured image, calibrate the pixels, and find the pixel distance. The processed pictures are shown in Figure 11. After traversing, the coordinates obtained for the top of the tower and the base point are (277, 309) and (240, 590), respectively. Thus, the pixel height of the tower can be found according to the formula.

(3)

Obtain the camera installation and tower heights and find the vertical distance between the camera and the pole and tower (PZ) using the distance combined with the pole and tower imaging model obtained in Step 2.

(4)

Find the distance between the camera and the horizontal component (PQ) using OI;

(5)

Compare the desired distance PZ and PQ with P1 and P2 and find the error rate.

The position information of tower key points and distance results are shown in Figure 12.

4.2. Results

According to the actual conditions of the experimental environment, the improved monocular imaging method was used to accurately perceive the tower’s profile after HSV processing. The Axial eccentric distance of the tower was obtained after 11 times of image capture and real-time analysis and calculation, as shown in

Table 1. Indoor test camera detection results.

Order Number	The Camera Is Far away from the Pole and Tower Pz (m)	Range Error (%)	Axial Eccentric Distance (cm)	Range Error (%)
1	14.8	1.33	1.6	2
2	14.6	2.67	1.7	1.5
3	14.8	1.33	1.8	1
4	14.5	3.33	1.7	1.5
5	14.6	2.67	1.5	2.5
6	14.7	2	1.6	2
7	14.6	2.67	1.8	1
8	14.5	3.34	1.7	1.5
9	14.6	2.67	1.7	1.5
10	14.7	2	1.8	1
11	14.8	1.33	1.5	2.5

.

The results from Table 1 indicate that the distance perception model combined with monocular imaging technology exhibits some errors in the distance calculation of fixed objects. The error range is approximately ±0.2 m within the perception range of around 15 m. Through the improved monocular visual monitoring method presented in this paper, the perception accuracy of the tower tilt is improved by 7.5%, and the eccentric axial distance, a critical indicator of the stability and attitude prediction of the tower, is accurate to ±2 mm”.

4.3. Field Test

In order to verify the tower tilt model and the algorithm’s accuracy, several field tests were conducted, and the model was corrected based on the results of these tests. The field test setup is shown in Figure 13.

Testing procedure:

(1)

Collect the parameters such as gear pitch, tower height, and tower width;

(2)

Measure the field data through the rangefinder;

(3)

Calculate the distance between the tower and the camera and the distance between its own horizontal components according to the tower tilt model and algorithm, combined with the parameters of the tower;

(4)

Compare the calculated data with the measured data, and correct the distance based on the comparison results.

Pictures were taken on the scene, and the marking result is shown in Figure 14a:

The test results are illustrated in Figure 14b, and the calculation results are given in

Table 2. Distance measurements results.

Measuring Position	Measured Value (m)	Actual Value (m)	Error (%)
The camera is far away from the pole and tower (Pz)	189	190	0.5%
Distance of the horizontal parts of the tower (PQ)	22.5	23.0	2.2%
Pole tower height (Py)	234.0	235.0	0.4%

.

Each node of the system monitoring device is according to the following method: the tower tilt monitoring camera is fixed the opposite of the test tower, determines the vertical line, cross two-level determine the benchmark plane, test tower movement in the process of shaking, which can simulate the wind caused by tower slight shaking, and when the inclination is too large can think the tower has collapsed or damaged, then lost the significance of online measurement. The tilt test is conducted for 0–60° in the XZ plane, and a measurement point is selected every 5°. The data of the tilt angle output by different units are shown in

Table 3. Comparison of tilt angle data under different filtering coefficients.

Actual Tilt Angle/°	Angular Mean Calculated from Acceleration/°	A = 0.25	Relative Accuracy/%	A = 0.5	Relative Accuracy/%	A = 0.75	Relative Accuracy/%
0	0.13	0.19		0.20		0.33
5	4.88	4.93	0.75%	4.85	2.2%	5.02	2.35%
10	10.13	9.32	1.06%	9.95	0.26%	10.14	2.11%
15	14.61	15.25	0.32%	14.52	0.76%	15.26	1.12%
20	20.32	20.21	0.11%	20.08	0.12%	20.23	0.12%
25	25.02	24.96	0.06%	24.93	0.08%	25.07	0.04%
30	29.96	30.11	0.14%	30.04	0.05%	30.02	0.01%
35	35.12	35.07	0.09%	35.03	0.01%	35.05	0.02%
40	40.24	40.13	0.08%	40.26	0.18%	40.03	0.01%
45	45.35	45.19	0.13%	45.21	0.13%	45.14	0.12%
50	50.25	50.15	0.13%	50.12	0.19%	50.17	0.11%
55	55.06	55.03	0.01%	55.15	0.23%	55.12	0.09%
60	59.91	60.02	0.01%	60.14	0.20%	60.06	0.02%

.

As shown in Table 3, the angle error calculated directly according to the unprocessed camera is significant. In contrast, the angle calculated by the complementary filtering method camera image is reduced to a certain extent under different coefficients. When the inclination angle measured by the camera occupies a higher weight, the output angle of the complementary filter is closer. According to the analysis, the reason should be that the acceleration on the rod is mainly gravity acceleration, and the acceleration of the bar can be suppressed after preprocessing from the perspective of relative accuracy. When the weight coefficient A = 0.25, the relative accuracy of the measured data can be controlled within 1.40%, which can fully meet the accuracy requirements of the tower online monitoring.

The inclination perception end is placed in the standard using an electronic level instrument to verify the error analysis further to measure a precise uniaxial rotary platform. One sampling point is taken every 20° during the rotary shaft rotation, and the obtained error is statistically analyzed after measurement and data processing. The measurement results and correction results are shown in

Table 4. Measurement and correction results.

Position	Measurements in the x-Axis Direction	Measurements in the y-Axis Direction	Measurements in the z-Axis Direction	Synthesis Error	Error Correction Results
					x-Axis Direction Error	y-Axis Direction Error	z-Axis Direction Error	Synthesis Error after Correction
0	26.50	42.45	0.00	0.344	0.025	−0.016	0.035	0.043
20	49.42	55.65	0.04	−0.404	−0.065	−0.039	−0.075	−0.179
40	42.04	−56.91	0.03	−0.213	−0.092	0.001	−0.009	−0.100
60	−39.47	55.12	−0.03	−0.347	0.024	0.061	0.064	0.149
80	10.17	41.46	−0.04	−0.232	−0.009	0.051	−0.034	0.008
100	47.17	−25.63	0.03	−0.343	0.072	−0.063	0.016	0.026
120	23.31	−46.86	−0.06	−0.308	−0.060	−0.036	−0.024	−0.120
140	−48.87	23.59	−0.03	0.022	0.018	0.072	0.026	0.116
160	−9.06	49.00	−0.07	−0.371	0.031	0.049	0.093	0.173
180	−34.47	3.39	−0.09	−0.331	0.045	0.042	0.080	0.167
200	38.60	−15.25	−0.09	0.247	−0.098	−0.051	0.056	−0.093
220	41.16	−54.55	0.03	−0.236	0.085	0.012	−0.010	0.087
240	22.61	2.53	−0.08	−0.146	0.064	−0.065	0.080	0.079
260	45.58	−41.30	0.05	0.113	−0.084	−0.049	−0.026	−0.159
280	−58.62	40.45	0.02	−0.313	−0.092	0.049	−0.013	−0.056
300	−32.48	−45.63	0.05	0.400	0.028	0.014	−0.002	0.041
320	−1.69	−22.75	0.10	0.268	−0.068	0.069	−0.026	−0.024
340	−24.19	43.16	−0.03	0.177	−0.080	−0.066	−0.053	−0.199
Average value	5.43	2.66	−0.01	−0.093	−0.01	0.00	0.01	−0.002

.

5. Conclusions

Existing tower tilt monitoring methods include the lead hammer method, theodolite, and manual inspection, but poles and towers are widely distributed, and several are deployed in inaccessible locations. In addition, transmission pole and tower accidents often occur during natural disasters, making these methods ineffective for monitoring the transmission poles and towers. Given the poor real-time performance of existing monitoring methods of tower tilt, this paper has proposed a monitoring method of tower tilt based on a monocular vision image. A mathematical model is developed by analyzing and calculating the collected information, including image processing and contour extraction. Using the improved monocular vision monitoring method in this paper, the perception accuracy of the tower tilt was improved by 7.5% by establishing the optimal model and experimentally testing its attitude in the laboratory. This is an essential indicator of tower stability and attitude, which can be predicted to a precision of ±2 mm with the axial eccentricity distance. A major advantage of this method is that it is real-time, simple to operate, and has a short response time for detecting tower inclination. It reduces the inspection intensity of the staff significantly, and it also makes it possible to gauge the tilting degree of the tower in time, which ensures a safe and stable transmission line operation.