Pointer Meter Recognition Method Based on Yolov7 and Hough Transform

Abstract

The current manual reading of substation pointer meters wastes human resources, and existing algorithms have limitations in accuracy and robustness for detecting various pointer meters. This paper proposes a method for recognizing pointer meters based on Yolov7 and Hough transform to improve their automatic readability. The proposed method consists of three main contributions: (1) Using Yolov7 object detection technology, which is the latest Yolo technology, to enhance instrument recognition accuracy. (2) Providing a formula for calculating the angle of a square pointer meter after Hough transformation. (3) Applying OCR recognition to the instrument dial to obtain the model and scale value. This information helps differentiate between meter models and determine the measuring range. Test results demonstrate that the proposed algorithm achieves high accuracy and robustness in detecting different types and ranges of instruments. The map of the Yolov7 model on the instrument dataset is as high as 99.8%. Additionally, the accuracy of pointer readings obtained using this method exceeds 95%, indicating promising applications for a wide range of scenarios.

1. Introduction

The substation is in a very important position in the power system, and its main task is to convert and distribute electric energy. The normal operation and safety maintenance of substations are issues that need to be paid close attention to in daily inspections. The readings of the meters in the substation can reflect the operation of many pieces of equipment. However, due to the complex external environment of the meter, the large number, and the dependence of the original equipment of the substation on the meter, traditional substation inspection cannot effectively guarantee the real-time validity of the meter reading [1].

The main method of traditional substation inspection is manual inspection and recording. The operator manually judges to read the pointer reading, which requires the operator to have a high level of technology and experience, so it is inevitably interfered with by many human factors, such as the operation angle of the operator observing the dial, the error caused by the observation distance, the long and boring recognition process, and visual fatigue are easily affected by subjective factors, resulting in low work efficiency and inaccurate and wrong recognition results [2]. Traditional substation manual inspection requires personnel to inspect each meter one by one and record data manually. This method requires a lot of time and labor costs, especially in large substations where the inspection process is more cumbersome and time-consuming. The substation environment is complex and there are certain safety risks, such as high-voltage electricity and explosions. During manual inspections, staff need to frequently contact high-voltage equipment, which poses certain safety risks and may cause casualties. Manual inspection can perform only regular inspections on the meters and cannot realize real-time monitoring of the meters, so it is difficult to quickly find and deal with abnormalities in the meters.

There are many meters in a substation, such as an oil temperature meter, pressure meter, ammeter, and so on. There are two common types of gauges: digital and analog. Figure 1 shows the digital and analog meters. These two types of meters have different characteristics. In the meter industry, the share of digital meters is increasing, because they have the advantages of easy reading and high precision; however, digital meters are not suitable in some special circumstances, such as harsh dust and oil environments. Compared to the digital meter, the pointer meter has greater advantages in this respect, such as being dustproof and waterproof, and having antifreeze, anti-interference, simple structure, low price, strong anti-interference ability, etc. [2]. Pointer meters are widely used in industrial production, automobile manufacturing, electric power industry, and other fields. They are mainly used to measure and display various physical quantities, such as current, voltage, temperature, and pressure, etc. Although digital meters are gradually replacing analog meters, there are still many analog meters in operation in remote areas and developing countries because of the high cost and time-consuming nature of replacing the meters [3].

However, there are still some challenges in pointer meter recognition technology. The pointer shapes and colors of pointer meters are very different, which makes it difficult to accurately classify and locate all pointers in image recognition and processing. Analog gauges are typically used in a variety of lighting conditions, such that changes in ambient lighting may affect the recognition of the pointer’s position. In addition, background noise, such as reflections, shadows, etc., will also affect pointer recognition, and these factors affect the recognition accuracy of meter readings. Analog gauges usually need to be photographed or scanned to obtain their image. However, if the image quality is poor, blurry, or low in pixel count, it can also become difficult to identify the pointer position. When the meter reading changes, it is necessary to identify and upload the data in real time, so the response speed of the identification system also needs to be further improved. Therefore, in future research, it is necessary to continuously improve and optimize the pointer meter recognition technology to improve its application effect.

Overall, the contributions of the work in this paper are as follows:

• Yolov7 object detection technology is employed, which is the latest Yolo technology, to accurately and quickly locate instruments in complex backgrounds and enhance instrument recognition accuracy.
• Relatively little research has been conducted on square pointer gauges to provide a formula for pointer readings. This formula fills the research gap in the field of square pointer instruments and provides an effective method for accurately reading the value indicated by the pointer.
• For instruments of different models and ranges, the PGNet method is used to identify scale values and models, which exhibits high robustness.

By combining these technologies and methods, this approach achieves accurate, fast, and robust detection and localization in pointer-type instrument recognition, suitable for various complex scenarios and different types of instruments. A detailed description of the specific process will be provided in the third part.

2. Related Work

At present, a lot of research has been conducted on the automatic identification of pointer meters. The technology of pointer meter recognition can be divided into three types: technology based on template matching, technology based on OpenCV, and technology based on deep learning. (1) Techniques based on template matching [4]. This technology is a computer vision technology based on statistics, and its basic idea is to compare the image of the pointer meter with the established template image, so as to determine the value of the pointer meter. (2) Technology based on OpenCV [5]. The technology uses the OpenCV algorithm to process the image of the pointer meter to recognize the value on the pointer meter. (3) Techniques based on deep learning [6]. This technology can use the deep neural network to automatically extract the characteristics of the pointer of the meter, so as to identify the value on the pointer meter more accurately. In terms pointer meter recognition technology, there are many mature methods, such as deep learning, convolutional neural network (CNN) [7], recurrent neural network (RNN), and so on. These techniques have all made great progress in improving the accuracy and robustness of pointer gauge recognition.

Liu et al. [8] first used Faster R-CNN to determine the position of the object meter, then used the feature correspondence algorithm and perspective transformation to obtain high-quality images, and finally determined the pointer position and read the readings through Hough transform. Wu et al. [9] proposed a meter image skew correction algorithm based on binary mask and improved Mask-RCNN, which achieved high-precision ellipse fitting. Li et al. [10] introduced deep learning to detect and recognize the scale value text in the dial, then rectified the image and determined the center of the meter, and used polar coordinate transformation to convert the arc scale area into a linear scale area to obtain the scale marks and pointers The position of the pointer was read by the distance method. Liu et al. [11] used a Gaussian scale space to enhance the scale invariance of the ORB algorithm. At the same time, the RANSAC was used to filter matching point pairs, which improves the accuracy of feature pointer matching. Finally, the pointer is fitted by Hough transformation and the meter reading is obtained. Cai et al. [12] proposed a new virtual sample generation technique to generate a large number of images from a small number of meter images to add to the training model. Zuo et al. [13] constructed a new deep learning algorithm, replacing RoiAlign in the existing mask RCNN with PrRoIPooling, classifying the meter types when fitting the pointer binary mask, and finally calculating the meter by the angle method readings. Wang et al. [2] used the Faster RCNN object detection method to locate the meter area and used the Poisson fusion method and the K-fold verification algorithm to expand the dataset and optimize the dataset quality. Finally, the pointer position was detected by Hough transform and the pointer reading was obtained. Laroca et al. [14] devised a two-stage approach to counter detection using the fast YOLO object detector and evaluated three different CNN-based methods for counter recognition. Salomon et al. [3] sorted out the UFPR-ADMR public real-world dialing table dataset, proposed a recognition baseline based on deep learning, and proposed the main problems and detailed error analysis in the process of recognizing the meter. Salomon et al. [15] introduced a new dataset UFPRADMR-v2 for ADMR based on [3], combining YOLOv4 with a new regression method (AngReg). Bishwokarma et al. [16] used a deep neural network YOLOv3 to detect and recognize meter counters and digits. Zhang et al. [1] proposed a digital meter method based on the connected domain analysis algorithm, using Faster R-CNN to locate the dial area, and then using YOLOv4 to detect the digital area. Combined with the connected area algorithm, it judged whether there was a decimal point after the number. Yan et al. [17] used Gaussian filtering and binarization methods to identify pointer circular meters. Meng et al. [18] obtained the key point information of the pointer meter through the deep learning algorithm, determined the pointer rotation center and radius according to the key point information, and finally obtained the meter reading according to the pointer angle and the meter range. Zhang et al. [19] proposed a pointer meter recognition algorithm combining meter detection and localization, deep learning and image processing techniques. The algorithm used the Faster R-CNN algorithm to classify three types of meters (voltmeter, ammeter, and digital meter), followed by image processing for more accurate needle readings. Laroca et al. [20] introduced a new stage called corner detection and counter classification, which corrects meter regions and rejects illegible or wrong meters before the recognition stage. Dong et al. [21] regarded the pointer as a two-dimensional vector whose initial point coincides with the end of the pointer and whose direction is along the direction from the end to the top. Bayhan et al. [22] used Faster-RCNN and YoloV4 models to train military action images taken by drones. The results showed that the accuracy of Faster-RCNN reached 93%, while the accuracy of YoloV4 was 88%. Ozkan et al. [23] used Faster-RCNN and SSD MobileNet V2 models to train military action images taken by drones and transplanted the trained models to the Raspberry Pi 4 Model B electronic board. The results show that the accuracy rate of the Faster-RCNN model reached 91%, while the accuracy rate of the SSD MobileNet V2 model was 88%.

The basic principle of pointer meter recognition technology is to analyze and recognize the image of the pointer meter through image processing and machine learning technology, so as to read the meter data. The key steps of this technology include image preprocessing, feature extraction, pointer positioning, pointer recognition, and data reading. By analyzing the characteristics of the pointer in the image, such as pointer length, color, shape, etc., the position and direction of the pointer in the image are determined, and finally the meter reading is obtained by identifying the scale. This technology can be applied to different types of pointer meters, such as gauges, gas meters, etc.

The following is the structure of this article, Section 2 introduces the related work of pointer meter recognition; Section 3 introduces the recognition algorithm of pointer meters, and introduces the related technologies used, including the principles of Yolov7, DeepLabv3+, PGNet, Thining, Hough transform, and reading; finally, Section 4 provides a detailed analysis of a large number of experiments.

3. Pointer Meter Recognition Method

This paper proposes a pointer meter recognition method based on Yolov7 and Hough transform. First, use Yolov7 for object detection to locate the position of the meter and crop the image: input the cropped image into the DeepLabv3+ image segmentation model to extract the pointer area, use Thinning and Hough transform to determine the precise position of the pointer, at the same time, use PGNet in the OCR (Optical Character Recognition) method to identify the scale value and model of the pointer meter, and use the maximum value of the obtained scale value as the range of the pointer; finally, use the precise position and span of the pointer to determine the reading of the meter. The flow chart of the pointer meter identification method is shown in Figure 2. The following will introduce Yolov7, DeepLabv3+, Thining, Hough transform, PGNet, and reading.

3.1. Yolov7

Object detection is an important task in computer vision, it refers to the identification of an object of interest from an image or video and determining its location. Some common object detection algorithms are Faster R-CNN [24], Single Shot Multi-Box Detector (SSD) [25], and YOLO [26].

Yolov7, the single-stage object detector, stands out as the swiftest and most precise real-time object detection system. The performance improvement of Yolov7 establishes a new and important benchmark [27].

The YOLO architecture is based on FCNN. The YOLO framework consists of Backbone, Neck and Head. Backbone is responsible for extracting image features, Neck is responsible for multi-scale feature fusion, and Head is responsible for target location and category prediction. The overall architecture of the Yolov7 model is shown in Figure 3.

First, preprocess the input image, align it into a 640 × 640 RGB image, and input it into the backbone network. According to the three-layer output in the backbone network, the head layer continues to output three layers of feature maps of different sizes through the backbone network. After RepVGG block and conv, the three types of tasks of image detection (classification, background classification, border) are predicted, and the final result is output [28].

3.2. DeepLabv3+

Image segmentation is an important task in the field of image processing, it refers to separating different parts of an image, identifying their boundaries, and assigning them to different categories or regions. The purpose of image segmentation is to enable the image processing system to identify different objects in the image for further analysis and processing. Figure 4 illustrates the overall architecture of the DeepLabv3+ model. The primary component of its Decoder is a DCNN that incorporates dilated convolutions. This DCNN has the capability to utilize commonly employed classification networks like ResNet. Additionally, the Decoder incorporates a spatial pyramid pooling module, known as Atrous Spatial Pyramid Pooling (ASPP), which also employs dilated convolutions and pooling operations. The ASPP module primarily aims to introduce multi-scale information into the model.

In comparison to DeepLabv3, DeepLabv3+ introduces the Decoder module, which further integrates both the underlying features and high-level features to enhance the accuracy of segmentation boundaries. This integration is achieved by leveraging the concept of an Encoder-Decoder architecture, building upon the foundation of DilatedFCN. Essentially, DeepLabv3+ incorporates the idea of an Encoder-Decoder to refine the segmentation process [29].

3.3. Thinning

The Thinning algorithm in OpenCV is a morphology-based image-processing algorithm for extracting skeletons or edges in binary images. The basic principle of the thinning algorithm is to gradually reduce the area of the object region through iteration until it reaches the final skeleton or edge. In each iteration, the algorithm performs Erosion on the object area and calculates the difference between the eroded result and the original area. If the difference is 0, it means that the region has reached the skeleton or edge; otherwise, the edge pixels in the difference value need to be removed, and then the erosion operation is performed again until the final result is reached. The refinement flow chart is shown in Figure 5. Common thinning algorithms include the Zhang-Suen algorithm, Guo-Hall algorithm, etc., which use different conditions and rules to achieve thinning operations.

3.4. Hough Transform

Hough transform is a classic image processing algorithm based on parameter space, it is used to detect various shapes in images, including straight lines, circles, ellipses, etc. Among them, line detection is one of the most commonly used applications of Hough transform.

The straight-line detection algorithm of Hough transform can be divided into the following steps: 1. Edge detection, for example canny edge detection. 2. Map the edge point to the Hough space and store it in the accumulator. 3. Find the most reasonable point in the accumulator through the threshold and other possible restrictions (for example, find the point with the largest value in the accumulator or the point greater than the threshold), and generate a straight line of infinite length. 4. Convert a straight line of infinite length to finite length and overlap it on the original image.

Hough transform can be well-applied to the detection of straight lines in the image, but due to the large amount of calculation and its sensitivity to noise, certain optimization and preprocessing are required. In practical applications, algorithm parameters can be adjusted according to specific scenarios and need to achieve the best detection effect. The flow chart of Hough transform is shown in Figure 6.

3.5. PGNet

Optical Character Recognition (OCR) is the process of converting scanned or digital images of text into editable and searchable text data. The technology behind OCR enables computers to recognize and process written or printed text, making it possible to extract, store, and manipulate textual information from a variety of sources.

In recent years, end-to-end OCR algorithms have made good progress, including the MaskTextSpotter series, TextSnake, TextDragon, PGNet series, etc. Among these algorithms, PGNet has features that other algorithms do not have: PGNet loss is designed to guide training: no character-level annotation is required, NMS and ROI-related operations are avoided, and the prediction speed is improved. A module for predicting the reading order within a text line Is proposed. A graph-based rectification module (GRM) is introduced to further improve the recognition performance. It makes the recognition accuracy higher and the prediction speed faster [30]. The schematic diagram of the PGNet algorithm is shown in Figure 7.

The input image will enter four different branches through feature extraction: the TBO (Text Edge Offset Prediction) module, the TCL (Text Centerline Prediction) module, the TDO (Text Orientation Offset Prediction) module, and the TCC (Text Character Classification) graph prediction) module. The output of TBO and TCL can get the text detection result after subsequent processing, while the TCL, TDO, and TCC are responsible for recognizing the text.

3.6. Reading

The following is the calculation formula for the reading: k represents the slope of the straight line, where k ≥ 0. r is the radian value between the straight line where the pointer is located and the horizontal line, with the constraint 0 ≤ r ≤ Π/2. d is the angle value between the straight line where the pointer is located and the horizontal line, with the constraint 0 ≤ d ≤ 90. Formula (3) is used to convert the radian value, r, from Formula (2) to an angle value. By using OCR, identify the model and scale value of the meter. Take the maximum scale value on the meter as the range of the meter and use this value as ‘max’ in Formula (4). ‘num’ represents the reading of the final pointer, and in Formula (4), 90 represents the rotatable range of the pointer, which is 90 degrees.

$$\mathrm{k}=\frac{{\mathrm{y}}_2-{\mathrm{y}}_1}{{\mathrm{x}}_2-{\mathrm{x}}_1},$$

(1)

$$\mathrm{r}=\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}\left(\mathrm{k}\right),$$

(2)

$$\mathrm{d}=\frac{\mathrm{r}}{\mathsf{\pi }}\times 180,$$

(3)

$$\mathrm{n}\mathrm{u}\mathrm{m}=\frac{\mathrm{m}\mathrm{a}\mathrm{x}\times \mathrm{d}}{90},$$

(4)

3.7. Comparison with Existing Technology

Previous studies have used a number of different methods and techniques to address the recognition and reading extraction problems of square pointer meters. These methods are shown in

Table 1. Methods used in previous studies.

Previous Research	Type	Positioning Instrumentation	Identification Pointers and Readings
Wang et al. [2]	Round pointer instrument	Faster-RCNN	Hough transform
Salomon et al. [3]	water meter	Faster-RCNN, Yolov3	Classification
Liu et al. [8]	Round pointer instrument	Faster R-CNN	Hough transform
Liu et al. [11]	Round pointer instrument	ORB algorithm	Hough transform
Zuo et al. [13]	Round pointer instrument	Mask RCNN	angle method
Meng et al. [18]	Round pointer instrument	/	key point detection

.

In past research, scholars mainly focused on exploring circular pointer meters and water meters, but research on the square pointer meters mentioned in this paper is relatively limited. There are obvious differences in shape and structure between the square pointer meter and the traditional circular pointer meter, so it is of great significance to study it deeply.

When locating instruments, commonly used technologies include Faster-RCNN, Yolov3, ORB algorithm, and Mask RCNN. This paper uses the newly proposed Yolov7 model, which significantly improves the accuracy of target detection.

In terms of identifying pointers and readings, previous studies such as Wang et al. [2], Liu et al. [8], and Liu et al. [11] directly used Hough transform to extract pointers, but this method is easily affected by complex backgrounds. However, this paper adopts the method of segmenting the pointer area by using image segmentation technology first, and then performing Hough transform, which greatly improves the robustness of instrument recognition and reduces the interference to complex backgrounds. Since the water meter has only 10 different states, Salomon et al. [3] use a classification method to distinguish different readings. However, this method is not suitable for complex ammeters or voltmeters. Conversely, Zuo et al. [13] only used the angle method to determine the reading of the pointer instrument, but in fact the pointer has a width, which may lead to errors in the reading. For this reason, this paper adopts the method of thinning first and then angle method, which can effectively extract the central axis of the pointer and improve the accuracy of reading. Meng et al. [18] used the key point detection method to determine the pointed center point and pointer endpoint of the meter, but due to the long-term use of the meter, the position of the key point may be blocked by stains, which seriously affects the accurate recognition rate of the meter center and pointer endpoint.

In addition, this paper also proposes the use of OCR technology to identify the instrument model and scale value, and take the maximum value of the scale value as the range of the instrument, which greatly improves the identification robustness in instruments of different models and ranges.

This paper proposes a series of new methods for square pointer instruments, including using the Yolov7 model for target detection, combining image segmentation and Hough transform, extracting pointer readings by thinning first and then angle method, and using OCR technology to identify instruments’ model and range. The application of these methods greatly enhances the accuracy and robustness of instrument recognition, making up for the shortcomings of previous studies.

4. Experiment

4.1. Experimental Environment

The experimental platform is the Ubuntu20.04 operating system, and the software used is CUDA11.6 and CUDNN8401. The experiment is performed under the deep learning framework based on Pytorch1.12.1 + cu116 and paddle2.3.2 [31]. The computer configuration is shown in

Table 2. Experimental hardware configuration.

Name	Type
CPU	i7-12700KF
Graphics card	R TX-3090
Memory	64G
Hard disk	6T

.

4.2. Experimental Dataset

The experimental data set was taken by a mobile phone in the real scene of the enterprise, with a total of 110 pictures, and the number of pictures was enhanced to 5000 through data enhancement technology. The robustness and generalization ability of the object detection algorithm are improved by augmenting the training data. Common data augmentation methods include scaling, translation, mirror flipping, color perturbation, etc. In this experiment, the image enhancement method of mirror flip is not adopted, because the pointer of mirror flip does not exist and is meaningless in reality. The scaling ratio is 0.5–2, and gray bars are added to the excess space in the reduction process to ensure the consistency of the image size. This experimental data set belongs to the engineering application data set in a specific scene and has certain practical value. Some pictures of the experimental data set are shown in Figure 8. The dataset consists of an object detection dataset, image segmentation dataset, and ocr dataset. The object detection data set is labeled with labelme [32], which contains six categories, namely square table, ROC table, round table, complex table, white table, and digital table. This experiment focuses on the reading of the square watch, and other types of watches will be studied in follow-up research. The image segmentation data set is annotated with Baidu PaddlePaddle’s interactive segmentation and annotation software EISeg [33], which is used to annotate the pointer part of the meter. The ocr data set is marked with PPOCRLabel [34], a semi-automatic marking tool of Baidu PaddlePaddle, to mark the model and scale value of the meter.

4.3. Experimental Results

The experimental results of the intermediate steps are shown in Figure 9. The nine pictures sequentially represent the original image, the object detection result, the cropped image of the square table, the overlay effect map of the image segmentation, the predicted mask result of the image segmentation, the thinned image, the Hough transform, the ocr result, and the reading.

4.4. Analysis of Experimental Results

4.4.1. Object Detection Evaluation Index

mAP50, FPS, params, and Gflops are indicators to measure the performance of object detection models [35]. mAP50 (mean average precision at 50) is an indicator used to measure the performance of the object detection algorithm, indicating the average accuracy of the matching degree between the detected object and the real object, where 50 means the IoU (intersection-over-union ratio) threshold is 0.5 [36]. The higher the mAP50, the higher the detection accuracy of the algorithm [37]. FPS (frames per second) indicates the number of image frames that the object detection model can process per second, that is, the inference speed of the model. The higher the FPS, the faster the detection speed of the algorithm. params (parameters) refers to the number of parameters of the model, which is an important indicator for evaluating the complexity of the model. The fewer params, the smaller the model and lower the requirement of computing resources and storage space. Gflops (giga-floating-point-operations-per-second) indicates the number of floating-point operations performed by the model per second and is an indicator to measure the computational efficiency of the model [38]. The higher the Gflops, the higher the computational efficiency of the model, which can complete the reasoning task faster. Their calculation formulas are in Formulas (5)–(9) [28].

$$\mathrm{A}\mathrm{P}=\frac{1}{\mathrm{m}}\sum _{\mathrm{i}}^{\mathrm{m}}{\mathrm{P}}_{\mathrm{i}}=\frac{1}{\mathrm{m}}\times {\mathrm{P}}_1+\frac{1}{\mathrm{m}}\times {\mathrm{P}}_2+\cdots +\frac{1}{\mathrm{m}}\times {\mathrm{P}}_{\mathrm{m}}=\int \mathrm{P}\left(\mathrm{R}\right)\mathrm{d}\mathrm{R},$$

(5)

$$\mathrm{m}\mathrm{A}\mathrm{P}=\frac{1}{\mathrm{C}}\left({\mathrm{A}\mathrm{P}}_1+{\mathrm{A}\mathrm{P}}_2+\cdots +{\mathrm{A}\mathrm{P}}_{\mathrm{c}}\right),$$

(6)

$$\mathrm{F}\mathrm{P}\mathrm{S}=\frac{1}{\mathrm{T}},$$

(7)

$$\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{a}\mathrm{m}\mathrm{s}=\mathrm{n}1\times \mathrm{m}1+\mathrm{n}2\times \mathrm{m}2+\cdots +\mathrm{n}\mathrm{k}\times \mathrm{m}\mathrm{k},$$

(8)

$$\mathrm{G}\mathrm{f}\mathrm{l}\mathrm{o}\mathrm{p}\mathrm{s}=\frac{\begin{array}{c}(2\times \mathrm{n}1\times \mathrm{m}1\times \mathrm{h}1\times \mathrm{w}1\times \mathrm{d}1+\\ \mathrm{n}2\times \mathrm{m}2\times \mathrm{h}2\times \mathrm{w}2\times \mathrm{d}2+\cdots +\\ \mathrm{n}\mathrm{k}\times \mathrm{m}\mathrm{k}\times \mathrm{h}\mathrm{k}\times \mathrm{w}\mathrm{k}\times \mathrm{d}\mathrm{k})\end{array}}{{10}^9\times \mathrm{T}},$$

(9)

In this experiment, we trained the model 300 times and compared the recognition effects of the five models. After analyzing the experimental results, we found that Yolov7 performed the best. The comparison of different models on the meter dataset is shown in

Table 3. The comparison of different models on the meter dataset.

Model	Map50	FPS	Params	Gflops	Map5095
Yolov5-s	99.5	204.08	7.03 m	15.8	99.1
Yolov5-m	99.5	85.47	20.87 m	47.9	99.2
Yolov5-l	99.3	51.02	46.14 m	107.7	99.2
Yolox-s	96.7	25.63	8.94 m	26.77	94.51
Yolov7	99.8	92.59	36.51 m	103.2	99.5

.

According to the table, the maps of Yolov5 and Yolov7 are above 99%, far exceeding Yolox. The Yolov7 model has the highest Map50 score of 99.8, indicating that it performs best in object detection accuracy. Although it has the highest computational resource requirements, it is still the best choice, especially when accuracy is a key metric. Yolov5-s has the least number of parameters and Gflops, so it is the most lightweight model and requires the least computational resources. Additionally, Yolov5-s has the highest FPS, so it is suitable for tasks that need to process images quickly. The choice of which model to use depends on the specific application scenario and requirements. If you need high-precision object detection and have enough computing resources, you can choose the Yolov7 model. If you need to process images quickly or have limited computing resources, you can choose Yolov5-s.

4.4.2. Image Segmentation Evaluation Index

In the field of image segmentation, the evaluation model quality is mainly judged by three indicators, accuracy rate (acc), average intersection over union (mean Intersection over Union, referred to as mIoU), and Kappa coefficient. The accuracy rate refers to the proportion of the pixels with correct category predictions to the total pixels. The higher the accuracy rate, the better the model quality [39]. The average intersection ratio is calculated separately for each data set category, the calculated intersection of the predicted area and the actual area is divided by the union of the predicted area and the actual area, and then the results obtained for all categories are averaged [40]. The Kappa coefficient, also known as Cohen’s kappa, is used to measure the accuracy of a classifier. The Kappa coefficient takes into account the agreement between the classifier and the random classifier, and its value ranges from −1 to 1, where 1 indicates complete agreement, 0 indicates agreement with the random classifier, and −1 indicates no agreement at all. The higher the Kappa coefficient, the better the model quality.

The formulas of accuracy rate, average intersection ratio and Kappa coefficient are shown in Formulas (10)–(13) [33].

$$\mathrm{A}\mathrm{c}\mathrm{c}=\frac{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N})}{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N})},$$

(10)

$$\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}=\frac{1}{\mathrm{n}}\times \sum \frac{\mathrm{T}\mathrm{P}}{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N})},$$

(11)

$$\mathrm{e}=\frac{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P})\times (\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N})+(\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N})\times (\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N})}{{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N})}^2},$$

(12)

$$\mathrm{k}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}=\frac{\mathrm{a}\mathrm{c}\mathrm{c}-\mathrm{e}}{1-\mathrm{e}},$$

(13)

Different models for image segmentation on the pointer dataset are shown in

Table 4. Results of different models for image segmentation on the pointer dataset.

Model	MioU	Acc	Kappa	Dice	Class IoU	Class Precision	Class Recall
ann	0.8084	0.9967	0.7639	0.8819	[0.9967, 0.6202]	[0.9984, 0.7609]	[0.9983, 0.7703]
pspnet	0.807	0.9966	0.7618	0.8809	[0.9966, 0.6175]	[0.9985, 0.7431]	[0.9981, 0.7851]
deplabv3	0.8029	0.9966	0.7555	0.8777	[0.9966, 0.6093]	[0.9984, 0.7475]	[0.9982, 0.7672]
deplabv3+	0.8773	0.9981	0.8604	0.9302	[0.998, 0.7565]	[0.9991, 0.8577]	[0.999, 0.865]
danet	0.8059	0.9965	0.7601	0.88	[0.9965, 0.6153]	[0.9985, 0.7337]	[0.998, 0.7921]

:

In the image segmentation experiments, we used the models in Table 4 to train and evaluate on the pointer dataset. The results show that the deplabv3p model has excellent performance in all aspects. The mIoU (mean Intersection over Union ratio), acc (pixel accuracy), Kappa coefficient, and Dice coefficient of the model are 0.8773, 0.9981, 0.8604, and 0.9302, respectively. In addition, the category IoU index of the model is [0.998, 0.7565], indicating the performance difference of the model on different categories; the category precision is [0.9991, 0.8577], reflecting the classification accuracy of the model for each category, and the category recall rate is [0.999, 0.865], indicating the recognition ability of the model for each category. These evaluation indicators show that the deplabv3p model has high mIoU, acc, Kappa coefficient, and Dice coefficient when performing image segmentation tasks, and can also achieve high IoU, precision, and recall on different categories. This shows that the model has high accuracy and stability when dealing with image segmentation tasks and can play a role in a variety of practical application scenarios.

4.4.3. OCR Evaluation Index

PaddleOCR calculates three OCR end-to-end related indicators, namely precision (precision rate), recall (recall rate), and Hmean (F1 value). Precision refers to the ratio of the number of correct characters in the OCR recognition result to the total number of characters. The recall rate refers to the ratio of the number of correct characters in the OCR recognition result to the total actual number of characters. The Hmean is the harmonic mean of precision and recall. Their calculation formulas are in Formulas (14)–(16) [41]:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}},$$

(14)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}},$$

(15)

$$\mathrm{H}\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}=\frac{2\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}},$$

(16)

Among them, TP represents the number of characters correctly recognized by OCR, and FP represents the number of characters incorrectly recognized by OCR [42].

In this experiment, the accuracy rate of PGNet is 98.04%, the recall rate is 81.97%, and the Hmean rate is 89.29%. The reason why the Hmean value is not high may be that the pointer will cover the scale value in some cases, resulting in the inability to accurately identify the scale, which is a common problem. As shown in Figure 10, the pointer covers the “0” scale, making it impossible to accurately identify the “0” scale value. However, under normal circumstances, the pointer will not reach the maximum value of the meter, which has little effect on the operation of taking the maximum scale value.

4.4.4. Meter Automatic Reading Evaluation Index

In this experiment, the absolute error, relative error, and accuracy are used as the measurement indicators of the meter reading, and their calculation formulas are in Formulas (17)–(19). Among them, the manual reading is performed by 20 workers, and their average value is taken as the manual reading. Results are rounded to two decimal places. The comparison between automatic identification and manual identification of pointer meters is shown in

Table 5. Comparison between automatic identification and manual identification of pointer meters.

AR	MR	AT	MT	AM	MM	WT	WM	AE	RE	ACC
8.22	11.90	A	A	75	75	Y	Y	3.68	0.05	95.09
9.18	8.76	kV	kV	12	12	Y	Y	0.42	0.04	96.50
10.55	10.40	kV	kV	12	12	Y	Y	0.15	0.01	98.75
71.32	73.10	A	A	150	150	Y	Y	1.78	0.01	98.81
21.75	23.32	A	A	50	50	Y	Y	1.57	0.03	96.86
120.78	115.25	V	V	300	300	Y	Y	5.53	0.02	98.16
10.68	10.05	A	A	15	15	Y	Y	0.63	0.04	95.80
161.35	155.88	V	V	250	250	Y	Y	5.47	0.02	97.81
3.47	3.05	A	A	20	20	Y	Y	0.42	0.02	97.90
123.56	128.92	V	V	150	150	Y	Y	5.36	0.04	96.43

. The interpretation of each abbreviation is shown in

Table 6. The interpretation of each abbreviation.

Name	Explanation
AR	automatic reading
MR	manual reading
AT	automatic identification of meter model
MT	manual recognition of meter model
AM	automatic recognition of the maximum range
MM	manual recognition of the maximum range
WT	whether the type is the same
WM	whether the maximum value is the same
AE	absolute error
RE	relative error
ACC	accuracy

.

$$\mathrm{A}\mathrm{b}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}=|\mathrm{A}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{R}\mathrm{e}\mathrm{a}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g}-\mathrm{M}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{R}\mathrm{e}\mathrm{a}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g}|,$$

(17)

$$\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}=\frac{\mathrm{A}\mathrm{b}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}{\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{m}\mathrm{R}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}},$$

(18)

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=1-\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r},$$

(19)

In this data table, we can see that there are certain errors between the automatic readings and the manual readings, but in most cases the relative errors are small, and the accuracy is high. The accuracy of the readings obtained by this scheme is not less than 95%. At the same time, we can find that the automatic identification meter type is consistent with the manual identification meter type, and the maximum range of automatic identification is basically the same as that of manual identification, which shows that the automatic identification algorithm of the meter proposed in this experiment is relatively reliable. Therefore, this solution can be used as an effective application of intelligent measurement technology in the industrial field, thereby improving the accuracy and efficiency of measurement and reducing the cost of manual operation.

5. Conclusions

In this paper, the deep learning method and computer vision technology are combined to propose a pointer meter recognition method based on Yolov7 and Hough transform, which realizes the automatic recognition and reading of pointer meters. Deep learning includes Yolov7, DeepLabv3+ and PGNet: use Yolov7 to determine the position of the meter; use DeepLabv3+ to extract the pointer part of the meter; and use PGNet to determine the model and range of the meter. Computer vision includes Thinning and Hough transforms, which can determine the precise position of needles and take meter readings. By harnessing the advantages of deep learning in object detection and image segmentation, it becomes possible to accurately locate the instrument and segment the pointer directly from the image, thereby avoiding the cumbersome process of traditional computer vision for pointer extraction. Therefore, the method effectively solves the problems of uneven illumination, complex background, image blur, and diversity of meter models in each image. The experimental results demonstrate that the utilization of the latest Yolov7 model achieved a remarkable map of 99.8% on the instrument dataset. The accuracy of the pointer readings obtained using this method is more than 95%. The method performs well in terms of anti-interference, accuracy, and robustness. When meter images are captured at highly standard shooting angles, they can be quickly and accurately recognized. However, when meter images are captured at larger angles, the recognition accuracy cannot meet the required standards. In the next phase of the work, efforts will be made to correct the meter images in order to improve the recognition accuracy.