A 3D Point Cloud Feature Identification Method Based on Improved Point Feature Histogram Descriptor

Abstract

A significant amount of research has been conducted on the segmentation of large-scale 3D point clouds. However, efficient point cloud feature identification from segmentation results is an essential capability for computer vision and surveying tasks. Feature description methods are algorithms that convert the point set of the point cloud feature into vectors or matrices that can be used for identification. While the point feature histogram (PFH) is an efficient descriptor method, it does not work well with objects that have smooth surfaces, such as planar, spherical, or cylindrical objects. This paper proposes a 3D point cloud feature identification method based on an improved PFH descriptor with a feature-level normal that can efficiently distinguish objects with smooth surfaces. Firstly, a feature-level normal is established, and then the relationship between each point’s normal and feature-level normal is calculated. Finally, the unknown feature is identified by comparing the similarity of the type-labeled feature and the unknown feature. The proposed method obtains an overall identification accuracy ranging from 71.9% to 81.9% for the identification of street lamps, trees, and buildings.

1. Introduction

Light detection and ranging (LiDAR) technology is a relatively new method for obtaining high-quality three-dimensional spatial data and is considered an emerging earth observation technology. One of the key advantages of LiDAR technology includes high data density, accuracy, and strong penetration. As such, LiDAR systems have become increasingly prevalent in many fields, such as terrain surveying [1], forest ecological research [2,3,4], coastal zone monitoring [5], urban 3D reconstruction [6], urban change detection [7], urban road detection and planning [8,9], robot environmental perception [10], and more.

However, while the hardware and acquisition technologies for LiDAR systems have rapidly developed, research on the post-processing and application of point cloud data has significantly lagged behind. Current 3D data processing methods face several issues, such as low automation and heavy manual processing workload. Despite significant efforts to develop data filtering, classification, and extraction algorithms for 3D point cloud data, there are still limitations to these methods, and large amounts of data are not being fully utilized. Additionally, there is a lack of robustness in current methods, and point cloud feature automatic recognition research is limited compared to point cloud automatic segmentation research.

However, in the fields of surveying and computer vision, the goal of point cloud processing methods is to achieve real-time or quasi-real-time automatic ground object recognition. This technology will be able to achieve automatic classification and recognition of ground objects in laser point cloud data like supervised and unsupervised classification techniques and deep learning techniques used in remote sensing image processing which can realize automatic identification [11]. Building on the early automatic segmentation method [12], this research is focused on developing an improved method for 3D point cloud feature identification, which addresses the expression of point cloud features in point cloud data (description) and the automatic recognition of ground features (identification).

2. Related Work

2.1. General Process

Laser point cloud data consist of points obtained with a laser scanner, which contain spatial coordinates, intensity information, and sometimes color information. In this paper, the point cloud feature is defined as a set of laser-scanned points that represent a specific entity, such as a tree, building, etc.

Hoffman and Jain [13] were the first to propose a complete processing process for laser scanning point cloud data, which includes five steps: data collection, data preprocessing, segmentation (or filtering classification), classification, and object modeling. Generally, point cloud data are processed according to this process [13].

As significant research has already been conducted on the segmentation of point cloud data [14], this paper will not cover segmentation algorithms in detail. Instead, the focus of this paper is on methods for point cloud feature description, classification, and recognition, based on the segmented point cloud.

2.2. Description Methods for Point Cloud Features

A single point or part of a point cloud feature is insufficient in representing the entire shape of the feature. All the key points of the point cloud feature together form the whole point cloud feature. The challenge lies in describing scattered points that do not have a clear topological relationship with each other. Moreover, for the same ground object, the position, quantity, and density of points in point cloud data collected at different times or by using different instruments may vary, making it difficult to describe them accurately.

Feature description methods are algorithms that convert point cloud feature points into vectors or matrices that represent the dataset. The description method should be stable and distinguishable to describe and identify features. Stability refers to the fact that for the same object, the description results should be stable and have high similarity. Distinguishability refers to the ability to differentiate different features, and the vectors or matrices should have a high degree of heterogeneity.

Feature matching is the process of comparing two features to determine whether they belong to the same category. The main process of feature description and feature matching is as follows: First, key points of the model are extracted, then the key points are converted to a descriptor array. Finally, the model and scene descriptor arrays are matched [15].

Salti and Tombari et al. propose a categorization of the main 3D description methods by dividing the state of the art into signatures and histograms [15,16,17]. The signatures describe the 3D surface neighborhood of a given point by defining an invariant local reference frame [17]; histograms describe the points by encoding counters of local topological entities into histograms according to a specific quantized domain [15].

Another descriptor named shape contexts proposed by Frome et al. [18] divides the point set into a grid along three coordinate directions: radial, directional, and elevation. The feature description vector is constructed by counting the number of points in the grid. Although the descriptor is simple in operation, its adaptability is poor.

The point feature histogram (PFH) [19] and fast point feature histogram description (FPFH) methods proposed by Rusu et al. [19,20,21,22] use a given point’s k-nearest neighbor points and its normal to build a histogram vector. The PFH algorithm takes computational complexity and uniqueness into account and has strong robustness. The SIFT algorithm proposed by Lowe et al. [23] extends the algorithm for two-dimensional color images to three-dimensional space [24,25]. This algorithm maintains invariance for rotation, scaling, brightness changes, etc., but has high descriptor dimensions and computational complexity. The normalized aligned radial feature (NARF) algorithm proposed by Steder and Rusu et al. [26] has rotational invariance, but the features on image edges are not obvious and are sensitive to noise [27]. The spin image method proposed by Johnson [28] generates description information of ground objects from different perspectives.

Overall, the current feature description algorithms have their own advantages and disadvantages, and their universality is poor. According to the description methods, there are mainly two methods for point cloud feature description: the signature method and the histogram method. Salti and Tombari et al. summarize the taxonomy of 3D descriptors [16,17] and methods’ category, unique local reference framework, and they whether support color information.

The next step after computing the signature and histogram descriptors for segmented point cloud data is to identify the point cloud features.

2.3. Identification Methods for Point Cloud Features

Point cloud feature identification refers to the process of matching unknown features with known ones to classify features. In recent years, there has been a growing number of automatic classification methods that use deep learning for spatiotemporal data, such as remote sensing images, laser point clouds, SAR, and others [29,30,31,32,33,34]. These methods are mainly based on the supervised classification of statistical learning data, which requires learning sample data in advance to determine model parameters and then using the obtained model to classify sub-data. Valuable references for machine-learning-based automatic ground object classification and filtering of ground point cloud data have been provided by Anguelov et al. [35], Triebel et al. [36], and Munoz et al. [37,38,39]. Charles, R.Q. et al. proposed the PointNet method for deep learning on point sets for 3D classification and segmentation [40], while Luis A. Alexandre performed a comparative evaluation on 3D point clouds, exploring both object and category recognition performance and describing existing feature extraction algorithms in a publicly available point cloud library [41]. Li, J. et al. applied the OFDV Net to standard public exterior large-scale point cloud dataset segmentation [33] and achieved good extraction effects. Pritpal Singh et al. provided a quantum-clustering optimization method for COVID-19 CT scan image segmentation [42] and a type-2 neutrosophic-entropy-fusion-based multiple thresholding method for brain tumor tissue structure segmentation [43]. However, these algorithms are mainly aimed at a certain type of data, such as airborne LiDAR data, LiDAR and image fusion data, or vehicle-borne LiDAR data. Currently, there are automatic or semi-automatic data segmentation and classification methods applied in actual production, but these methods need continuous improvement in terms of data extraction correctness, accuracy, efficiency, applicability, automation, and dependency on human experience.

3. Methodology

3.1. Overview of the Proposed Framework

The overall technical approach is as follows:

1. Point cloud segmentation: In our recent work [12], we proposed an improved DBSCAN method with automatic Eps estimation for point cloud segmentation and use it in this research for point cloud segmentation. In the improved DBSCAN method, the average of k-nearest neighbors’ maximum distances is used to fit a curve to estimate the important radius parameter ε in the DBSCAN method which can segment different types of LiDAR point clouds with higher accuracy in a robust manner.

2. Point cloud feature description and expression: Based on the point cloud data segmentation results, the 3D grid sampling method is used to remove the noises in the feature’s point set. Then, the feature descriptor is computed using the improved PFH and FPFH methods. The 3D grid sampling method divides 3D point cloud data into multiple small 3D grids based on point cloud density. The closest point to the center point is retained in all grids, while other data within the grid are ignored.

3. Case-Based Reasoning database establishment: Firstly, we label the point cloud feature histogram descriptors and establish a Case-Based Reasoning database as known cases. Then, we use the correlation coefficient to design the retrieval and matching method of the case library.

4. Point cloud feature identification: By using the retrieval and matching method in the database, we can identify the type of the unknown point cloud features.

The technical approach of this paper is shown in Figure 1.

3.2. PFH and FPFH Descriptor

3.2.1. PFH Descriptor

Rusu et al. [19] proposed the point feature histogram (PFH) method that contains a set of methods for building feature point representations. The PFH method is used for accurately labeling points in a 3D point cloud, and the representation is based on the k-neighborhood points and their surface normal [19,20,44]. Detailed theoretical primer can be referred to in [19,20,44].

For a query point $P_q$, Figure 2 presents an influence region of $P_q$ for computing PFH features. The final PFH descriptor is computed as three tuples $〈\alpha ,\mathrm{\varnothing },\theta 〉$ for pairs of points in the neighborhood, and its computational complexity is $O\left(k^2\right)$ [19,45].

The final PFH representation for the query point is created by binning the set of all three tuples $〈\alpha ,\mathrm{\varnothing },\theta 〉$ into a histogram [20,44]. When 5 binning subdivisions are used, the final histogram result is 125-dimensional vectors ($5^3$) with float values.

3.2.2. Fast PFH Descriptor (FPFH)

To simplify the computation of PFH histogram features, fast PFH (FPFH) is performed on the query point to its neighbors but not all pairs of its neighbors [22] as shown in Figure 3. The FPFH reduces the computational complexity to $O\left(nk\right)$ and retains most of the discriminative power of the PFH [22].

3.3. Improved PFH Method

3.3.1. Disadvantage of PFH

The original PFH method relies on the fitting surface at a given point P to determine the method vector of the point. However, this approach may not be effective in distinguishing objects with smooth surfaces, such as cylinders, planes, and spheres.

To highlight this difference, we selected 3D basic shapes, including corners, edges, cones, planes, cylinders, and spheres, for comparison (as shown in Figure 4). The PFH descriptors are calculated using the method explained above, resulting in a 125-dimensional vector array which is compressed to 25 dimensions for presentation purposes (as shown in Figure 5).

As shown in Figure 5, the vector values for corners, edges, and cones are evidently dissimilar, underscoring a high level of heterogeneity. However, the same cannot be said for cylinders, planes, and spheres, for which all vector values are identical, resulting in a value of 100 at the 13th dimension and 0 for all other dimensions. Consequently, the PFH descriptor is unable to distinguish between 3D objects such as cylinders, planes, and spheres, as their surface curvatures are relatively smooth. In the process of binning vector values to histograms, the small differences in surface curvature are insufficient to create separate bins for these objects. As a result, the PFH method cannot effectively distinguish among smooth surfaces.

3.3.2. Improvement of PFH

To address the issue of distinguishing smooth surfaces, we propose an enhanced version of PFH which is capable of distinguishing objects with smooth surfaces. For points $p_s,p_m$ and their normals $n_s$,$n_m$, as shown in Figure 6, the improved PFH process is as follows.

First, we establish the normal $n_m$ original point and the feature’s middle point $p_m$, and the direction is from $p_m$ to the farthest point to $p_m$ in the feature’s point set.

Next, we compute the fitting plane and normal for each point in the point cloud.

In the third step, we calculate difference between normal $n_s$ and $n_m$ for each point $p_s$ in the feature’s point set and its corresponding fitting plane normal.

Finally, we bin the results into tuple $〈\alpha ,\mathrm{\varnothing },\theta 〉$ arrays and generate histograms.

To compare the differences between various 3D basic shapes, we compute their descriptors using the improved PFH method, which utilizes feature-level normal information. The results, shown in Figure 7, demonstrate that cones, corners, and edges remain distinctly recognizable, while cylinders, planes, and spheres are also well distinguished from one another. Additionally, the peaks of each object differ significantly not only in size but also in dimension. Therefore, the improved PFH method helps to overcome the limitations of PFH, which cannot accurately distinguish between objects with smooth surfaces, such as cylinders, planes, and spheres.

It can be observed from Figure 7 that cones, corners, and edges are still clearly distinguishable, while cylinders, planes, and spheres are also well distinguished.

3.4. Identification Method

3.4.1. Point Cloud Feature Descriptor Database

The point cloud feature, which serves as the fundamental research unit for object recognition, can be described using the following database structure:

PointCloud Case = {

       Name,

       Class,

       Overall-Descriptor,

       Detailed-Descriptor

}

• The Name field includes a feature’s name, ID, corresponding file name, and other descriptive information.
• The Class field provides category information for the feature, defining its place within the wider set of point cloud features.
• The Overall Descriptor provides an overview of the feature, including spatial boundary range, length, width, height, and volume—where the volume is calculated from the minimum bounding cube of point cloud features.
• The Detailed Descriptor contains the statistical histogram’s peak values calculated using the improved PFH or FPFH method, providing more specific information about the feature.

3.4.2. Identification Method

To compare the similarity between the elements of a point cloud, two types of spatial geometric form information, namely the overall descriptor and detailed descriptor, are used. These descriptors are utilized in a similarity calculation model constructed as follows:

$$Similarit{y}_{Case(i,j)}=w_1\times S_{r\left(Case\right(i,j\left)\right)}+w_2\times S_{a\left(Case\right(i,j\left)\right)}$$

(1)

where ${Similarity}_{Case(i,j)}$ is the similarity coefficient of point cloud feature $i\mathrm{a}\mathrm{n}\mathrm{d}j$; $S_{r\left(Case\left(i,j\right)\right)}$ is the overall similarity coefficient of feature $i\mathrm{a}\mathrm{n}\mathrm{d}j$; $S_{a\left(Case\left(i,j\right)\right)}$ is the detailed similarity coefficient of feature $i\mathrm{a}\mathrm{n}\mathrm{d}j$; $w_{1}and{w}_2$ are the weight coefficient for overall and detailed descriptor, respectively. $w_1\mathrm{a}\mathrm{n}\mathrm{d}{w}_2$ can be determined based on the type of feature object. The sum of $w_{1}and{w}_2$ is 1.

The formulas for $S_{r\left(Case\left(i,j\right)\right)}$ and $S_{a\left(Case\left(i,j\right)\right)}$ are shown in (2) and (3).

$$S_{r\left(Case\left(i,j\right)\right)}=1-\frac{\left|V_i-V_j\right|}{Max(V_i,V_j)}$$

(2)

where $V_i\mathrm{a}\mathrm{n}\mathrm{d}{V}_j$ are the volumes of feature $i\mathrm{a}\mathrm{n}\mathrm{d}j$, respectively. In this study, the overall descriptor of the point cloud feature is calculated as the volume of the minimum bounding cube. This method is relatively simple and easy to implement.

The correlation coefficient method is used for the detailed similarity coefficient $S_{a\left(Case\left(i,j\right)\right)}$ calculation.

$$S_{a\left(Case\left(i,j\right)\right)}=\frac{1}{n-1}\sum _{i=1}^n\left(\frac{X_i-\overline{X}}{{\sigma }_X}\right)\left(\frac{Y_i-\overline{Y}}{{\sigma }_Y}\right)$$

(3)

In Equation (3), n represents the set of dimensions where the peak values of feature are $i\mathrm{a}\mathrm{n}\mathrm{d}j$. $\frac{X_i-\overline{X}}{{\sigma }_X}$, $\overline{X}$, and ${\sigma }_X$ are the standard score, sample mean, and sample standard deviation, respectively.

To ensure accurate calculation of the correlation coefficient, it is necessary to discard the dimensions with all peak values at 0, and the set of dimensions is dynamically selected with peaks between two scenarios to participate in similarity calculation. This is because having most of the peaks at 0 may lead to an inaccurate calculation of the correlation coefficient.

The identification process involves establishing a sample library of known feature types and calculating the similarity between the unknown feature type and each sample in the library. Finally, the resulting similarity values are sorted, and the feature type with the highest similarity score is determined as the unknown type.

4. Materials and Experiments

4.1. Datasets

The dataset used in this paper is the segmentation results of our previous work that used an algorithm based on the DBSCAN density clustering method [12]. The dataset, acquired by using a mobile survey system, covers the study area of a 500 m long street and encompasses trees, street lamps, buildings, and other objects, as depicted in Figure 8.

4.2. Experiments and Analysis

4.2.1. Labeling the Point Cloud Features

We selected simple-shaped objects such as street lamps, complex-shaped objects such as trees, and building facades as experimental objects. Representative entities from each class were carefully chosen and labeled to construct the sample database. In addition, unlabeled objects were selected to form a test database. The sample database comprised 19 trees, 22 street lamps, and 8 buildings, as illustrated in Figure 9, while the test database contained 553 unlabeled objects, as demonstrated in Figure 10.

4.2.2. Point Set Sampling

We performed feature point set sampling using the 3D grid sampling method on the segmented point cloud features to enhance the representativeness of the point set, reduce data redundancy, and remove noises, thereby improving the calculation speed of the feature descriptor. The average distance between points was calculated using the distance between k-nearest points and its range $d\in \left(0.354,2.040\right).$ For point set sampling, we chose a radius of $r=0.5$ for the 3D grid sampling method. The resulting point set samples for street lamps, trees, and houses are displayed in

Table 1. Point set sampling.

Type	Original Point Set	Sampled Point Set
Street Lamp
Tree
Building

.

4.3. Feature Description

Following point set sampling, a point cloud feature descriptor database is generated by organizing the name, class, overall description, and detailed description for each feature in the sample database. The overall description is obtained by computing the minimum bounding cube of point cloud features, while the detailed descriptor is calculated using the improved PFH and FPFH methods. For detailed description calculation, the radius for normal estimation is set to $r_n=0.9$, and the nearest neighbor search radius is set to $r_l=0.9$. The structure of the feature descriptor database is illustrated in Figure 11.

The PFH and FPFH descriptors for representative features are presented in

Table 2. PFH description of partial samples.

ID	Point Cloud Feature	Improved PFH Descriptor	Improved FPFH Descriptor
S1
S2
S3
T1
T2
T3
B1
B2
B3

.

Upon analyzing the result table, we observed that the street lamp exhibited peak values near the 60th dimension and relatively smaller peaks near dimensions 15, 38, 88, and 115. The tree, on the other hand, had two or three higher peaks close to dimensions 15, 38, and 115 and two higher peaks at dimensions 62 and 88. The building facade demonstrated high peaks in dimensions 15, 38, 62, 88, and 115. This observation indicates that the histogram features of different features exhibit distinct peak positions, values, and numbers. Nevertheless, in building facades 2 and 3, although the peak positions are similar, the number of peaks at each position varies, allowing for distinction between the two.

From Table 2, it is evident that when using the improved FPFH method, the street lamps exhibit three higher peaks near dimensions 5, 10, 16, 23, and 30, respectively. For a single tree, high peaks are observed near dimensions 2, 4, 10, 16, 21, 29, 30, and 31. Building facade 1 shows high peaks in dimensions 0, 5, 16, 22, and 27, whereas building facade 2 exhibits high peaks in dimensions 0, 5, 16, and 22. Building facade 3 displays high peaks in dimensions 0, 5, 10, 16, 22, and 33. The histograms of different features vary significantly, and buildings of different forms within the same category have both similarities and differences. For instance, the position and number of peaks in the histograms of street lamps, single trees, and building facades are markedly different. While building facades 2 and 3 have similar peak positions, the value of building facade 3 in the 10th dimension is substantially higher than that of the other two building facades. Therefore, different individuals in the same category can be distinguished.

4.4. Feature Identification Results

We calculated the improved PFH and FPFH descriptors for the test database and measured the similarity between the test database and sample database. When $Similarit{y}_{Case(i,j)}\ge$0.8, features i and j are labeled as the same class. The identification results are shown in

Table 3. Identification results.

Descriptor	Class	Total	Original Method		Improved Method
			Correct	Accuracy	Correct	Accuracy
PFH	Street lamp	83	65	78.31%	74	89.10%
	Tree	190	143	75.26%	167	87.90%
	Building	13	8	61.54%	10	84.60%
	Average	-	-	71.70%	-	87.20%
FPFH	Street lamp	71	51	71.83%	58	81.70%
	Tree	177	125	70.62%	140	79.10%
	Building	10	6	60.00%	8	80.00%
	Average	-	-	67.48%	-	80.27%

.

A total of 83 street lamps are identified using the improved PFH method with detailed descriptions. After manual judgment, nine of them are identified as noise or recognition errors, resulting in an accuracy of 89.1% while the original method has an accuracy of 78.31%. A total of 190 individual trees are identified, and 167 are correct, while the rest are noise and incorrect ones, resulting in an accuracy of 87.9% while the original method has an accuracy of 75.26%. A total of 13 houses are identified, resulting in an accuracy of 84.6% while the original method has an accuracy of 61.54%. The incorrect recognition occurred when rows of trees are identified as houses. The average accuracy of the improved PFH method is 87.20% compared to the original method which has an accuracy of 71.70%.

For the improved FPFH method, a total of 71 street lamps are identified. After manual judgment, 13 of them are incorrect and are identified as noise, resulting in an accuracy of 81.7%. A total of 177 trees are identified, and 140 are correct with an accuracy of 79.1% while the original method has an accuracy of 70.62%. For buildings, the accuracy of the improved method is 80% while the original method has an accuracy of 60% The average accuracy of the improved FPFH method is 67.48% compared to the original method which has an accuracy of 80.27%.

From the identification results, it can be seen that the accuracy of the improved PFH and FPFH methods is higher than the original ones. When calculating PFH, the relationship between adjacent points was taken into account, leading to more detailed descriptions but also requiring a larger initial calculation. The improved method was simplified, resulting in reduced calculations while maintaining a high recognition accuracy. In practical applications, the selection of detailed description methods should be based on the actual situation. The matching results are shown in Figure 12 and Figure 13.

5. Conclusions

Upon comparing the difference in PFH descriptors for 3D basic shapes, such as corners, edges, cones, planes, cylinders, and spheres, we discovered that the PFH method cannot differentiate between shapes with smooth surfaces, such as planes, cylinders, and spheres. To compensate for this deficiency, we improved the PFH method by including the feature-level normal. Our experiments for identifying street lamps, trees, and buildings showed that the identification method, which compares the similarity of PFH or FPFH descriptors and volumes, has the capability to identify point cloud features.

Future work may focus on improving the computation method for similarity, such as using deep learning methods. Additionally, to obtain more accurate results, we may need to increase the number of samples.