1. Introduction
Intelligent transportation unmanned autonomous driving urgently needs high-precision navigation maps, and obtaining information on the location and details of feature targets in complex road scenes through telemetry is an important support for achieving accurate positioning and navigation. In traffic management, the 3D geometric, topographical and semantic information of trees in road scenes has certain reference value for fine lane navigation, traffic accident analysis and vehicle safety assessment. For the construction of high-precision navigation maps and intelligent traffic management needs, fast and accurate extraction of relevant features in the driving environment is the current research hot spot in the construction of "smart cities". However, at the data source level, targets such as trees with large differences between top and bottom geometric features cannot be extracted finely by deciphering traditional single 2D images, and the current mainstream tree recognition extraction method uses 3D LiDAR (Light Detection and Ranging) as the data source. LiDAR technology has gradually become one of the important data sources for earth observation and target classification and identification with its ability to acquire information about the surrounding environment quickly and in real time. LiDAR is a high-precision sensor for measuring the position and shape of objects and forming high-quality 3D point cloud images [
1,
2], and it has been widely used in recent years for autonomous driving [
3,
4,
5], 3D modeling [
6,
7,
8,
9], generating high-precision maps [
10,
11], tree biomass estimation [
12,
13] and other fields.
Street tree extraction is divided into airborne LiDAR and ground-based LiDAR (including vehicle-mounted, backpack and base station types) depending on the LiDAR data source. Airborne LiDAR point clouds are characterized by rapid acquisition and large geographical coverage; therefore, airborne LiDAR is mainly used for tree information extraction and biomass estimation, reducing data costs while preserving the extent of individual tree canopies and facilitating simplified tree population statistics extraction and biomass estimation. The ground-based LiDAR point cloud can capture the geometric information of trees with high accuracy in all directions, facilitating the extraction of salient features of individual trees to obtain accurate tree parameters for road environment refinement management and stock estimation.
Currently, tree classification extraction based on LiDAR point cloud data consists of two parts: extraction of trunks and extraction of crowns.
(1) Trunk extraction methods can be classified as: model-based extraction, structural feature-based extraction and knowledge template matching-based extraction methods.
1) Model-based extraction method: the tree trunk presents a cylindrical-like geometry. Liang et al. [
14,
15] and Cabo et al. [
16] applied a 3D cylindrical model to extract the trunk; Lehtomaki et al. [
17] applied coaxial cylinders of different size radii to fit the trunk; Li et al. [
18] applied an adaptive radius cylindrical model to fit the trunk from the bottom up; Raumonen et al. [
19] and Hackenberg et al. [
20] used a columnar surface fitting approach; and there are also a number of methods that apply RANSAC (random sample consensus) linear fitting models based on verticality [
21,
22,
23]. However, such methods for fitting trunks are strongly influenced by the density and geometry of trunk points, and regular model-fitting methods cannot extract the trunk completely when it is obscured by other types of feature points (e.g., bushes) or when the trunk is tilted to a large extent.
2) Extraction methods based on structural features: tree trunks have rod-like features with high verticality, density and continuity. Many studies have used clustering and segmentation methods to extract trunks based on structural features. Lehtomaki et al. [
24] combined a priori knowledge of tree height, location and number of points to cluster and separate trunks; Archige et al. [
25] used trunk orientation and geometric information to segment trunks; Xia et al. [
26] used multi-scale shape and size geometric features to achieve extraction; Tao et al. [
27] applied DBSCAN (Density-Based Spatial Clustering of Applications with Noise) to distinguish trunks by density; Hao et al. [
28] extracted horizontal deviations of points from centroids in a hierarchical manner in the z-direction; and Wu et al. [
29] searched for upper and lower voxels over a range of heights to achieve vowelized trunk extraction. Thanh et al. [
30] applied trunk points using horizontal profile analysis and minimum vertical height criterion extraction, and Tang et al. [
31] applied Euclidean distance clustering and minimum cut extraction. The computation of structural features depends on the sparsity of neighborhoods points and is influenced by noisy points, and the extraction accuracy of structural feature-based extraction methods is low for small target trunks and incomplete irregular trunks. Hui et al. [
32] proposed a street tree extraction and segmentation method based on the spatial geometric features of object primitives, applying a normal vector angle threshold between neighborhood points to divide the original point cloud into different primitives, and extracting trunk points through the linear features between primitives and the aspect ratio of the primitive bounding box, which can improve the robustness and accuracy of stem detection, but there is room for improvement in the integrity performance of stem detection for some irregular stems, such as the detection of severely curved trunk points.
3) Extraction methods based on knowledge template matching: knowledge templates are combinations of features, and threshold combinations of different feature values can extract different features, separating tree trunks based on the combined feature differences between tree trunks and other feature targets. Pu et al. [
33] achieved feature classification based on knowledge-based feature recognition, and super voxels are a feature fusion method; Wu [
29] et al. and Fan [
34] et al., in the segmentation process, applied feature hyper voxels of tree trunks; Li [
21,
35] combined prior knowledge, pole extraction and component separation to distinguish trees from other features; and Wang et al. [
36] applied the Hough forest of hyper voxel neighborhoods to detect pole-like features including tree trunks. Voxelization is often applied to template matching methods, but the choice of size of voxelization is critical to the extraction results. Moreover, this method requires a large amount of a priori knowledge, and the processing is time consuming.
(2) The acquisition of a single tree canopy point cloud can be divided into two aspects: classification and extraction.
1) Methods for canopy classification: The canopy layer of trees is a distinct part that distinguishes them from other features and has significantly different features from other features in the application of machine learning classification methods. Huang et al. [
37] used Euclidean clustering for point cloud coarse processing and then applied SVM to extract single trees; Huang and You [
38] used statistical pole description to conform to the application of SVM for different pole features (including trees, streetlights, billboards, etc.); Wu et al. [
39] used super voxel processing and then applied SVM and random forest processing to classify different pole features, respectively, as well as applying 3D convolutional neural networks for large-field point cloud classification [
40]. The machine learning approach requires a large amount of manual labeling and training, is suitable for target classification in large scenes, is time consuming and has low extraction accuracy when targeting specified features, and the extraction accuracy depends on the sample.
2) Methods for canopy extraction: The canopy point cloud consists of a haphazard collection of leaves and branches extending from the tree trunk, and there are fewer regular features used for extraction. The current crown extraction relies on the location and geometric shape features of the trunk, and the crown points are attributed to the specified trunk by means of region growing algorithms and clustering. Weinman et al. [
41] designed a mean drift clustering and shape analysis method to extract the crown points; Xu et al. [
42] used a bottom-up hierarchical clustering method; Yadav et al. [
43] and Xu et al. [
44] attributed the canopy points to tree vertices.
The region growth algorithm is the most commonly used method for extracting crown points through trunk points. The application of the algorithm to tree extraction involves innovations in two main areas: the selection of seed points and the choice of growth paths (growth constraints). Firstly, there is the selection of seed points based on local elevation differences and density [
45], the selection of seed points based on the criterion that the trunk is near the horizontal center of the canopy and smaller in diameter than the canopy [
46], and the use of the planar coordinates of the center of gravity of the super voxels as seed points for growth [
34]. Secondly, for the selection of growth paths, researchers have designed competing region growth [
29], breadth-first search algorithms [
34] and dual growth methods [
46], and to solve the problem of inter-canopy layer contact or overlap, region growth algorithms combined with Dijkstra’s method [
47,
48] and Voronoi constraints [
49,
50,
51] to achieve single tree segmentation. However, the shape and size of the tree, the distance between the tree and the LiDAR resulting in non-uniformity of the canopy density distribution and the absence of trunk points when the trunk is blocked by cars and green belts, which can easily lead to incorrect selection of seed point locations or incomplete growth during area growth, are still bottlenecks for improvement.
In summary, the above methods of street tree extraction have the following shortcomings:
The extraction of trunks relies on more regular geometric features and is less accurate if parts of the trunk are bent or heavily obscured;
The results of canopy extraction depend to a large extent on the position of the trunk, which cannot be extracted when the trunk is heavily obscured;
When there is a large gap between the canopy and trunk or between the canopy and crown, it is not possible to apply the regional growth algorithm to extract the canopy layer;
The informatization of tree geometric parameters is the key to forestry surveys and 3D applications, and there is a lack of methods for the fine-grained analysis of tree parameters.
To address the above-mentioned existing methods where the density of both trunk and crown points are unevenly distributed, this paper proposes a mobile LiDAR point cloud and region growth algorithm based on Gaussian distribution to extract the trunk and crown of the road environment and combines Voronoi (Tyson polygon) constrained segmentation to automate the segmentation of overlapping single trees to achieve 3D fine information and geometric parameter extraction of single trees.
The remainder of this paper is organized as follows:
Section 2 illustrates the methodology,
Section 3 reports the experimental area and results, and
Section 4 reports the discussion. Finally,
Section 5 draws the conclusions of the work.
2. Materials and Methods
2.1. Technical Process
In this paper, a region growth algorithm based on Gaussian distribution (GDRG) is designed to extract the point clouds of street trees in the road environment based on the mobile LiDAR point cloud data. Firstly, the original mobile LiDAR point cloud data is pre-processed, including ground point filtering and filtering the building point cloud based on the projected area of the building façade; then, the location of a single tree is located based on the vertical linear features of the tree trunk and the trunk point cloud is extracted by clustering based on the GDRG algorithm; secondly, the pseudo-trunk is removed based on the simple geometric rules of the trunk; then, the GDRG algorithm is applied again to extract the canopy point cloud of each tree separately. Finally, the Voronoi diagram constraint is used to partition the tree group and calculate the geometric features of the individual trees. The detailed technical flow of the method is shown in
Figure 1.
2.2. Non-Tree Points Filtering Pre-Processing
The original point cloud is filtered by a slope-based filtering algorithm, and the candidate points are mainly the infrastructure target point clouds on both sides of the road, where the building façade points and some shrub points also interfere with the extraction of the tree trunk point cloud. This paper distinguishes and filters the non-tree point sets such as building facades, vehicles, shrub points and tree points by the difference in projection geometric scale between the building facades, vehicles and shrubs.
Candidate points were obtained by applying the Euclidean distance clustering algorithm to cluster clusters with a relative elevation threshold (1.5 m is used in this paper, is a range value, approximately equal to about 1/2 the height of a standard tree in the study area and is mainly used to separate part of the trunk from the crown for subsequent projection). Each cluster was stratified into upper and lower parts for each clustering cluster.
The overall and lower parts of the clusters are projected raster in the horizontal plane, with the number of pixels containing the point cloud for each cluster being
for the overall raster projection,
for the lower part raster projection, and the area of the enclosing rectangle being
. The overall and lower part projections of the buildings, trees and shrubs are shown in
Figure 2.
a. Area difference
is calculated. The calculation formula is shown in (1).
In the candidate point , the projection of the building is striped and the empty raster inside the outer rectangle is the majority; the area difference is close to 0. The projection of other infrastructure such as trees is circular and there is no empty raster inside the outer rectangle; the area difference is close to 1. Calculate the average value of the area difference for all projection units, then eliminate the projection cells with an area difference less than , so the building point can be removed.
b. Proportional calculation
. The calculation formula is shown in (2).
The ratio of vehicles and shrubs is close to 1; while the ratio of trees is close to 0, calculate the average value of the proportional factor for all projection units, and eliminate the projection cells with a proportional factor more than , by which the shrub and vehicle points can be removed.
After removing some of the noisy clusters, the candidate point was obtained.
2.3. Regional Growth Algorithm for Gaussian Distribution (GDRG) to Extract Trunks
The trunk portion of the street tree in candidate point
has strong vertical geometric properties, which can be used to extract part of the trunk point cloud and locate the position of the trunk. The verticality eigenvalues are calculated via principal component analysis (PCA) [
52,
53] of the global point cloud and solution for the eigenvalues, which are calculated as follows (3).
indicates the verticality eigenvalue of each point in the range 0 to 1; and ( > ) are the first and second eigenvalues, respectively.
The candidate point set includes a large number of trees and some point clouds containing pole parts such as streetlights and billboards. Calculating the vertical linear features can extract the pole parts of different infrastructures and filter part of the point cloud set that does not have vertical features (such as cars and canopy of trees). The verticality of the point cloud is calculated based on the neighborhood points of each point, then some points are incorrectly represented as vertical, e.g., the points in the tree canopy have irregularity and the vertical feature values of some points are calculated to meet the threshold, or some points in the tree trunk are lacking of extraction from the same trunk because they are obscured and subject to neighborhood noise points that do not reach the threshold.
The candidate point set was extracted after verticality to obtain a point set with missing points in the pole part of the infrastructure point cloud and to extract an incomplete trunk point set.
This section proposes a method for finding missing rod points in the extracted point set from the original point cloud data, a region growth algorithm based on Gaussian distribution (GDRG). The extracted points are used as seed points to find the points that match the rules in the original point cloud through the Gaussian distribution law, and the grown points are clustered and merged into a single trunk point set.
The GDRG algorithm is an interval estimation method that focuses on the points within the confidence interval of the Gaussian distribution in the sample point set as the candidate seeds for growth. Confidence interval estimation for a Gaussian distribution involves creating an interval containing the parameters to be estimated for the normal distribution of the sample at a given probability value. This confidence interval [
] is an interval of values derived from the sample that may contain the overall parameters and is usually obtained by adding or subtracting the estimation error from the sample statistic. The lower and upper limits of the confidence interval
and
are calculated in (4).
where
denotes the mean of the estimated parameters;
denotes the value in the table of normally distributed
values at a confidence level of
; n denotes the number of samples;
denotes the standard deviation of the sample, reflecting the degree of dispersion of the whole sample from the sample mean; and
is the standard error of the sample.
The seed points are selected from the trunk points and searched in the neighborhood. The set of points searched is used as a sample, and the difference between the sample points and the seed points in the
and
directions,
and
, respectively, are used as parameters for interval estimation. The distribution of the trunk point set is robust, with possible trunk and seed points having parameter errors within the confidence interval of the Gaussian distribution, while the outliers have larger errors and are not within the confidence interval, and the points within the confidence interval are added to the growing point set. The GDRG algorithm is shown in
Figure 3. The pseudocode of the algorithm set is in
Table 1.
Find the black neighborhood points (points to be judged) based on the yellow points (points that have grown). Randomly select a yellow point as the initial seed point to find the 20 nearest neighbor points, calculate the distance difference between the neighborhood points and the seed point in the x and y directions, and then calculate the Gaussian distribution of the difference, set to establish a confidence level of 0.05 confidence interval. The points that meet this confidence interval are added to the red point set, and the next nearest point in the remaining neighborhood point set is repeatedly grown as a new seed point. The algorithm is terminated when there are no new points to be added or when all remaining points to be added do not qualify for the confidence interval.
The set of points that ends the growth is the set of clustering points of the first trunk; the new initial seed points that are not marked by the growth are reselected in the candidate point set, and the algorithm is repeated to obtain the set of clustering points of the next N trunks, which is the set of clustering points of the trunks in the candidate point set T2.
The extracted point set is processed by a GDRG algorithm to obtain a candidate point set T2. The candidate point set T2 complements the clustering of various rod point sets, but there are still pseudo-rod point sets clustered due to the irregular growth of the canopy point cloud.
2.4. Single Tree Segmentation Method with Voronoi Range Constraint
2.4.1. Canopy Group Extraction
The rod clusters were labeled in the original point cloud and the canopy point set was grown using the rod point set
as seed points again applying an RG (regional growth) algorithm with optimized seed points method (
Figure 4). Three improvements were made to the RG algorithm:
(1) Due to the distance between some of the tree canopies and the distance at the trunk apex, the neighborhood search of the RG algorithm for the canopies applies the nearest neighbor method, using the trunk apex as the initial point to search for growth.
(2) To prevent seed points from growing from the trunk vertex down to the noise point, the height of the next canopy candidate at the initial seed point should be greater than the initial seed point height .
(3) Due to the large gaps between some of the canopies, a vertical constraint is added to the selection of the next canopy candidate point of the initial seed point, i.e., the next batch of seed points of the trunk vertex is the nearest neighbor of a vertex within a cylindrical range with the vertex as the center and a horizontal radius and a height greater than .
Figure 4.
RG algorithm with optimized seed points method (red points represent trunk points, green points are tree crown points, the red point in the black circle is the top point of the trunk, the set of points in the the black cylinder are candidate seed points, and the arrows on the cylinder represent the subsequent growth direction of the candidate points).
Figure 4.
RG algorithm with optimized seed points method (red points represent trunk points, green points are tree crown points, the red point in the black circle is the top point of the trunk, the set of points in the the black cylinder are candidate seed points, and the arrows on the cylinder represent the subsequent growth direction of the candidate points).
2.4.2. Pseudo-trunk Identification
The trunk extracted in
Section 2.3 above has a partial set of pseudo-trunk points. Considering that there may be trunks in the study area that are too heavily obscured to extract the correct trunk and grow a crown, the pseudo-pole sections and pseudo-trees were identified after the full crown was grown through the pseudo-pole.
Pseudo-trunks were identified by applying a crown-to-trunk matching method. Each full-grown canopy cluster has trunks. A minimum enclosing box is applied to the set of pseudo-trunk points in each canopy to calculate geometric features and set geometric rule thresholds for the different features, and the set of pseudo-trunk points that do not meet the threshold conditions are eliminated. The geometric rules include: (1) Volume , which calculates the volume of the rectangular minimum enclosing box for each set of rod clustered points; this rule is valid for the recognition of streetlights and billboards. (2) Height difference , the height difference from the lowest to the highest point of the enclosing box of the rod part of the clustering cluster; this rule is valid for the recognition of partially noisy points. (3) Groundedness , calculation of the average DEM of each region and the difference in elevation between the lowest points of the minimum enclosing boxes of the clustered clusters. Pseudo-clustered clusters of poles larger than the threshold are excluded, and this rule is valid for the identification of pseudo-trunks in the canopy.
The eligible rod clusters are identified after filtering by the three rules above, and the correct trunk is matched to the corresponding canopy. If a crown has no matching trunk, the tree is severely missing trunk points; if a crown cluster has multiple trunks, the canopy area may overlap tree groups and is a focus for subsequent single wood segmentation.
2.4.3. Single Tree Segmentation Method with Voronoi Range Constraint
In the experimental area, there is an overlapping situation between arrays, where trees are close to each other or the canopies grow in opposite directions and overlap and shade each other, resulting in multiple trees clustering into one category after the area has grown. In order to partition single trees, this paper applies Voronoi diagrams (Tyson polygons) [
52] to solve the problem of multiple trees overlapping each other.
The solution for the 3D gravity of each trunk cluster is used to represent the true location of each tree, and the gravity is displayed as 3D points. The absence of noise points above the set of tree points in the candidate points and the fact that each tree is not of the same height results in the coordinates of each prime not being uniform in height. The main purpose of the method in this section is to segment overlapping tree groups, and if a 3D Voronoi diagram is created with the gravity coordinates, it may lead to over-segmentation of the different tree canopies, so the gravity is projected in 2D. The 2D Voronoi diagram automatically divides the plane into n polygonal regions centered on the gravity according to the positions of the gravities in the Euclidean plane, and the polygons are non-overlapping and adjacent.
After the 2D projection of the center of mass, a Voronoi diagram is created for the 2D center of mass points at the same height to determine the extent of the plane to which a single tree belongs, and then all canopy points with height information within the plane are obtained. As shown in
Figure 5, the Voronoi diagram interval partition is established with the centroid point of the trunk in the experimental area.
After the candidate points are filtered via the noise point filtering process, the shrub points within the canopy range are eliminated and the candidate points in each polygon buffer created with the Voronoi diagram area contain only tree points. The candidate points in each buffer are extracted separately to achieve automatic segmentation of a single tree, and a single tree point cluster is obtained.
2.5. Calculation of Tree Geometry Parameters
(1) Position of the street tree. In
Section 2.4.3, the coordinates of the projection of the center of mass of the trunk on the XOY plane are solved to represent the position of the street tree.
(2) Breast dimension. The diameter of the raster cell after projection of the trunk cluster in the XOY plane is the breast dimension.
(3) Trunk height. The height of the apex of the trunk cluster is the height of the trunk.
(4) Canopy crown width. The extracted canopy is projected on the XOY plane to approximate the shape of a circle, and the diameter calculated by the formula for the area of a circle is the canopy width of each tree.
(5) Tree height. The height of the apex of the tree crown.