An Evaluation of the Effects of UAS Flight Parameters on Digital Aerial Photogrammetry Processing and Dense-Cloud Production Quality in a Scots Pine Forest

Abstract

The application of unmanned aerial systems (UAS) in forest research includes a wide range of equipment, systems, and flight settings, creating a need for enhancing data acquisition efficiency and quality. Thus, we assessed the effects of flying altitude and lateral and longitudinal overlaps on digital aerial photogrammetry (DAP) processing and the ability of its products to provide point clouds for forestry inventory. For this, we used 18 combinations of flight settings for data acquisition, and a nationwide airborne laser scanning (ALS) dataset as reference data. Linear regression was applied for modeling DAP quality indicators and model fitting quality as the function of flight settings; equivalence tests compared DAP- and ALS-products. Most of DAP-Digital Terrain Models (DTM) showed a moderate to high agreement (R² > 0.70) when fitted to ALS-based models; nine models had a regression slope within the 1% region of equivalence. The best DAP-Canopy Height Model (CHM) was generated using ALS-DTM with an R² = 0.42 when compared with ALS-CHM, indicating reduced similarity. Altogether, our results suggest that the optimal combination of flight settings should include a 90% lateral overlap, a 70% longitudinal overlap, and a minimum altitude of 120 m above ground level, independent of the availability of an ALS-derived DTM for height normalization. We also provided insights into the effects of flight settings on DAP outputs for future applications in similar forest stands, emphasizing the benefits of overlaps for comprehensive scene reconstruction and altitude for canopy surface detection.

1. Introduction

Remote sensing (RS) includes the use of both active and passive sensing technologies for the measurement of surface characteristics from a distance [1,2]. Given its variable spatial scale, systematic acquisition schedule of some platforms, and diversity of sensors, RS data have been applied in various topics, such as land cover processes and atmospheric, hydrologic, oceanographic, and especially forest studies [1,3,4,5]. In this context, RS applications for forests are complementary to traditional frameworks for data collection, minimizing spatial and cost limitations of the latter approach, and offer opportunities for accurate landscape-scale estimation of forestry inventory variables [6,7].

Recently, advances in unmanned aerial systems (UAS) have enhanced the ability of RS applications to provide very high spatial resolution data with more flexible and automated acquisition schedules, and typically at lower costs [4,5,8,9]. Moreover, the possibility of the integration of different sensors, multiple acquisitions, and optimized timing for the generation of orthophotos and dense point clouds [2,10] makes these systems suitable for several forestry purposes. For example, in a review conducted by Torresan et al. [5], UAS have been applied in activities for mapping and assessment of forest cover and dynamics, prediction of structural properties, monitoring of physiological and phenological changes, and assessment of ecosystem services and functions provided by these environments. Furthermore, many studies have demonstrated the benefits of using UAS-derived products on the accuracy of biomass estimates for forest inventories. For instance, Puliti et al. [9,11] verified that the integration of UAS data, even from external datasets, can improve models for forest parameter estimation from satellite data.

Since most applications of UAS consist of the spatial modeling of tridimensional parameters of trees or forest stands, the digital aerial photogrammetry (DAP) approach emerges as a promising method for acquiring such data. This technique is commonly used for reconstructing elevation models from overlapping images, using structure-from-motion and multi-view stereopsis, identifying repeated features in an array of images, and extracting 3D information based on camera positions [4,12,13,14]. The main advantages of DAP are the automatic identification and correspondence of features (tie-point matching) across images regardless of scale, angle, and camera orientation, iterative optimization of internal camera parameters along with processing [4], accuracy checking through redundancy among images [15], and also the incorporation of spectral information into point cloud datasets [4,16,17], which reduce errors in predictions from these datasets [10]. Hence, improvements in comparison to traditional stereo photogrammetry enable the production of 3D point clouds with similar quality to those generated by airborne laser scanning (ALS), such as light detection and ranging (LiDAR) sensors [18,19,20].

However, DAP still exhibits some limitations in the case of dense canopies and hence detecting ground points, which causes inaccurate estimates of treetops and canopy gaps [12,21]. Given the wide range of equipment, parameters, systems, sensors, and stands characteristics, it remains unclear how changes in flying altitude, photographic overlap, and resolution, amongst other factors, affect the quality of DAP processing and the quality of resultant processing products. Flying altitude (in combination with camera parameters) determines ground sampling distance (GSD) (i.e., image spatial resolution) and hence the detail level of scene objects [16], which is important for feature recognition within DAP frameworks. On the other hand, overlaps determine the number of varying view-angles of a given object within the scene as well as the degree of redundancy between images, which benefits feature matching [4]. In this regard, some studies have investigated the effect of different conditions and sampling and processing frameworks on the quality of DAP products (e.g., [16,22,23]). For example, Fraser and Congalton [22] analyzed the effects of flying altitude on DAP-outputs and verified that increasing altitude had a positive effect on image alignment and hence on the reconstruction of features within the scene. Furthermore, Dandois et al. [16] evaluated the influence of a set of factors (i.e., environmental conditions, flying parameters, and processing settings) on the accuracy of canopy height estimates and verified that the accuracy of these values was affected by overlaps and flying altitude. In their study, positional errors were increased by decreasing overlaps and increasing altitudes.

However, there is a scant understanding of how flight parameters affect data collection and processing. Our study aimed to evaluate the effects of flight settings on the quality of DAP processing, and the ability of resultant processing products to reproduce ALS information. Ultimately, in order to address the abovementioned shortcomings (i.e., limitation to detect features through dense canopies, the difficulty for reconstructing reliable terrain models), we propose an optimal combination of flight parameters that could be applied in future studies in our study area and other similar forested stands using similar UAS.

2. Study Area

Digital images were taken over a single 8.7-ha site located at Lahemaa National Park (LNP; 59°34′16″N 25°48′1″E) (Figure 1), employing varying combinations of flight altitudes and lateral/longitudinal overlaps. LNP is the largest national park in Estonia, with an area of 75,000 ha (of which 48,000 ha is land area), and its purpose is to protect the nature and cultural heritage characteristics of Northern Estonia. Forests cover more than 73% of the LNP area, with Scots pine (Pinus sylvestris L.) as the main tree species (80% of forest cover), and, more specifically, the landscape classified as a Cladonia pine heath system [24].

The flight area has a flat relief and is located over loamy soils that originated from Lower Cambrian Deposits [25]. According to the Forest Register Database of the Estonian Environmental Board [26], the main tree species in the stands is Scots pine (100% of individuals in the top layer). Stands are classified mainly as mature developmental stage; the average age of trees within the flight area is 105 years, varying between 84 and 165 years. The number of trees per hectare varies between 424 and 1136, the average height of tree individuals within the plots is 19 m (varying between 17 and 26 m), average diameter at breast height is 22 cm (from 19 to 26 cm), and average standing wood volume in the dominant layer between 208 and 286 m³/ha.

3. Methodology

3.1. Data Acquisition

Digital images were taken using an eBee Plus equipped with a sensor optimized for drone applications (SOD.A) (SenseFly, Cheseaux-sur-Lausanne, Switzerland). Aerial surveys were completed between and 7 and 17 June 2019. Surveys were conducted between 10:30 and 16:30 (local time) at these dates to reduce the influence of shadows due to low solar angles in the datasets [16]; temperature ranged from 16 to 30 °C, and average wind speed varied from 2.2 to 8.1 m.s⁻¹ (

Table 1. Description of flight settings and weather conditions during data acquisition provided by CustomWeather (CustomWeather Inc., Mill Valley, CA, USA) stations; AGL: above ground level.

Date	Flight ID	Local Time	Temperature Range (°C)	Wind Speed (m/s)	Wind Direction	Light Condition	Altitude AGL (m)	GSD * (cm)	Lateral Overlap (%)	Longitudinal Overlap (%)
07/jun/19	01	11:18	18–29	2.2	SE	Sunny	100	2.63	90	90
	02	11:50					100	2.70	80	90
	03	12:04	29–30	3.3	SE	Sunny	100	2.72	70	80
	04	14:40					120	3.08	70	80
	05	15:09					120	3.15	90	90
	06	15:36					90	2.57	80	70
	07	15:54					150	3.82	80	70
16/jun/19	08	13:42	22–23	4.4	NW	Passing Clouds	150	3.81	90	70
17/jun/19	09	10:30	16–24	3.6	W	Sunny	120	3.13	90	70
	10	11:00					90	2.45	80	90
	11	11:25					120	3.15	80	90
	12	11:42					90	2.49	70	90
	13	12:01	22–24	8.1	W	Sunny	90	2.57	70	70
	14	12:17					150	3.88	80	80
	15	12:33					120	3.12	90	80
	16	15:15					150	3.81	70	70
	17	15:32					100	2.71	80	80
	18	16:02					120	3.16	80	70

* Average ground sampling distance.

). The SODA is specifically designed for photogrammetric analysis and presents the following parameters: 20-megapixel sensor; 1” sensor size; 13.1328 mm sensor width; 8.7552 mm sensor height; a global shutter with speeds from 1/30 to 1/2000 s; focal length of 2.8–11/10.6 mm (35 mm equivalent: 29 mm); 5472 × 3648 pixels image size; ISO range from 125 to 6400; radial distortion parameters 0.003/-0.209/0.315; and exposure compensation in ±2 steps increments. The fixed-wing platform also had an on-board Global Navigation Satellite System Real-time Kinematic/Post-Processed Kinematic (GNSS RTK/PPK) receiver activated. Data recorded during the flights were subsequently post-processed using differential correction data (RINEX) provided by the Estonian Land Board’s virtual reference stations [27]. We opted for this approach based on Tomaštík et al. [21] findings, which reported higher horizontal and vertical accuracies using PPK than using ground control points (GCP).

Flight planning, operation, and post-processing were performed on eMotion 3 Release 3.5.0 (SenseFly, Cheseaux-sur-Lausanne, Switzerland), which enables the operation of autonomous flight missions and post-processing. Flights were planned along parallel lines with a north–south inclination. According to the manufacturer, the UAS is more resistant to side winds than to front/back winds. Therefore, real-time changes in wind direction during the flights could affect the actual spacing between images and cause slight deviations in the chosen overlaps [9]. In this regard, minor on-site adjustments for wind direction were made before the flight. This last step was conducted to ensure consistency in image capture spacing, motion blur, and to reduce impairments on subsequent processing effectiveness. The experimental design of flights was based on combinations of flight altitude, as well as lateral and longitudinal overlaps. Flight settings were designed to replicate other studies using eBee platforms in forest environments (e.g., [9,11,17,21,22,28,29,30,31]).

The literature reports flight altitudes ranging from 50 m above canopy level [22] to 325 m above ground level (AGL) [29,30]. Many studies use flight altitudes around 120 m AGL, which is a common UAS flight ceiling height set by some national aviation authorities. Thus, we assessed the following flying altitudes (hereafter called ALT) above ground level: 90, 100, 120, and 150 m (i.e., maximum allowable flight altitude in Estonia). It is noteworthy that ALT values below 90 m were not assessed in the current study due to the potential for loss of visual line-of-sight (VLOS) (as similarly reported by Fraser and Congalton [22]). Loss of VLOS and potential loss of communication between the UAS and the ground control station is more probable in forested system and/or areas of significant topographic relief.

In addition, other studies have reported both lateral and longitudinal overlaps between 70% [22] and 95% [17] for forestry applications. According to the settings reported in previous studies, we carried out flights with 70%, 80%, and 90% lateral and longitudinal overlaps (LAO and LOO, respectively). Theoretically, lateral and longitudinal overlaps are expected to affect similarly DAP processing [15]. However, due to varying resistance of the UAS used in this study to wind direction, we opted by analysing LAO and LOO separately. In order to evaluate its potential effects on data processing, we randomly selected 18 from 36 possible combinations with the selected settings (4 ALT × 3 LAO × 3 LOO) for conducting actual flights (Table 1). To avoid confusing the effect of flight settings with weather and light conditions, the order of flights was also randomly defined.

Finally, the Estonian Land Board’s ALS dataset (overall accuracy of ≈0.03 m) [32] was used as reference data for comparisons. The ALS data were measured in spring 2018 using a Riegl VQ-1560i scanner. Specifications of the Estonian Land Board’s 2018 flight mission are described in

Table 2. Overall description of the Estonian Land Board’s 2018 flight mission using Riegl VQ-1560i scanner [32].

Specification	Value
Altitude	2600 m
Flight Speed	278 km h−1
Wavelength	1064 nm
Scan Frequency	2 × 350 kHz
Field of View	58°
Mean Point Density	2.1 points m−2
Mean Point Spacing	0.72 m

.

3.2. Data Processing

For image processing and generating densified point clouds from each flight, we used the Pix4D Mapper v. 4.3.27 (Pix4D, Lausanne, Switzerland). The following workflow settings were implemented: for initial processing, a full tie-point image scale was set; for point cloud densification, point density was set to “High” (slow processing), a minimum of three tie points per processed image was required, and a 0.5 image scale based on image quality was used with a multi-scale view. The choice to use a high-density option was based on the findings of Dandois et al. [16] and Fraser and Congalton [22], who indicate that increasing point cloud density minimizes occlusions in scene reconstruction and decreases output data variability. For all processing steps, we used a workstation with an Intel Core i7-8700K CPU at 3.7 GHz, with 16 GB of RAM, and NVIDIA Quadro P1000 and Intel UHD Graphics 630 graphics cards. Processing all 18 datasets required in total 15.3 h for initial processing, and point cloud densification.

Similar to Fraser and Congalton [22], we considered the following processing quality parameters to describe the performance of photogrammetric processing and the quality of dense point cloud generation: the number of acquired images per flight (hereafter called IMG), image alignment (ALG), camera optimization (OPT), the number of tie-points per image (TIE), matching per image (MAT), average re-projection error (REP) in pixels, and average point cloud density (PCD). ALG refers to the proportion of images successfully calibrated and admitted for subsequent steps. OPT refers to the difference in the camera’s focal length before and after processing. In this way, OPT provides a measure of how well the actual camera settings fit the software camera model, using it for geometric calibration. TIE denotes the amount of automatically identified features in each image that can be used for linking different images. MAT relates to the presence of tie-points that can be identified in different images (at least three, as set in processing workflow) and is included in model construction; this metric was represented by the median number of matches per calibrated image. REP refers to the difference of distance between observed and projected 3D points, usually dependent on the quality of camera geometric calibration [33]. PCD refers to the average number of 3D points per cubic meter obtained for each dataset. In contrast to Fraser and Congalton [22], we opted to not include GSD as a quality indicator, since it was used to set the altitude during the flight planning and thus would not be independent. Respective values of the indicators were gathered from the Quality Report generated by Pix4D.

DAP point clouds were filtered using the statistical outlier removal (SOR) filter and classified using the cloth simulation filtering (CSF) algorithm within the CloudCompare [34] interface. The SOR filter estimates statistical parameters in the surroundings of each point in the point cloud and filters those that fall outside the standard range (i.e., mean ± standard deviation) of the set of points. Based on Zhang et al. [35], we set the SOR-filter’s neighbours to 50 adjacent points and used one standard deviation. The CSF algorithm inverts the point cloud along the z-axis, fits a surface to the highest points of the inverted point cloud, and classifies the points as ground or not ground according to a distance threshold. Based on Klápště et al. [36] sensitivity analysis, we applied the “Flat terrain” option, enabled slope processing, and set the cloth resolution to 0.1 m, and classification threshold to 0.1. In such a way, points were classified either as “ground” or “off-ground”. After this, DAP and ALS point clouds metrics were processed using “lidR” package [37] for R software [38], which produces outputs in raster format. Some often-used metrics, such as maximum and mean height, percentage of points relative to a specific height threshold, percentiles, and variability indicators, were calculated (

Table 3. Description of the selected point-cloud standard metrics from “lidR” package.

Metric Name	Description
Hmax	Maximum height
Hmean	Mean height
SDH	Standard deviation of height distribution
SKEWH	Skewness of height distribution
KURTH	Kurtosis of height distribution
PH>mean	Percentage of points above Hmean
PH>2m	Percentage of points above 2 m
Pxth	xth percentile of height distribution (in 5% steps; e.g., P5th, P10th, P15th)
PH=ground	Percentage of returns classified as “ground”

). Digital Terrain Models (DTM) and Canopy Height Models (CHM) were also constructed. Both DAP and ALS raster datasets were generated at a uniform resolution of 2 m; k-Nearest Neighbours with an Inverse Distance Weighting (KNN-IDW) and p2r [37] interpolators with their default parameters were used for the generation of DTMs and CHMs, respectively. DAP point cloud height distributions were normalized using two different ground references: (1) using their respective DTM to subtract point cloud height values, and (2) using ALS-DTM to subtract DAP-derived point cloud height values. The latter approach was conducted to assess the applicability of DAP-based CHM in cases where ALS-DTM is available.

3.3. Statistical Analyses

Initially, we performed an exploratory data analysis using Spearman’s correlation coefficient (r) to assess relationships across the selected quality indicators (i.e., IMG, ALG, OPT, TIE, MAT, REP, and PCD). For regression analysis, a stepwise procedure was applied for the selection of independent variables; ALT, LAO, and LOO were set as independent variables. The procedure started with a full model (Equation (1)) with the placement of variables in both directions (i.e., removing or including each step), considering the Akaike Information Criteria (AIC) for the selection of models. The AIC provides an index that describes the amount of information explained by models, penalizing the inclusion of additional model parameters. In this way, it retrieves (adjusted to the number of parameters) a measure of model quality that is used to compare relative statistical models generated from a given dataset. Before multiple linear regressions, data were transformed whenever identified as non-normal by Shapiro–Wilk tests. In such cases, a Box–Cox transformation was used. For these analyses, the sampling unit was the processing results from each of the 18 flights.

$$y = {\beta }_0+{\beta }_1\times LAO+{\beta }_2\times ALT+{\beta }_3\times LOO+ϵ$$

(1)

where: y = dependent variable (i.e., quality indicators); ${\beta }_0$ = y-intercept (constant term); ${\beta }_1$, ${\beta }_2$, ${\beta }_3$ = slope coefficients for each independent variable (i.e., LAO, ALT, and LOO); ϵ = the model’s error term.

Linear regressions and equivalence tests were applied to evaluate whether flight settings influence DAP’s ability to generate point clouds with similar height distributions as ALS point clouds. For the purposes of this analysis, the sampling unit used was 100 randomly selected pixels, within the 2 m resolution gridded datasets, to allow for their application in a paired scheme on equivalence tests.

To assess the similarity between point clouds, DAP- and ALS-derived metrics (Table 3) were fitted using simple linear models (Equation (2)), in which an ALS metric was set as the dependent variable and its respective DAP equivalent was set as the independent variable. For example, a linear model fitted ALS-H_max (predicted value) as the function of DAP-H_max (observation). In this way, a similar procedure was carried out for the other metrics. Additionally, the quality of model fit was described according to the coefficient of determination (R²; Equation (3)), root-mean-square error (RMSE; Equation (4)), mean absolute error (MAE; Equation (5)), and mean squared error (MSE; Equation (6)) values.

$$y = {\beta }_0+{\beta }_1\times x+ϵ$$

(2)

where: y = ALS-estimated metric (e.g., ALS-H_max, ALS-H_mean, ALS-SD_H, etc.); ${\beta }_0$ = y-intercept; ${\beta }_1$ = slope coefficient; x = DAP-estimated metric (e.g., DAP-H_max, DAP-H_mean, DAP-SD_H, etc.); ϵ = the model’s error term.

$$R^2=1- \frac{{\displaystyle \sum }{\left({\widehat{y}}_i-y_i\right)}^2}{{\displaystyle \sum }{\left(y_i-\overline{y}\right)}^2}$$

(3)

$$RMSE= \sqrt{{\displaystyle \sum }({\left({\widehat{y}}_i-y_i\right)}^2)/n}$$

(4)

$$MAE= {\displaystyle \sum }\left|{\widehat{y}}_i-y_i\right|/n$$

(5)

$$MSE= {\displaystyle \sum }({\left({\widehat{y}}_i-y_i\right)}^2)/n$$

(6)

where: $y_i$= observation i value of the dependent variable; $\overline{y}$ = mean value of dependent variable; ${\widehat{y}}_i$ = model predictions; n = number of observations.

Equivalence tests assessed the dissimilarity between models, reversing the usually tested hypothesis (i.e., observations and predictions are equal). In these tests, the null hypothesis was that observations and predictions are different, which provides more robust evidence about similarity among datasets than other traditional methods (e.g., Student’s t-test, Mann–Whitney test, etc.) [39,40]. Equivalence tests were carried out using a regression-based framework (see more details in Robinson et al. [40]) using “equiv.boot” function from R package “equivalence” [41]; a detailed description of the procedures for equivalence tests is presented in Supplementary Material 1.

Finally, we evaluated whether flight setting variables affected the quality of the fit of the models used to compare ALS- and DAP-derived metrics for the equivalence tests using linear regressions and stepwise selection, similarly to previous analyses. Equation (7) details the linear models assessed in this analysis.

$$y = {\beta }_0+{\beta }_1\times LAO+{\beta }_2\times ALT+{\beta }_3\times LOO+ϵ$$

(7)

where: y = model fitting indicator (i.e., R², RMSE, MAE, MSE); ${\beta }_0$ = y-intercept; ${\beta }_1$, ${\beta }_2$, ${\beta }_3$ = slope coefficients for each independent variable (i.e., LAO, ALT, and LOO); ϵ = the model’s error term.

4. Results

4.1. Effects of UAS Flight Setting on Quality of DAP Processing

Table 4. Summary of quality indicators from the 18 flights performed using different settings.

Flight ID	ALT (m)	LAO (%)	LOO (%)	Images Acquired	Images Calibrated (%)	Camera Optimization (%)	Number of tie-Points (per Image)	Matches (per Calibrated Image)	Re-Projection Error (pixels)	Point Cloud Density (pts.m−2)
01	100	90	90	248	94	0.37	81,420	2504	0.19	411
02	100	80	90	128	73	0.39	81,980	2201	0.17	387
03	100	70	80	100	23	0.42	79,878	1169	0.14	386
04	120	70	80	82	34	0.35	80,466	1756	0.15	247
05	120	90	90	214	100	0.38	80,383	2371	0.17	225
06	90	80	70	159	100	0.35	66,441	2425	0.14	504
07	150	80	70	106	100	0.40	56,298	2337	0.14	106
08	150	90	70	208	100	0.43	80,620	3423	0.20	138
09	120	90	70	262	100	0.42	76,634	3306	0.16	207
10	90	80	90	153	77	0.40	78,711	2103	0.14	565
11	120	80	90	122	100	0.41	79,975	2313	0.15	247
12	90	70	90	100	57	0.36	79,477	2016	0.14	483
13	90	70	70	115	33	0.47	79,747	2066	0.16	432
14	150	80	80	109	100	0.41	81,298	2369	0.17	133
15	120	90	80	231	100	0.43	80,483	2333	0.18	228
16	150	70	70	82	54	0.36	81,070	1443	0.18	126
17	100	80	80	147	55	0.35	81,027	1468	0.16	362
18	120	80	70	127	89	0.36	78,950	1513	0.16	208

summarizes the quality indicators, from the 18 flights and the varying flight settings selected, based on the Quality Reports generated by Pix4D. In summary, the average coverage degree of flights (i.e., percentage of study area reconstructed by DAP models) was 71.1% of the total flight area, varying from 13.7% to 100%. According to Shapiro–Wilk tests, the OPT, MAT, REP, and PCD distributions did not differ significantly from normal distributions. IMG and TIE did not follow a normal distribution and were therefore transformed (lambda-hat of −0.52 and 18.82, respectively). ALG distribution also differed from normality, but Box–Cox transformation was not able to transform the data sufficiently to obtain a normal distribution. In this case, no further transformations were performed, and the original ALG values were used in the subsequent analysis.

Based on Spearman’s correlation coefficient (Figure 2), it was verified that ALG and MAT had a strong positive correlation (r = 0.83; p-value < 0.01), as well as TIE and REP (r = 0.75; p-value < 0.01). Other indicators showed no significant correlation with each other (i.e., p-value > 0.05). PCD had a moderate negative but non-significant correlations with ALG (r = −0.37; p-value = 0.13) and REP (r = −0.45; p-value = 0.06), while it was positive for OPT and MAT (r = 0.31; p-value = 0.21).

According to the regression model fitting (Equation (1)) for each dependent variable, the best model to explain ALG included both LAO and ALT with positive effects, although ALT’s slope was not statistically significant (p-value = 0.15). This model (Equation (8)) had an R² of 0.68. Variable selection indicated that LOO had a positive effect on TIE_BOX-COX; however, the linear model (Equation (9)) was not significant (p-value > 0.05). MAT was positively affected by LAO; the model (Equation (10)) had an R² of 0.48. Similar to the ALG model, LAO and ALT were selected for the REP model; however, only the first variable had a significant effect on REP. Even so, the model (Equation (11)) was significant and had an R² of 0.30. Finally, increasing ALT decreased PCD (Equation (12)), explaining 89% of its variability. Conversely, IMG and OPT were not affected by any of the independent variables based on the stepwise procedure.

$$ALG=-1.75+0.0277\times \mathrm{LAO}+0.0026\times ALT$$

(8)

$$TI{E}_{BOX-COX}=-4.26+0.05\times \mathrm{LOO}$$

(9)

$$MAT=-2215.37+54.86\times \mathrm{LAO}$$

(10)

$$REP= 0.045+0.0011\times \mathrm{LAO}+0.00024\times ALT$$

(11)

$$PCD= 1002.37-6.08\times ALT$$

(12)

In addition to these results, a visual check of point clouds showed an enhancement in the detection of the ground and high vegetation points with increasing LAO (Figure 3) and ALT (Figure 4), respectively.

4.2. Effects of UAS Flight Setting on the Point Cloud Metrics and Comparison with ALS

According to equivalence analysis of linear models, most of the assessed DAP-based metrics were not simultaneously proportional (i.e., regression slope equal to 1) and unbiased (i.e., regression intercept equal to mean observation of ALS-estimates) in comparison with ALS-based metrics; only the DTMs achieved both equivalences when considering a 75% region of equivalence (

Table 5. Summary of linear regression analysis and equivalence tests for digital aerial photogrammetry (DAP) and airborne laser scanning (ALS)-derived digital terrain model (DTM) heights; dots in equivalence tests refers to significant similarity.

Flight	Regression Slope	Intercept	R2	RMSE		MSE (m2)	MAE (m)	Slope Equivalence Test						Intercept Equivalence Test
				(m)	(%)			1%	5%	10%	25%	50%	75%	1%	5%	10%	25%	50%	75%
01 *	0.00	48.57	0.00	0.16	103.4	0.25	0.42												•
02	0.99	−15.95	0.91	0.10	32.3	0.02	0.11	•	•	•	•	•	•						•
03	0.95	−13.85	0.89	0.50	21.2	0.01	0.08	•	•	•	•	•	•						•
04	0.43	20.60	0.22	0.10	52.5	0.07	0.18												•
05	0.90	−10.16	0.90	0.16	31.8	0.02	0.10	•	•	•	•	•	•						•
06	0.96	−13.96	0.96	0.26	20.2	0.01	0.07	•	•	•	•	•	•						•
07	0.10	42.07	0.10	0.50	97.7	0.23	0.39												•
08	0.63	7.34	0.61	0.26	64.5	0.10	0.21												•
09	0.97	−14.90	0.98	0.17	12.9	0.00	0.05	•	•	•	•	•	•						•
10 *	0.00	48.94	0.01	0.09	102.5	0.25	0.42											•	•
11	0.97	−14.83	0.97	0.06	18.2	0.01	0.06	•	•	•	•	•	•						•
12	0.92	−11.33	0.91	0.15	33.0	0.03	0.11	•	•	•	•	•	•						•
13	0.87	−8.41	0.74	0.20	37.7	0.03	0.12					•	•						•
14	0.96	−14.43	0.92	0.48	29.2	0.02	0.10	•	•	•	•	•	•						•
15	0.74	0.49	0.71	0.31	55.7	0.07	0.17												•
16	0.96	−13.94	0.85	0.18	42.0	0.04	0.14	•	•	•	•	•	•						•
17	0.79	−3.27	0.74	0.27	52.6	0.07	0.20					•	•						•
18	0.77	−1.61	0.88	0.14	35.8	0.03	0.12												•

* Flights 01 and 10’s linear models were not statistically significant at the 95% confidence interval.

). DTMs generated from flights 02, 03, 05, 06, 09, 11, 12, and 14 were statistically proportional to ALS-derived DTM within the narrowest tested region of equivalence (i.e., 1%). On the other hand, increasing the region of equivalence for slope’s test to 75% allowed the validation of DTM derived from flights 13 and 17.

In contrast, the other metrics were only equivalent to ALS-derived estimates while testing the shifted intercept; this means that some DAP-metrics estimated mean values similar to ALS values. For instance, P_H=ground from flight 15 and P_H>mean and P_H>2m from flight 09 provided average values equivalent to its ALS respective while considering a region of equivalence of 25%. The CHMs from flights 07 and 09 (both derived from DAP-DTM), as well as from flights 01, 02, and 06 based on ALS-DTM, were similar ALS values. Within this region, only mean estimates of height percentiles from flights 07 and 09 using DAP-derived DTMs were equivalent to the mean value of height percentiles from ALS: H_max to P65^th in flight 09, and from H_max to P70^th in flight 07. Increasing the region of equivalence from 25% to 50% demonstrated the ability of DAP-data to describe top canopy layers with similar precision as airborne ALS. For example, CHM, P45^th, P50^th, and from P75^th to H_max in flight 05, all percentiles from H_max to P55^th in flight 09, and from H_max to P60^th in flight 07 showed to be equivalent to ALS average estimates using a region of equivalence of 50% when using DAP-derived DTMs. Flights 01, 02, 11, 14, and 18 also obtained similar values until P90^th using ALS-normalized point clouds. Finally, considering a 75% region of equivalence, flights 05, 07, and 09 using DAP-derived DTMs yielded equivalent average estimates until P40^th, P50^th, and P45^th, respectively. When using ALS-DTMs, flights 01, 06, 11, 14, 16, and 18 yielded equivalent estimates until P65^th, P65^th, P70^th, P75^th, P85^th, and P70^th, respectively.

However, DAP-derived CHMs were not proportionally equivalent to the ALS-derived ones, even using different DTMs for calculation. Scatter plots (Figure 5) showed that most of the flights with an altitude higher than 120 m and lateral overlap larger than 80% had a high proportion of points concentrating around the 1:1 line, especially in flights 05, 07, and 09 using DAP-based normalization (Figure 5e,g,i, respectively). In addition, CHM generated from different DTMs showed small differences regarding the quality of model fitting. For example, R² for DAP-CHM normalized by DAP-DTM in flight 07 was 0.419, while it was 0.421 for the DAP-CHMs normalized by ALS-DTM; R² was 0.217 and 0.271 for DAP-CHMs derived using DAP- and ALS-DTMs in flight 09; in flight 05, R² was 0.189 for both CHMs; and in-flight 15, R² was 0.192 for DAP-CHM using DAP-CHM. Other metrics showed the same pattern. Flight 07 had RMSE of 0.300 and 0.289, with DAP- and ALS-normalization, respectively, MAEs of 3.1 and 3.0 m, and MSEs of 23.8 and 22.1 m². In-flight 05, values of RMSE were 0.305 and 0.305, MAEs were 3.2 and 3.2 m, and MSEs were 24.6 and 24.6 m². In addition, flight 09 had RMSE of 0.258 and 0.258, with DAP- and ALS-normalization, respectively, MAEs of 2.6 and 2.6 m, and MSEs of 17.6 and 17.6 m². Furthermore, CHMs derived using different DTM sources showed low agreement between them (Figure 6), with correlation coefficients varying between −0.160 and 0.328.

When considering only the metrics that had at least one type of equivalence (i.e., shifted intercept and/or slope), we generated linear models to evaluate whether flight settings affected the quality of model fitting, indirectly affecting equivalence between DAP- and ALS-metrics. Based on these results (Figure 7), increasing LAO promoted decreasing RMSE, MSE, and MAE, and increasing on R² for both DAP- and ALS-normalized models. Additionally, ALT was included in RMSE, MSE, and MAE’s linear models, but only had a significant effect on R². LOO showed a significant negative effect on R² for the most superficial layers and a positive effect on MAE; both in point clouds normalized using ALS-DTMs. On the other hand, we verified that none of the fitting quality metrics of DTM equivalence models were significantly affected by flight settings (Figure 7).

Regarding the quality of height percentile models (i.e., Px^th), LAO presented the main effect on these metrics above the 40^th percentile; this variable increased R² values and led to a decrease in the other metrics. LOO revealed a degrading trend in the quality of fitting in top layers of 3D models. In contrast, ALT increased R² in intermediate percentiles of height distribution and was included with negative coefficients for explaining RMSE and MAE.

5. Discussion

5.1. Effects of UAS Flight Setting on Quality of DAP Processing Results

The results regarding the influence of flight settings demonstrated that LAO was usually the most influential variable affecting the quality of DAP processing. In this perspective, larger overlaps can enhance the completeness of the coverage of the study area and increase the number of detected features for the construction of point clouds. This is consistent with the findings described in other studies (e.g., [12,23]). For example, Ni et al. [23] observed a greater number of undetected areas in their 3D models in flights with 60% overlaps, which they attributed mainly to the magnitude of feature changes following different viewing angles. Furthermore, Jayathunga et al. [12] reported that undetected areas represented 0.6% of their flight area while using a 95% overlap.

Under those circumstances, the identification of common features between images may be benefited by a reduced variation of texture [23,42]. As a result, large overlaps should enhance the rates of automatic matching of features due to the higher degree of similarity between images [15]. According to Dandois et al. [16], the algorithm used for scene reconstruction requires a certain degree of stability in imagery for better performance, as directional blur can increase error and noise sources [43]. Consequently, the lack of consistency among feature characteristics can decrease matching success due to large errors in the tie-point association. In this regard, our results (Figure 2, and Equations (9)–(11)) demonstrated and confirmed the association between these variables. Thus, the effect of larger overlaps on the completeness of the coverage seems to be associated both with the greater number of viewing-angles and with reduced changes in the texture of a single feature.

In addition to these results, visual inspection of the DAP point clouds shows an enhancement of ground point detection with increasing lateral overlap. This effect can also be explained by an improvement in canopy penetration that larger overlaps promote. For example, Dandois et al. [16] observed that canopy penetration (measured by the coefficient of variation of the point cloud height values) increased from 15% to 18% (R² = 0.93) with LAO ranging from 20% to 80%. Similar to the patterns observed in our study, the authors attributed this improvement to the larger quantity of viewing-angles, which enabled the same locations to be captured more times and the production of additional key points on the ground beneath the forest canopy.

In this context, the detection of ground points in different images increases due to the additional information that was acquired from the forest floor. To illustrate, our exploratory analysis showed that the correlation coefficient between MAT and ALG was 0.83; similar associations were described by Leberl et al. [15] and Fraser and Congalton [22]. Nurminen et al. [18] also observed an increase in ground point density with flights using an 80% overlap compared to the 60% overlap. Hence, larger MAT values may be achieved with increasing overlaps, since there would be enough information for the matching algorithm. As a result, it would be possible to obtain an improved output with reduced distortions [15] and increased modeling accuracy.

Furthermore, other results showed that ALT had a moderate influence on quality indicators. This variable had a major influence on PCD (Equation (12)). However, this effect is simply a result of changing GSD, which affects the number of pixels per area unit [16]. Interestingly, a visual check of point clouds indicated that ALT might also affect the quality of processing through the reduction of sensitivity to textural changes in the canopy surface within the imagery, improving the detection of high vegetation points (Figure 4).

Some studies stated that reduced textural variability impairs output quality [14,16,44]. In this case, the effect arises from difficulties in calculating the magnitude and orientation of gradients on features and then in finding suitable key points for matching imagery; which impairs reconstruction. In this context, Westoby et al. [14] highlighted the challenges of 3D reconstruction of vegetation due to the high degree of homogeneity among images. In contrast, besides the short range of variation in pixel size in our study (2.45–3.88 cm; Table 1), our results suggest that the increase in ALT promotes the averaging of spectral information from surrounding features, which reduces the influence of light reflection from different viewing-angles in vegetation surface. Ni et al. [23] reported similar findings. The authors verified not only that overlaps influence occlusions (as already mentioned), but also that increasing pixel size was beneficial to decrease the extent of undetected areas. Altogether, the disagreement about the effects of texture and the lack of detection of canopy points in flights lower than 100 m suggests that there is an optimal ALT (or GSD) for the adequate reconstruction of this layer, in which textural homogenization is positive for the identification of common features in image dataset and for the addition of texture to canopy surface by matching GSD and the scale of features; our results indicate that this optimal ALT value is around 120 m above ground level. Furthermore, it should be mentioned that correlations between PCD and the other processing quality parameters can be affected differently with varying ALT values. In such a way, future research should focus on the effects of flight parameters on the quality descriptors and the feedbacks between those metrics.

5.2. Effects of UAS Flight Setting on the Ability to Reproduce ALS Metrics

Following our previous findings, LAO had the main effect on the indicators of the quality of the models comparing DAP- and ALS-derived metrics. As shown in Figure 7, we verified that increasing LAO reduced all metrics describing the deviation from models, usually with a more explicit effect in models that described the upper layers of the forest canopy. A similar effect was verified by other studies (e.g., [16,18,23]), which also identified a positive effect of LAO on the ability to detect features through canopies. For example, Ni et al. [23] observed the effect of overlap on percentiles of height; in their study, a 60% overlap between images yielded the poorest estimations. However, the authors verified that overlap did not influence the estimation of height in percentiles lower than 80% (P80^th), in disagreement with our results where influence was evident until P40^th (Figure 7). In this perspective, our results (Figure 7) also indicate that LAO mainly affect the CHM’s quality, in contrast to canopy detection, which relied on ALT, but that was not included in models for the different metrics. In disagreement with our results, Nurminen et al. [18] indicated there is no bias in average height between DAP- and ALS-derived CHMs. Figure 5 showed that most of the DAP-normalized flights with an altitude higher than 120 m and lateral overlap larger than 80% had a high proportion of points concentrating around the 1:1 line. However, there was still a poor agreement between the estimations of CHMs, especially in ALS-normalized models.

In this context, equivalence between datasets may have been impaired by large deviations from this line. Four main factors appear likely to be causing these deviations: potential occurrence of fallen trees after ALS-data collection; the inability of DAP to adequately reconstruct canopy surface, especially small gaps; the potential displacement of treetops in windy conditions during the flights; and altitude misalignment between datasets. Deviations caused by fallen trees were not unexpected due to the temporal mismatch between the datasets, which likely leads to estimation discrepancies [23,45]. In that case, DAP-derived CHMs would detect lower height values than ALS-derived ones in these areas. On the other hand, the difficulty to reconstruct small gaps can be attributed to the low penetration of DAP through them, and the low representation of adequate ground points. A similar effect was observed by Jayathunga et al. [12], which promoted consistent deviations from ALS-derived CHMs. Additionally, the potential displacement of treetops is in agreement with the correlation between TIE and REP from the exploratory analysis. In such a case, as the number of detected common features (TIE) increases, an increasing REP should be observed due to the variable position of the specific feature in each image. This phenomenon indicates that the applied GSD may not be adequate for DAP processing in pine forests in windy conditions. In such cases, excessive displacement due to windy conditions may obscure the detection of specific trees or increase blurring of the treetops due to excessive movement [17] and thus cause errors in the model. It is consistent with the findings of Ni et al. [23], in which degrading image resolution had a positive effect on the reconstruction of the canopy surface. Lastly, Figure 5 and Figure 6 and Table 5 illustrated the misalignment between datasets, showing that ALS-normalized CHMs had displacements between 10 and 20 m in comparison to the ALS-CHM. Additionally, ALS-normalized predictions were biased compared to the DAP-normalized (Figure 6). For this reason, flights 01, 02, 11, and 18 presented equivalent average estimates to the ALS-CHM, even though these flights had poor reconstruction of the canopy surface. Therefore, the errors observed in DAP-normalized CHMs were likely caused by the occurrence of fallen trees, inadequate reconstruction of gaps, and displacement of treetops. On the other hand, ALS-normalized CHMs were additionally affected by the altitude mismatch between DAP- and ALS-datasets.

Altogether, the effect of LAO on CHM’s model fitting quality can be attributed to an improvement in the positioning accuracy of individual points in DAP-based point clouds [16]. Equations (8) and (9) corroborate the effect of both overlaps on TIE and MAT. In this way, a higher TIE value during processing has the potential to generate a structure more resistant to reconstruction errors [15]. In this manner, our results suggest that canopy surface modeling is not only affected by ALT (concerning the detection) but also by LAO on the accuracy of its reconstruction. Therefore, flight settings affect the quality of DAP-derived CHMs, but a more conclusive statement about their validity requires the use of more contemporary ALS datasets for analysis.

Finally, DAP- and ALS-derived DTMs had moderate to good agreement between them; nine out of 18 flights had R² > 0.70 (Table 5). However, the observation of proportionality in DTMs was not a surprise, since the terrain should be the most stable feature in this environment between different sampling periods. Thus, our DAP-flights were able to replicate forest floor height variability with a high level of detail and accuracy equivalence to the previously derived ALS ones and should provide reliable estimations of ground profile over relatively flat reliefs. These findings are contrary to other studies (e.g., [19,45]), which identified better performance in ALS-derived models. Moreover, this improved performance by ALS was attributed to an improved canopy penetration capacity of the latter. However, surprisingly, the quality of these models was not affected by any of the assessed flight settings. Therefore, in addition to the influence on the detection of ground points, our results indicate that different flying settings may generate DTMs with similar accuracy.

5.3. Implications for Data Acquisition and Processing in Forest Environments

This study has provided general insights into the effects of flight settings on the quality of DAP processing and model outputs for future research and applications in similar forest stands. The results and main findings can be summarized to highlight the benefits that increased levels of image overlap have for more comprehensive scene reconstruction, the value of high altitude flights for canopy surface detection, and the combined effect that these settings can have on the quality of the results. Firstly, the improvement in the completeness of coverage provided by large LAO values is a key finding for more effective and detailed datasets for forestry purposes, given that less information would need to be interpolated during 3D modeling of the scene [22]. The results suggest that LAO should be maximized to provide better quality data for subsequent analyses, expanding the potential applications of UAS-based imagery. Additionally, a further positive effect of the coverage degree and the acquisition of extra, redundant features, is that datasets with high overlaps should be less dependent on GCPs for accurate geolocation of scene reconstruction [15], which has often been proved to be challenging in forest environments [21].

Secondly, our results have illustrated that is possible to detect both forest floor and canopy surface in a single acquisition by choosing the appropriate flight settings. Using 15cm-resolution imagery, Nurminen et al. [18] were only able to reconstruct canopy surfaces in their study, whilst the terrain and understory vegetation produced unreliable models. In contrast, Ni et al. [23] had findings similar to ours; the authors obtained simultaneous detection of both ground and canopy surfaces when using high-resolution images and flight designs with large overlaps. Maximizing LAO and ALT has been shown to be effective in providing a more complete characterization of the stand and reduction of model errors [16]. Moreover, given that DAP-derived DTMs were shown to be of a similar quality to ALS-derived ones, they have the potential to be applied in future research about forest floor processes, such as on deadwood dynamics [46], with the benefit of including optical information for the point clouds. However, the improvement in the detection of ground points could only be achieved due to the relatively low canopy density in our study area. This low canopy density allowed more viewing angles of the ground surface beneath the forest canopy. In contrast, the reconstruction of DTMs in denser forest stands or more complex reliefs may remain problematic, since it relies on detecting features through scarce canopy gaps [12,21]. For instance, Puliti et al. [17] reported large errors in steeper areas in comparison to flatter ones. In such cases, alternative methods and sensors (e.g., ALS) should be considered for data acquisition to improve the reliability of estimations.

Lastly, our results suggested that ALS-derived CHMs still outperformed DAP-derived ones. Model fitting suggested that ALS-normalized CHMs performed slightly better than DAP-normalized models. However, none of the DAP-based CHMs retrieved proportional estimations of height variability when compared to ALS-CHM. If DAP-derived CHMs were not proportional, whilst comparing pixel-based estimates, some flight setting combinations produced CHMs with no differences between average values compared to ALS-CHMs for a region of equivalence of 25%. The application of these models using an Area-Based Approach (ABA) seems to be a potential alternative for operationalizing the use of these products for forestry or carbon stock assessment purposes. Modeling of forest inventory variables using ABA was shown to produce robust estimators of these variables (e.g., [47,48]). Nevertheless, it is worth mentioning that the use of ALS-normalized CHMs with this approach may be impaired by the observed displacement between datasets. Therefore, future research should assess the potential use of DAP-derived point clouds for replicating ALS products, especially CHMs, as well as using datasets collected at the same period and phenological stage. As discussed, this would facilitate a further and better evaluation of the flight setting effect on these products and reduce stochastic uncertainties.

In summary, these findings demonstrated that careful planning and selection of flight mission settings are fundamental to provide information that is fit for purpose, and directly addresses research objectives. Furthermore, this paper has identified that the optimal combination of flight mission settings, to be used in future studies at LNP and similar forest stands, should include a 90% LAO, a 70% LOO, and a minimum ALT of 120 m. However, it worthy to mention that this combination might be limited to the same UAS and sensor used in this study. These flight parameters would ensure that both forest floor and canopy surface could be sampled with a quality closer to ALS data, independent from the availability of an ALS-DTM for data normalization. Nevertheless, it is necessary to keep in mind that the findings of this study are to some degree limited to the type of UAS platform deployed and the sensor payload used; and, to a degree, the forest structure. Moreover, DAP-CHM estimations still showed considerable disagreement when comparing with ALS-CHMs, which should be addressed in future investigations. Likewise, many other factors may affect DAP processing, such as cloud cover, solar angle, and terrain, amongst others [16,20], requiring full consideration of the factors influential in flight planning to maximize the quality of DAP results. Additionally, we suggest not using IMG as a descriptor of processing quality, since the flight line inclination could also affect the number of lines covering the entire flight area. In this regard, the same combination of flight parameters would provide a varying number of images depending on the line inclination.

6. Conclusions

In this study, we have assessed the effects of flight settings on processing results and the quality of DAP products, comparing its equivalence to an ALS point cloud. The results have demonstrated that LAO and ALT were the most influential factors in DAP data quality in this analysis. Whilst ALT appeared to have a moderate influence on textural changes in image acquisition, which improved the detection of features in the canopy surface, LAO was shown to be influential in affecting data quality, through higher redundancy of detected features, a reduction in the extent of undetected areas in the scene reconstruction, and improvement of the detection of ground points. It is proposed that these variables should be cautiously considered in the planning of the acquisition of digital images to retrieve more cost-effective and accurate modeling outputs and products.