Improved LSTM Model for Boreal Forest Height Mapping Using Sentinel-1 Time Series

Abstract

Time series of SAR imagery combined with reference ground data can be suitable for producing forest inventories. Copernicus Sentinel-1 imagery is particularly interesting for forest mapping because of its free availability to data users; however, temporal dependencies within SAR time series that can potentially improve mapping accuracy are rarely explored. In this study, we introduce a novel semi-supervised Long Short-Term Memory (LSTM) model, CrsHelix-LSTM, and demonstrate its utility for predicting forest tree height using time series of Sentinel-1 images. The model brings three important modifications to the conventional LSTM model. Firstly, it uses a Helix-Elapse (HE) projection to capture the relationship between forest temporal patterns and Sentinel-1 time series, when time intervals between datatakes are irregular. A skip-link based LSTM block is introduced and a novel backbone network, Helix-LSTM, is proposed to retrieve temporal features at different receptive scales. Finally, a novel semisupervised strategy, Cross-Pseudo Regression, is employed to achieve better model performance when reference training data are limited. CrsHelix-LSTM model is demonstrated over a representative boreal forest site located in Central Finland. A time series of 96 Sentinel-1 images are used in the study. The developed model is compared with basic LSTM model, attention-based bidirectional LSTM and several other established regression approaches used in forest variable mapping, demonstrating consistent improvement of forest height prediction accuracy. At best, the achieved accuracy of forest height mapping was 28.3% relative root mean squared error (rRMSE) for pixel-level predictions and 18.0% rRMSE on stand level. We expect that the developed model can also be used for modeling relationships between other forest variables and satellite image time series.

1. Introduction

Timely assessment and monitoring of forests forms the basis for the definition and implementation of preventive and corrective measures for sustainable forest management and forest restoration after disturbances [1]. The dynamics of forest structural variables provides information on forest status and forest changes and represents key information for forest management purposes [2,3]. Traditional forest inventory variables, such as tree height, basal area, diameter at breast height and others can be used as inputs for forest biomass and carbon stock estimation, etc. Furthermore, many users require information on traditional forest inventory variables as well, for example, private forestry companies with smaller areas of interest, to support their forest management decisions.

Satellite-based operational forest inventories often use satellite optical data augmented by reference plots to produce forest maps and estimates [4,5]. When the use of optical satellite data is compromised due to near-permanent cloud coverage, a possible solution is to use synthetic aperture radar (SAR) sensors, relying on longer and denser time series. Radar-based monitoring offers flexibility in forest monitoring applications, when results are requested in a fixed (e.g., yearly) schedule. Primary SAR data used in forest inventorying are L-band data because of the smaller saturation of the biomass-to-backscatter relationship compared to shorter wavelengths [6,7,8]. The use of advanced SAR temporal and textural features and imaging modes can improve the accuracy of forest variable prediction [9,10].

The European Copernicus program has opened new opportunities in forest mapping with the launch of Sentinel-1 satellites thanks to their high spatial and temporal resolution, their ability to form long image time series and their data provision at no cost to users [11]. The Sentinel-1 mission consists of a constellation of two polar-orbiting satellites mounting a C-band SAR imaging system. They offer a repeat cycle of six days with all-weather and day-and-night monitoring capabilities.

Multitemporal C-band SAR data were extensively used for evaluating and monitoring growing stock volume of both boreal and tropical forests, as well as in thematic mapping purposes [12,13,14,15,16,17,18,19,20]. The present consensus is that further research is required on methods exploiting dense time series of C-band SAR measurements, including multitemporal approaches, to achieve performance similar to L-band SAR data [8]. Important issues that need improvement are the relatively poor prediction accuracy and the lack of consistent ways to use SAR time series data [9]. One popular approach is to use multivariable models where each measurement/observation from time series is treated as an independent classification feature (or predictor variable in regression tasks). Such approaches are quite popular within machine learning and statistical non-parametric methodologies and have already been demonstrated with Sentinel-1 time series (or similar C-band sensors) in forest variable prediction and classification [10,15,21,22,23]. However, such approaches ignore explicit temporal dependencies between consecutive images and at best use only multitemporal variability as an independent classification feature [23]. As C-band SAR data have pronounced multitemporal variability and seasonal dependencies in the boreal zone [24,25,26], our expectation is that introducing multitemporal dynamics into prediction models can improve prediction accuracies. One possible solution to introduce an explicit temporal context is using Long Short-Term Memory (LSTM) models that can capture temporal relationships between consecutive images (timestamps).

LSTMs were previously demonstrated in several remote sensing applications, particularly land cover mapping and crop monitoring [26,27,28,29,30,31,32,33,34]. However, to date, the use of LSTMs in forest attribute prediction utilising Earth Observation (EO) data was limited if at all reported with SAR image time series. Additionally, the conventional LSTM still suffers from the gradient vanishing problem, making it less suitable for learning relationships using long image time series [35]. The presence of irregular time intervals between consecutive acquisitions due to, e.g., sensor malfunction or maintenance makes modeling even more complicated. Another general issue is the lack of supervision data, which could require semi-supervised approaches in model training. In this context, while consistency regularisation has demonstrated a success in semi-supervised classification tasks [36,37], there appear to be only a few reports on its successful use in regression models for predicting continuous forest variables [38].

In this manuscript, our primary aim is to develop an improved LSTM-based model for producing forest variable predictions using time series of Sentinel-1 data. In particular, we consider non-regular sampling of image datatakes and introduce a novel LSTM model that naturally takes the varying time variable into account. We compare our approach with the classical LSTM model and several other pixel-based regression approaches that are often used in satellite-based forest inventory. We use forest tree height as a representative forest structural variable in our paper.

The main novel contributions of our work are:

• We introduce timestamps as an additional feature to better capture the relationship between SAR backscatter and forest variables. To accommodate possible irregular intervals between image datatakes, we propose a novel Helix-Elapse (HE) projection to explicitly model the circular seasonality pattern of the long image time series.
• We introduce Skip-LSTM, a hybrid LSTM block featuring with skip-link structure to better capture long-term dependencies in time series data.
• We employ a novel semi-supervised strategy called Cross-Pseudo Regression (CPR) to improve the model prediction performance with limited reference data.
• We benchmark the developed improved LSTM model with other state-of-the-art versions of LSTM, as well as more conventional machine learning and statistical models, for the purpose of most precise forest height predictions using Sentinel-1 time series data. To the best of our knowledge, this is the first use of LSTM modeling of any kind for the purpose of forest inventory mapping.

The paper is organised as follows. We describe our study site, acquired Sentinel-1 time series dataset and developed modeling approaches in Section 2 and Section 3. The performance of the developed model is analysed and compared to several benchmark approaches in Section 4. The computational complexity of the model is discussed in Section 5, as well as the impact of different stand sizes and dominant tree species on stand-level predictions, while potential challenges and opportunities are outlined in Section 6.

2. Materials

2.1. Study Site

The study site featuring the Hyytiälä forestry station is located in Central Finland, centre coordinates 37°2′N, 6°11′E (WGS84). It represents a square area covering 2500 km${}^2$. The location of the site is shown in Figure 1 along with an RGB color composite of three Sentinel-1 images. Typical southern boreal forest types are present in the study area, such as Norway spruce (Picea abies), Scots pine (Pinus sylvestris) and birch (Betula pendula, Betula pubescens). The terrain is generally flat with the elevation ranging from 95 m to 230 m above sea level. Frozen conditions often start in October in this area. The minimum temperature in winter can drop to −25 °C. The first snow often falls in November and melts away completely by early May. The snow layer depth can reach from 20 cm to 70 cm depending on weather conditions. Weather conditions over the study area are shown in Figure 2 according to the information from Finnish Meteorological Institute.

2.2. SAR and Reference Data

Our dataset is represented by a long time series of Sentinel-1A IW-mode backscatter intensity images. Overall, there were 96 dual-polarisation (VV+VH) images acquired from 9 October 2014 to 21 May 2018. The time interval between adjacent observations ranges from 12 to 36 days. The original SAR data were acquired with Sentinel-1 satellites and initially preprocessed (focused and detected) and distributed by the European Space Agency (ESA) as ground range detected (GRD) products. SAR image orthorectification and radiometric normalisation were carried out using VTT in-house software [39]. Multilooking with a factor of 2 × 2 (range × azimuth) was performed before orthorectification. Radiometric normalisation with respect to the projected area of the scattering element was performed to eliminate the topography-induced radiometric variation [40]. In this way, a time series of coregistered “gamma-naught” backscatter images was formed, with a pixel size of 20 m by 20 m. Each image in the stack has size of 2500 px × 2500 px corresponding to an area of 50 km × 50 km.

Airborne laser scanning (ALS) data collected by National Land Survey of Finland in summer 2015 are used as reference data. The forest height was computed by averaging relative heights of ALS cloud points after the ground removal within each mapping unit. The height ranges from 0 m to 25.5 m and the mean is 11.2 m. For additional comparisons with other conventional methods, stand-level estimates were also calculated from ALS data using a forest stand mask from the Finnish Forest Centre.

We split the dataset and corresponding reference into three subsets: training, validation and testing as shown in Figure 1c. Firstly, the whole area was equally divided into nonoverlapping tiles, the size of each tile was 128 px × 128 px. Further, as depicted with red colour in Figure 1c, 50% of the tiles were randomly selected and pixels within them were extracted as the test subset. In a similar way, 10% of tiles were used to populate the validation subset and pixels from the remaining area composed the training dataset. The numbers of pixels are 1.5 mln, 0.375 mln and 1.778 mln for training, validation and testing subsets, respectively.

3. Methods

Here, we first briefly describe the fundamentals of LSTMs and introduce a Helix-Elapse (HE) projection concept to deal with the non-regular time interval between image timestamps. Further, we describe a semi-supervision regression and develop a model that combines several mentioned approaches. Baseline models that will be used in the benchmarking are also briefly described in this section.

3.1. Long Short-Term Memory Networks

Recurrent neural network (RNN) can capture temporal dependencies from a time sequence [41] thanks to its memory structure, in contrast to convolutional neural network (CNN) [42]. However, classic RNN structures suffer from gradient vanishing or explosion problems and fail to capture long-term dependencies. To handle this issue, a revised cell structure was proposed in [43]. As shown in the diagram in Figure 3c, the structure can be described as the “remember or forget” mechanism. Compared to the classical RNN cell, not only the previous hidden state ${\mathit{h}}_{\mathit{t}-\mathbf{1}}$ but also the previous memory ${\mathit{c}}_{\mathit{t}-\mathbf{1}}$ are fed into the current LSTM cell at timestamp t. They bring the history from previous timestamps into the current workflow. Then by integrating three gates (input gate ${\mathit{i}}_{\mathit{t}}$, forget gate ${\mathit{f}}_{\mathit{t}}$ and output gate ${\mathit{o}}_{\mathit{t}}$), the flow of information is regulated and fused and a decision on whether to keep it is made using activation functions $tanh$. Finally, the new current states, ${\mathit{c}}_{\mathit{t}}$ and ${\mathit{h}}_{\mathit{t}}$, are calculated and fed forward into the next timestamp.

The corresponding equations are mathematically described as

$$\begin{array}{cc}\hfill {\mathit{i}}_{\mathit{t}}& =\sigma \left({\mathit{W}}_{\mathit{i}\mathit{x}}{\mathit{x}}_{\mathit{t}}+{\mathit{W}}_{\mathit{i}\mathit{h}}{\mathit{h}}_{\mathit{t}-\mathbf{1}}+{\mathit{b}}_{\mathit{i}}\right),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\mathit{f}}_{\mathit{t}}& =\sigma \left({\mathit{W}}_{\mathit{f}\mathit{x}}{\mathit{x}}_{\mathit{t}}+{\mathit{W}}_{\mathit{f}\mathit{h}}{\mathit{h}}_{\mathit{t}-\mathbf{1}}+{\mathit{b}}_{\mathit{f}}\right),\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill {\mathit{o}}_{\mathit{t}}& =\sigma \left({\mathit{W}}_{\mathit{o}\mathit{x}}{\mathit{x}}_{\mathit{t}}+{\mathit{W}}_{\mathit{o}\mathit{h}}{\mathit{h}}_{\mathit{t}-\mathbf{1}}+{\mathit{b}}_{\mathit{o}}\right),\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\mathit{y}}_{\mathit{t}}& =\mathit{tanh}\left({\mathit{W}}_{\mathit{y}\mathit{x}}{\mathit{x}}_{\mathit{t}}+{\mathit{W}}_{\mathit{y}\mathit{h}}{\mathit{h}}_{\mathit{t}-\mathbf{1}}+{\mathit{b}}_{\mathit{y}}\right),\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\mathit{c}}_{\mathit{t}}& ={\mathit{i}}_{\mathit{t}}\odot {\mathit{y}}_{\mathit{t}}+{\mathit{f}}_{\mathit{t}}\odot {\mathit{c}}_{\mathit{t}-\mathbf{1}},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\mathit{h}}_{\mathit{t}}& ={\mathit{o}}_{\mathit{t}}\odot \mathit{tanh}\left({\mathit{c}}_{\mathit{t}}\right),\hfill \end{array}$$

(6)

where $\sigma$ and $tanh$ represent Sigmoid and Hyperbolic activation function separately, ⊙ denotes pointwise multiplication, $\mathit{W}$ and $\mathit{b}$ are weight matrix and bias vector.

3.2. Helix-Elapse Projection

Timestamps of acquisitions contain seasonal information and have explicit correlations with SAR image features, which may be considered as a priori knowledge for modeling. In particular, time series with irregular timestamps often happen in real world practice, for instance, the acquisition time interval differs from 12 to 36 in our case. How to alleviate the effect of irregular timestamps is an urgent problem. With timestamps as attributes, we can explicitly denote the acquisition interval differences and thus guide the training of the model.

We already knew that the seasonal pattern apparently exists in forest remote sensing images. Other than just bringing raw timestamps as linear attributes, we map them into a two-dimensional space, where the transformed timestamps can form a helix curve. The mapping is performed by an HE projection module, which can be mathematically described in the following equations:

$$\begin{array}{cc}\hfill {\mathit{t}}_{\mathit{h}}& =[t_1,t_2],\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill t_1& =t\ast sin\frac{2\pi t}{n},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill t_2& =t\ast cos\frac{2\pi t}{n},\hfill \end{array}$$

(9)

where t is the day index since 1 January 2014; n is the total number of days in a year—here, we simply assign it as 365; ${\mathit{t}}_{\mathit{h}}$ is the projected timestamp vector.

As visualised in Figure 4, let the origin point be the start date (1 January 2014); the circulating angle of the helix curve indicates the date of a year and the diameter indicates the growing year. The projected timestamps thus simulate the circulation of seasons in a heuristic way, as well as the year growing. We embed this HE projection module into our model and then stack the output, helix time attributes ${\mathit{T}}_{\mathit{h}}=[{\mathit{T}}_{\mathbf{1}},{\mathit{T}}_{\mathbf{2}}]$, as two additional vectors together with the original input.

3.3. Skip-LSTM

In order to capture temporal dependencies within the SAR time series, we introduce LSTM as the backbone of our model. Despite the carefully designed memory structure, LSTM in practice often fails to capture very long-term correlations [35]. In the case of satellite observations spanning several years, the time series would be particularly long. This type of long-term dependency can hardly be captured by off-the-shelf recurrent units. Inspired by Dilated Convolution in CNN [44] and Recurrent-skip in LSTNet [35], we embed a Skip-LSTM module to alleviate long-term correlation issues. The structure of Skip-LSTM is shown in Figure 5. It consists of two parts: convolutional layer and Skip-LSTM layer.

Consider the input time series $\mathit{X}$ with a size of $T\times N_f$, where the length of the time series is T and the number of input channels is $N_f$. A convolutional layer is first applied to capture short-term features in the temporal domain as well as possible correlations between input channels. The convolutional layer is composed by multiple convolutional filters that sweep through the whole time series and extract short-term features from the original input. The kernel size of each filter is $h\times w$, where h decides the range of short-term features and w is equal to $N_f$. After a rectified linear unit ($\mathit{ReLU}$) activation, the output has a size of $T\times N_k$, where $N_k$ is the number of filters.

Further, a Skip-LSTM layer is applied to the product of the convolutional layer. As shown in Figure 5, the skip-link structure of Skip-LSTM can jump over multiple timestamps, thus shortening the length of the time series. Following the skip-link, the raw time series are converted into a group of skip time series. Within each skip time series, non-adjacent timestamps get appended together. The total number of converted series depends on the skip-factor s. Then the new grouped time series are fed into the backbone LSTM module as the input. By means of this “dilated receptive field”, long-term periodic patterns are captured in the temporal domain at a larger scale.

3.4. Cross-Pseudo Regression

A novel Cross-Pseudo Regression (CPR) strategy [45] is further converted to a wall-to-wall regression task to allow training the model in a semi-supervised way, thus compensating for the possible lack of training data. Both labeled and unlabeled data are included in the strategy. The model is constructed using two branches with the same structure but initialised differently, as shown in the flowchart of Figure 6.

Firstly, we train the two branches separately in a normal supervised way, as shown in Figure 6a; only labeled data are taken into the training. Specifically, we select mean squared error (MSE) to measure the distance between the predictions and reference, as mathematically described in the following equation:

$$\mathrm{Loss}(\widehat{\mathit{Y}},\mathit{Y})=\frac{1}{n}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2,$$

(10)

where $\widehat{\mathit{Y}}$ denotes the prediction and $\mathit{Y}$ denotes the reference; n is the total number of the samples. So the supervised-loss can be denoted as

$${\ell }_s=\mathrm{Loss}({\mathit{P}}_1,\mathit{R})+\mathrm{Loss}({\mathit{P}}_2,\mathit{R}),$$

(11)

where ${\mathit{P}}_1$ and ${\mathit{P}}_2$ denote the predictions of Branches #1 and #2, respectively, and $\mathit{R}$ represents the supervised reference.

Secondly, we ignore the label information and treat all the data as unlabeled, as illustrated in Figure 6b. The results predicted by one branch can be naturally treated as pseudolabels of the other, instead of the real reference. The backpropagation process can still be carried out according to the cross-pseudo-loss, which is defined as

$${\ell }_c=\mathrm{Loss}({\mathit{P}}_2^{\prime },{\mathit{P}}_1^{\prime })+\mathrm{Loss}({\mathit{P}}_1^{\prime },{\mathit{P}}_2^{\prime }),$$

(12)

where ${\mathit{P}}_1^{\prime }$ and ${\mathit{P}}_2^{\prime }$ denote the pseudolabels of Branches #1 and #2.

From the viewpoint of consistency regularisation [46,47], different initialisations of the branches would bring perturbations to the model. When fed the same input, both branches are encouraged to predict the same results, even though the perturbation is imposed. By minimising the cross-pseudo-loss, the discrepancy between predictions of both branches would also be minimised. In this way, a more compatible and representative feature space is learned by the model.

Finally, we combine supervised-loss and cross-pseudo-loss together. The combined loss of CPR can be defined as

$$\ell ={\ell }_s+{\lambda }_c{\ell }_c+{\lambda }_w\frac{1}{n_w}\sum _{j=1}^{n_w}{\left(w_j\right)}^2,$$

(13)

where ${\lambda }_c$ controls the contribution of cross-pseudo-loss, which is simply set to 0.5 in our study. The $l_2$ regularisation item is added to the final loss to alleviate the overfitting, which is also known as the weight decay. $w_j$ denotes the j-th weight in the model, $n_w$ is the total number of weights and ${\lambda }_w$ decides the trade-off. On the prediction stage, we simply select one of the trained branches, transfer all its weights to the task model and give the final regression of the forest height.

3.5. Overall Structure of CrsHelix-LSTM Model

We refer to the proposed Cross Helix LSTM method as CrsHelix-LSTM for short. The overall architecture of the model is shown in Figure 7.

The model consists of two branches based on CPR strategy. In each branch, the helix time attributes are firstly mapped and attached to the input data. Then, the input is fed into two separate components to obtain comprehensive features in the temporal domain. Basic LSTM is used to extract the overall invariant features of the time series. A convolutional layer is utilised to extract short-term features, followed by a Skip-LSTM layer to capture the seasonal pattern at a larger scope. At last, the feature maps from both sides are stacked together. The regression header finally projects the features to the forest height.

The hidden sizes of LSTM and Skip-LSTM are set to 128 and for convolutional layer to 64. The kernel size of the CNN module is 5 and the skip-factor is set to 12. The total time step is 96, equivalent to the length of Sentinel-1 backscatter time series. To avoid potential overfitting, Dropout layers are implemented before the regression header. The Dropout coefficient is set to 0.5. At the training stage, the Adam optimisation algorithm is used to minimise the loss function. The batch size is 2048. The OneCycleLR learning rate strategy takes care of the training progress in cases of overfitting [48]. The training procedure is summarised in Algorithm 1.

Algorithm 1: Training Procedure of CrsHelix-LSTM

Input: The semi-labeled training time series $\mathit{X}={\mathit{X}}_l\bigcup {\mathit{X}}_u$; The reference $\mathit{R}$ of labeled subset ${\mathit{X}}_l$; Input: Iteration number K;  1: Differently initialise the parameters of both Helix-LSTM branches ${\theta }_1$, ${\theta }_2$.  2: for$iteration=0,\cdots ,K$do  3:    for branch $i=1,2$ do  4:      Feed $\mathit{X}$ into the branch;  5:      Extract features using Helix-LSTM, whose architecture is shown in Figure 7;  6:      Predict the inferences ${\mathit{P}}_i$ for labeled subset ${\mathit{X}}_l$, ${\widehat{\mathit{P}}}_i$ for unlabeled ${\mathit{X}}_u$, respectively.  7:    end for  8:    Calculate the supervised-loss ${\ell }_s$ according to Equation (11): ${\ell }_s=\mathrm{Loss}({\mathit{P}}_1,\mathit{R})+\mathrm{Loss}({\mathit{P}}_2,\mathit{R})$.  9:    Let ${\mathit{P}}_i^{\prime }={\mathit{P}}_i\bigcup {\widehat{\mathit{P}}}_i$, calculate the cross-pseudo-loss ${\ell }_c$ according to Equation (12): ${\ell }_c=\mathrm{Loss}({\mathit{P}}_2^{\prime },{\mathit{P}}_1^{\prime })+\mathrm{Loss}({\mathit{P}}_1^{\prime },{\mathit{P}}_2^{\prime })$. 10:    Summarise the combined loss of CPR according to Equation (13): $\ell ={\ell }_s+{\lambda }_c{\ell }_c+{\lambda }_w\frac{1}{n_w}{\sum }_{j=1}^{n_w}{\left(w_j\right)}^2$. 11:    Back-propagate ℓ, update ${\theta }_1$, ${\theta }_2$ using gradient descent. 12: end for 13: return${\theta }_1$ (or ${\theta }_2$) as the final model for testing.

3.6. Baseline Models

To access added value of developed models, several more traditional and widely used regression methods were used as baselines for comparison in our study. It should be noted that, similar to our models, all these methods operate on a pixel-level considering only temporal features. The following methodologies are included for comparison:

• MLR, RF, LightGBM
  Multiple Linear Regression (MLR) and Random Forest (RF) are mature regression methods, which have been widely used in forest remote sensing tasks. Light Gradient Boosting Machine (LightGBM) [49] is a modern Gradient Boosting Decision Tree (GBDT) model. Since being proposed by Microsoft in 2017, its precision and efficiency in regression have been proven in different application areas [50,51].
• LSTM, Attn-BiLSTM
  Since the proposed model is an improved version of LSTM, basic LSTM and its variant, Bidirectional LSTM with attention mechanism (Attn-BiLSTM) [33,52], are also included as baseline models for comparison. Bidirectional LSTM (BiLSTM) consists of two LSTMs with the same structure but opposite directions. Temporal dependencies are obtained from both directions. Furthermore, with the self-attention mechanism, attention weights establish the correlations between timestamps, which reportedly can better address the gradient vanishing problem and obtain long-term correlations [53]. Attn-BiLSTM combines both features and has been introduced into SAR remote sensing tasks [33].

To decrease the number of independent variables in MLR, a principal component analysis (PCA) module was applied before the modeling. This dimensionality reduction was not necessary for other methods, as they have built-in feature selection modules. All the models were fine-tuned with Optuna using 5-fold cross-validation. A 10% random sample of the training dataset was used in the fine-tuning.

3.7. Method Implementation

The overall methodology utilising the developed CrsHelix-LSTM model (as well as other benchmark models for comparison purposes) is illustrated by the flowchart shown in Figure 8. Sentinel-1 images are preprocessed as described in Section 2.2. The forest stand mask is applied to limit training and prediction to forested areas. ALS-based forest height reference data are used in model training and accuracy assessment. The training, validation and testing areas do not overlap after splitting is performed as described in Section 2.2. Importantly, even though Sentinel-1 images from the testing subset area is supplied into model training, those data are used as unlabeled for semi-supervised training purposes. Forest tree height predictions are compared to the reference data from testing areas on both pixel- and stand-levels.

3.8. Model Performance Accuracy Assessment

The prediction accuracy of various regression models was calculated using the following accuracy metrics, including root mean squared error (RMSE), relative root mean squared error (rRMSE), the coefficient of determination (R${}^2$), mean absolute error (MAE) and the index of agreement (IOA) [54]:

$$\begin{array}{cc}\hfill \mathrm{RMSE}& =\sqrt{\frac{{\sum }_i{\left(y_i-{\widehat{y}}_i\right)}^2}{n}},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill \mathrm{rRMSE}& =\frac{\mathrm{RMSE}}{\overline{y}}·100\%,\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\mathrm{R}}^2& =1-\frac{S{S}_{\mathrm{res}}}{S{S}_{\mathrm{tot}}}=1-\frac{{\sum }_i{\left(y_i-{\widehat{y}}_i\right)}^2}{{\sum }_i{\left(y_i-\overline{y}\right)}^2},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill \mathrm{MAE}& =\frac{{\sum }_i\left|y_i-{\widehat{y}}_i\right|}{n},\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill \mathrm{IOA}& =[1-\frac{{\sum }_i{\left(y_i-{\widehat{y}}_i\right)}^2}{{\sum }_i{\left(\left|y_i-\overline{y}\right|+\left|{\widehat{y}}_i-\overline{y}\right|\right)}^2}]·100\%,\hfill \end{array}$$

(18)

where $y_i$ is the reference forest height of pixel i, ${\widehat{y}}_i$ is the predicted forest height, $\overline{y}=\frac{1}{n}{\sum }_{i=1}^n{y}_i$ and n is the total number of samples. Stand-level estimates of forest height were calculated using spatial averaging at the extent of each stand (available from forest stand mask), with reported accuracies calculated using exactly the same Equations (14)–(18) for aggregated stand-level units. It is important to keep in mind that forest measurements for both training and accuracy assessment data were conducted in 2015, while SAR time series span more than three years. However, we primarily focus on relative performance of various prediction methods and exactly the same SAR and reference data are used for all benchmarked methods.

4. Experimental Results

4.1. Experimental Settings

The experiments were performed using Windows Server with Intel Xeon E5-2697 v4 CPU and NVIDIA GTX3060 GPU accelerated by CUDA 11.3 toolkit. LSTM, Attn-BiLSTM and the proposed model were built with a neural network library, Pytorch 1.11.0. MLR and RF were implemented with Scikit-learn machine learning toolbox. LightGBM was implemented with LightGBM Python-package provided by Microsoft.

4.2. General Performance Evaluation

We examined the performance of the developed model versus several other LSTM-based models, as well as a representative set of benchmarking models often used in forest mapping, on both pixel and stand levels. The obtained ablation results for LSTM based models are gathered in

Table 1. Ablation Study. LE stands for Linear-Elapse projection, HE for Helix-Elapse projection, Skip for Skip-LSTM block and CPR for CPR strategy.

		HE	Skip	CPR	RMSE (m)	rRMSE (%)	R${}^2$	MAE (m)	IOA (%)
Pixel-level	LSTM				3.26	29.16	0.49	2.51	80.53
	Attn-BiLSTM				3.22	28.87	0.50	2.48	81.11
	LSTM+LE	LE			3.26	29.16	0.49	2.52	80.28
	LSTM+HE	√			3.22	28.88	0.50	2.49	80.99
	Helix-LSTM	√	√		3.19	28.59	0.51	2.46	81.76
	CrsHelix-LSTM	√	√	√	3.16	28.31	0.52	2.42	82.46
Stand-level	LSTM				2.11	18.95	0.62	1.64	86.02
	Attn-BiLSTM				2.08	18.63	0.63	1.61	86.62
	LSTM+LE	LE			2.12	18.97	0.62	1.64	85.79
	LSTM+HE	√			2.09	18.69	0.63	1.62	86.47
	Helix-LSTM	√	√		2.04	18.26	0.64	1.58	87.32
	CrsHelix-LSTM	√	√	√	2.01	18.01	0.65	1.55	87.96

and the accuracy metrics for benchmarking models are shown in

Table 2. Experimental results compared to benchmarks.

		RMSE (m)	rRMSE (%)	R${}^2$	MAE (m)	IOA (%)
Pixel-level	MLR	3.50	31.38	0.40	2.74	75.40
	RF	3.45	30.91	0.42	2.74	72.73
	LightGBM	3.30	29.55	0.47	2.56	79.26
	LSTM	3.26	29.16	0.49	2.51	80.53
	Attn-BiLSTM	3.22	28.87	0.50	2.48	81.11
	CrsHelix-LSTM	3.16	28.31	0.52	2.42	82.46
Stand-level	MLR	2.37	21.22	0.52	1.85	81.21
	RF	2.38	21.36	0.51	1.90	77.68
	LightGBM	2.17	19.42	0.60	1.69	84.72
	LSTM	2.11	18.95	0.62	1.64	86.02
	Attn-BiLSTM	2.08	18.63	0.63	1.61	86.62
	CrsHelix-LSTM	2.01	18.01	0.65	1.55	87.96

. Examples of forest maps produced by various examined approaches are shown in Figure 9. Prediction performances are gathered and illustrated with scatterplots between predicted and reference forest heights in Figure 10.

4.2.1. Ablation Study

Firstly, we verified the effectiveness of different blocks in the ablation study, the results are shown in Table 1. In general, the forest height prediction at the stand level demonstrates a larger accuracy compared to the pixel level, taking advantage of averaging within the homogeneous stands. When we handled the irregular time intervals by using Linear-Elapse (LE) projection, the LE attribute was stacked onto the input as a new feature. The regression results were not improved compared to basic LSTM. This indicates the elapsing of time is not properly modeled in this case. On the contrary, by simply substituting the LE projection with our HE projection, RMSE is somewhat improved from 3.26 m to 3.22 m at the pixel level. The results are close to Attn-BiLSTM, which is considered a much stronger baseline when it comes to long time series tasks. This indicates that HE projection can better model the annual dynamics and help establish relationships between forest variables and seasonal patterns, thereby approximating the role of the attention mechanism at a lower computational cost.

When the Skip-LSTM block was embedded in the model, the regression performance of Helix-LSTM further improved at both the pixel and the stand level. The rRMSE is 28.59%, a 0.28% improvement from Attn-BiLSTM for the pixel level and 0.37% for the stand level. Considering Attn-BiLSTM is only better than basic LSTM for 0.29% and 0.32%, the improvement is quantitatively considerable. Finally, by wrapping up two Helix-LSTM branches with the CPR strategy, CrsHelix-LSTM obtained the best regression performance in the ablation study. The best rRMSE is 28.31% for the pixel level and 18.01% for the stand level, the best MAE is 1.55 m. IOA also obtains the optimal in this case as 87.96% and R${}^2$ of the final model is as high as 0.65. Note that its rRMSE is 0.28% further decreased compared to Helix-LSTM, even though the backbone models were the same. This indicates that a positive impact is imposed by the semi-supervised learning strategy, the model learns the forest representations from not only the labeled but also the unlabeled data.

4.2.2. Method Performance Comparison with Baseline Approaches

In Table 2, we also compare our methods to some existing machine learning models. MLR, which is most widely used in forest mapping, presents the most fundamental performance, which is 3.50 m of RMSE for the pixel level and 2.37 m for the stand level. Other approaches generally perform better than MLR; among them, the use of LSTM somewhat improves the prediction performance at both pixel and stand levels. While the developed CrsHelix-LSTM approach provided consistently better results compared to other benchmarking models. Its rRMSE for the pixel level reaches to 28.31%, 1.24% decreased compared to LightGBM. Note that the rRMSE of LightGBM is only 1.83% decreased compared to MLR. From this aspect, the improvement of our model is considerable. This is also confirmed by MAE, which is less sensitive to outliers. MAEs of CrsHelix-LSTM are 0.32 m improved for pixel level and 0.30 m for stand level compared to MLR.

By visualising the prediction results, as shown in Figure 9, we can better observe the prediction discrepancy over forests at different height levels. Compared to other methods, CrsHelix-LSTM is more sensitive to undergrowth forests whose height is within 5 m. The corresponding areas are showing as light brown according to the colormap, typical areas are highlighted with polygons in Figure 9. Similar observations can also be made from the analyses of the scatter plots shown in Figure 10, particularly on stand level. Compared to other benchmarking methods, the samples predicted by our CrsHelix-LSTM are more inclined to follow the diagonal line. Taller referenced samples around 20 m are less biased, which indicates a better prediction performance for taller forest stands.

5. Discussion

5.1. Computational Complexity Analysis

For computational complexity analysis, when using the floating point operations (FLOPs) as the evaluation criteria [55], for a dummy input tensor with dimensions 10,000 × 96 × 4, the FLOPs of LSTM, Attn-BiLSTM and CrsHelix-LSTM are 66.85 G, 133.70 G and 312.68 G, respectively. The FLOPs of CrsHelix-LSTM are approximately five times as large as LSTM, due to its dual branches and unique architecture.

However, such analysis is not applicable to conventional baselines like MLR or RF, as those methods usually utilise CPU while deep learning models are running in parallel accelerated by GPU and CUDA. With this in mind, we compared the processing times of different methods directly, including both training and testing times, as presented in

Table 3. Processing time of the proposed and baseline methods.

	MLR	RF	LightGBM	LSTM	Attn-BiLSTM	CrsHelix-LSTM
Training time, s	37.39	485.89	70.17	886.58	1326.10	4182.08
Testing time, s	0.036	33.21	40.17	80.13	66.23	72.05

. Although the prediction performance of MLR is modest, it uses the least amount of time for training, only 37.39 s. RF and LightGBM are next. It is worth noting that their computation complexity may vary strongly depending on the hyperparameters. For example, RF with more decision trees, node leaves and bigger depth would take considerably more time. All LSTM-based methods are time consuming. After the same number of epochs, the training time is generally consistent with their FLOPs. The proposed CrsHelix-LSTM takes 4182.08 s for training purposes. However, there is no need to retrain the model from scratch every time the mapping is needed. By using an already trained CrsHelix-LSTM model, the testing time is only 72 s, which is even less than basic LSTM.

5.2. Stand-Level Performance against Different Stand Sizes and Dominant Tree Species

Since stand-level averaging within the homogeneous stands can help improve the prediction performance, in this section we firstly analyse the effects of different stand sizes on stand-level prediction. As shown in Figure 11, we divide all stands into five subsets according to their size. There are 13,197, 7474, 4639, 2839 and 4912 stands for each area range, from less than 1 ha to over 4 ha. In general, the prediction accuracy improves with increasing stand size. However, the improvement becomes minor when the stand size is large enough. Such saturation is less pronounced with our proposed model. For each subset, CrsHelix-LSTM demonstrates the most accurate prediction accuracy, varying from 1.58 m RMSE for stands that are larger than 4 ha (R${}^2$ of 0.78 and MAE of 1.23 m) to 2.34 m RMSE for smaller stands under 1 ha.

Another factor that influences the stand-level prediction performance is the dominant tree species. In total, there are 1542 birch, 6701 pine and 12,194 spruce dominated stands in the testing area. The prediction performance for each strata is presented in Figure 12. Among the three studied tree species, tree height predictions for birch stands are somewhat worse compared to other species, while predictions of spruce dominated stands are the most accurate. Among all the methods, CrsHelix-LSTM still shows the best prediction accuracy for all studied tree species. It can be more clearly observed using R${}^2$ as an evaluation criterion.

5.3. Comparison with Other Studies and Outlook

Observed experimental results are encouraging further investigations and are generally in line with other reported studies in boreal forest biome [3,8,56,57]. The obtained accuracies are notably higher than several other studies, in which Sentinel-1 or Sentinel-2 datasets or their combinations were used and compared well versus earlier multisensor EO data studies [3,10,21,56,57,58,59]. Several datasets with high potential for forest variable retrieval, e.g., TanDEM-X, have relatively sparse coverage (both in geographic and temporal domains) and limited availability and thus are not fully suitable for large-area mapping and persistent monitoring purposes.

When using Sentinel-1 or Landsat data to study boreal forests, existing studies report their variable prediction accuracies within the range of 35–60% rRMSE [3,56], while the proposed model utilising Sentinel-1 time series data obtains an rRMSE as small as 18%. Our predictions obtained using traditional machine learning (ML) models were within the same accuracy range as in recently published studies where Sentinel-2 and Landsat data were used [60], while our predictions using different versions of LSTM models appear more accurate. The usage of attention mechanism and Helix attribute in Attn-BiLSTM and the proposed CrsHelix-LSTM are instrumental in providing larger accuracies compared to the basic LSTM model.

There is relatively limited literature using SAR data for forest height predictions, with predictions often reported on stand level or coarser resolution spatial units [8]. However, our stand-level tree height predictions were at the same accuracy level or even better than reported retrievals with TanDEM-X interferometric SAR data, even though TanDEM-X is deemed much more suitable for retrieving vertical forest structure [61,62,63]. To the best of our knowledge, the obtained accuracy level of 18% RMSE for boreal forest using CrsHelix-LSTM and SAR time series is superior to earlier results reported in the literature [8,56,60].

We expect that LSTM-based models and particularly CrsHelix-LSTM can be used to provide better predictions with Sentinel-1 time series (compared to conventional machine learning methods) also in other forest biomes where seasonal trends with C-band SAR data or irregular sampled image datakes are present, e.g., in hemiboreal or temperate forests.

6. Conclusions

Our study demonstrated the potential for applying LSTM approaches with Helix attribute for predicting forest structural variables such as forest height. Introducing temporal context into prediction models even using basic LSTM model has indicated improvement of prediction performance compared to scenarios when each SAR observation is treated independently (benchmarked MLR, RF and LightGBM models).

Furthermore, a novel LSTM model incorporating Helix-Elapse projection, Skip-LSTM and Cross-Pseudo Regression has been developed and tested in the study. The developed model was demonstrated using long time series of Sentinel-1 data but can be applicable to other SAR time series. The CrsHelix-LSTM model provided larger accuracies in boreal forest height mapping in Central Finland compared to other evaluated LSTM approaches and a set of representative machine learning approaches often used in forest mapping. At best, the achieved accuracy of forest height mapping was 28.3% rRMSE for the pixel level predictions and 18.0% rRMSE on the forest stand level.

Obtained results are generally much better than those reported in the literature with SAR datasets. Superior performance can be explained by better capturing temporal dependencies within SAR time series with skip-link based LSTM blocks. Better performance of CrsHelix-LSTM compared to other studied LSTM models indicates that seasonal patterns and irregular datatakes can be better modeled by leveraging the Helix attribute. Additionally, we suggest taking advantage of semisupervised learning, such as CPR incorporated into the CrsHelix-LSTM model, to improve prediction performance with limited reference data.

Future work will concentrate on introducing other datasets particularly suitable for retrieving vertical structure of forests, such as Sentinel-1 interferometric SAR and TanDEM-X datasets, as well as studying other forest variables, such as growing stock volume and above-ground biomass.