A Parsimonious Yet Robust Regression Model for Predicting Limited Structural Responses of Remote Sensing ^†

Abstract

Small data analytics, at the opposite extreme of big data analytics, represent a critical limitation in structural health monitoring based on spaceborne remote sensing technology. Besides the engineering challenge, small data is typically a demanding issue in machine learning applications related to the prediction of system evolutions. To address this challenge, this article proposes a parsimonious yet robust predictive model obtained as a combination of a regression artificial neural network and of a Bayesian hyperparameter optimization. The final aim of the offered strategy consists of the prediction of structural responses extracted from synthetic aperture radar images in remote sensing. Results regarding a long-span steel arch bridge confirm that, although simple, the proposed method can effectively predict the structural response in terms of displacement data with a noteworthy overall performance.

1. Introduction

Structural health monitoring (SHM) of bridge structures [1,2,3,4] must fully account for various environmental actions such as ambient temperature, wind, moisture, and possible chemical attacks [5]. In most cases, bridges, especially long-span ones, are slender and are therefore susceptible to vibrations [6,7,8]. The SHM of such structures is indeed of paramount importance for our interconnected communities [9].

Health monitoring has to exploit sensor equipment and data measurements through data analysis and decision-making strategies. The choice of an appropriate sensing technology and of the measurement of the structural response to different natural or man-made excitation sources is critical to provide data sensitive to the structural state. The process of data analytics is often conducted through data cleaning, compression, fusion [10], data augmentation [11], data prediction [12], data normalization [13], and feature extraction [14]. Different machine learning algorithms within the realms of unsupervised learning [15,16,17,18] and supervised learning [19] can be adopted for decision-making about whether the bridge has suffered damage or can still operate normally.

Recently, the technology of remote sensing has opened a golden window to the SHM of bridge structures [20,21,22,23]. With this technology, it is possible to access synthetic aperture radar (SAR) images and extract structural displacements at different spots of the structures without any sensor installation or labor-intensive field measurements. Despite such important benefits, there are some limitations related to this technology. First, the products of spaceborne remote sensing can be claimed to be Big Data, requiring ad hoc analysis. In most cases, speckle noise in a SAR image is a major challenge for displacement extraction. Second, unlike the contact-based sensing systems, it is not trivial to collect structural responses hourly. Hence, small data is the other important challenge related to the SAR-based SHM. Third, it may be unnecessary to use feature extraction tools like interferometric techniques to obtain information in terms of local displacements with remote sensing, particularly in view of recent progress of machine learning algorithms. All in all, the most appropriate solution to such challenges can be to leverage machine learning-aided prediction capabilities.

A small dataset is one of the challenges of SHM via remote sensing, as a machine learner with insufficient data cannot operate properly. For the problem of data prediction, the same issue can affect the applicability. Since most of the predictive models are developed from regression techniques, the use of small datasets increases the probability of underfitting or overfitting. The best solution in this case is to take advantage of parsimonious yet robust predictive models, featuring simple configurations but providing reliable and robust predictions without any concern related to the underfitting and overfitting problems.

From the aforementioned discussion, it stems that the main goal of this research is to propose a parsimonious and robust regression model to predict partial structural displacements retrieved from a few SAR images. The proposed method is a coupling of a regression artificial neural network (RANN) featuring a fully connected architecture and Bayesian hyperparameter optimization (BHO). The RANN undertakes the prediction of the structural response to temperature variability, while BHO tunes the hyperparameters of the RANN. To assess the effectiveness of the proposed model, partial displacement responses of a long-span bridge are adopted. Results show that the proposed RANN–BHO method is quite successful in predicting the bridge response, even in the presence of small training datasets.

2. Supervised Artificial Neural Network for Regression

2.1. Network Configuration

A RANN is an ANN specifically tailored for regression problems. It is a feedforward, fully connected neural network showing the standard input layer, a number of hidden fully connected layers, and the output layer. The network input is defined as the predictor data. Each fully connected layer handles the input data by means of a weight matrix and a bias vector; an activation function (e.g., the rectified linear unit, hyperbolic tangent, sigmoid function, and linear function) can provide nonlinear transformations of the information/data, see [24,25,26]. A backpropagation algorithm is adopted to tune the weights of the RANN, managing a loss function (as a prediction error between the input and the output) to be minimized with the stochastic gradient descent algorithm. Finally, the predicted response is given as the network output. Figure 1 shows a graphical representation of the RANN, wherein N denotes the number of fully connected (hidden) layers.

The strength of the RANN is the ability to learn both linear and nonlinear relationships between the predictor and the response data due to employing different activation functions, see Hagan et al. [27]. The main hyperparameters of the RANN are the number N of hidden layers, and the number of neurons of each layer. These hyperparameters must be determined before the learning stage.

2.2. Model Tuning via Bayesian Hyperparameter Optimization

Hyperparameter optimization in machine learning deals with the choice of the best values of key parameters of a given machine learning model so that the highest overall performance on a validation set can be attained [28]. Although there are different approaches to hyperparameters tuning, Bayesian hyperparameter optimization (BHO) is the most useful in a case when reaching a reliable overall performance is challenging, like in the presence of small datasets to learn from. Bayesian optimization is an approach that uses Bayes theorem to find the minimum of an objective function. The BHO keeps track of past evaluation outputs and uses them to develop a probabilistic model through the mapping of hyperparameters to a probability of a score on the objective function. For the problem of data prediction, the BHO is designed to minimize the following objective function:

$$F=\mathrm{l}\mathrm{o}\mathrm{g}\left(1+E_{MSE}\right)$$

(1)

where $E_{MSE}$ denotes the cross-validation mean-squared-error (MSE) between observation and estimation; this is achieved iteratively. At each iteration, the objective function $F$ in Equation (1) yields a logarithmic transformed validation loss-value computed for the regression model, along with the relevant optimal set of hyperparameters. As mentioned, the BHO not only handle this function, but also incorporates a probability distribution model to be updated at each iteration. BHO thus defines an acquisition function and the next set of hyperparameters. Hence, it can be considered to deliver a posterior probability distribution model for each hyperparameter. The best hyperparameter values can be selected after reaching a good match between real and predicted data.

In relation to the proposed predictive model, it has been already reported that BHO makes attempts to tune two key hyperparameters of the RANN: the number N of the fully connected layers and the number of neurons in each layer. Apart from hyperparameters, a machine learning model may rest upon other unknown elements, which are termed model parameters. The main difference between hyperparameters and model parameters is that the former should be determined before the learning stage, while the latter can be adjusted during the same [29]. For instance, the weight and bias of each neuron of the RANN represents its model parameters.

3. Method Performance Evaluation for a Steel Arch Bridge

The Lupu Bridge is a steel arch bridge crossing the Huangpu River in Shanghai as shown in Figure 2. It has a total length of 750 m, comprising a main span of 550 m and two side spans of 100 m. Figure 2b provides the elevation view of the Lupu bridge as well as its main sizes. The girder in the side span is a closed steel box, with a width of 41 m and a height of 2.7 m [30].

A long-term SHM program of the bridge was carried out by Qin et al. [30] with the aid of spaceborne remote sensing to inspect the variability in the displacement response of the bridge dome and main span. Fifty-five SAR images from TerraSar-X were collected to extract the displacements of the mentioned bridge components. During the monitoring period, the air temperature was also recorded to incorporate seasonal and thermal effects in the SHM program. Figure 3 illustrates the collected displacement and temperature data. From the regression viewpoint, the displacement and temperature samples are the main dependent (response or output) and independent (predictor or input) data, respectively.

By dividing the dataset with a ratio 80:20, respectively referring to the training and test subsets, the total numbers of training and test samples turn out to be 44 and 11. For the learning process, the training samples are also subdivided on their own into the training and validation sets, leading to 35 and 9 samples to handle respectively. Using these datasets, the BHO is adopted to tune the number of layers and the corresponding neuron sizes. The outcome of the optimization procedure is collected in

Table 1. Tuned hyperparameters of the RANNs via BHO.

Element	Number of Layers	Neuron Sizes
		1st Layer	2nd Layer	3rd Layer
Dome	2	3	2	–
Span	3	2	2	3

. Figure 4 shows the good convergence rate of the trained RANNs adopted for the dome and main span of the bridge, as obtained with the minimization of the objective function $F$ after 30 iterations. Furthermore, Figure 5 depicts the results of displacement predictions obtained with these trained models at the same dome and main span locations. As it can be observed, target and predicted data points match well with each other. To also provide a quantitative evaluation of the agreement, in the figure it is reported that the proposed predictive model yields R-squared values are equal to 0.8509 and 0.9223 at the two locations.

4. Conclusions

This paper has discussed the issues linked to the health monitoring of bridges in cases of limited/small datasets, like those typically collected with remote sensing systems. To address the limitations of small data for prediction, a parsimonious yet robust predictive model has been proposed by combining RANN and BHO. BHO has been exploited to tune the main hyperparameters of the RANN—the number of hidden layers and their neuron sizes.

Displacements along with air temperature related to a long-span steel arch bridge, have been used to verify the capability and performance of the proposed method. The results have demonstrated that the RANN–BHO-based method is an effective and simple predictive tool, featuring reliable estimations in the presence of small datasets to be exploited for the prediction of the structural health.