Multivariable NARX Based Neural Networks Models for Short-Term Water Level Forecasting ^†

Abstract

In this work a novel application for multivariable forecasting is presented, applied to hydrological variables and based on a multivariable NARX model. The proposed approach is designed for two hydrological stations located at the Atrato River in Colombia where the variables of water level, water flow and water precipitation are correlated by using the NARX model based on a neural network structure. The structure of the NARX-based neural network is designed in order to consider the complex dynamics of hydrological variables and their corresponding cross-correlations. A short-term water level forecasting is designed based on the NARX model, to be used as an early warning flood system. The validation of the proposed approach is performed by comparing the estimation error with an ARX dynamic model. As a result, it is shown that a NARX model structure is more suitable for water level forecasting than simplified structures.

1. Introduction

The accurate modeling of hydrological variables is crucial for effective flood forecasting and the design and operation of water resource systems, as stated in the reference [1]. To achieve this, two types of techniques are commonly used: white-box algorithms, which rely on mathematical modeling, and black-box algorithms, which employ non-linear neural network techniques based on artificial intelligence. In the context of early warning systems, the latter technique has been used to great effect, as highlighted in [2], which discussed the use of artificial neural networks (ANNs) for prediction and forecasting. Furthermore, the combination of ANNs with the Soil and Water Assessment Tool (SWAT) has been applied for runoff prediction and water management resources, as described in [3].

Hydrological models have been developed to support flood early warning systems through the use of estimation and prediction algorithms. In [4], the authors focused on developing a flood forecasting system (FFS) capable of providing early warning to UDS managers of potential flooding, using a nonlinear autoregressive neural network with exogenous inputs (NARX) to predict the impact of a storm. Meanwhile, in [5], a neural network short-term memory model (LSTM) was proposed for flood forecasting, using daily flow and rainfall as input data, and analyzing features that may affect the model’s performance. In [6], the authors presented flood early warning systems that used machine learning (ML) techniques, comparing the performance of five ML classification techniques for short-duration flood forecasting. Lastly, Bande and Shete [7] described an IoT-based flood monitoring and flood prediction system based on artificial neural networks (ANN), which aimed to monitor humidity, temperature, pressure, rainfall, and water level of rivers and analyze their temporal correlation for flood prediction. The system was designed to improve the scalability and reliability of the flood management system.

Several studies related to flood forecasting and monitoring using various ANN techniques have also been developed. For example, in [8], the authors evaluated the bias correction of real-time precipitation data and the improvement of hydrological models using the ANN bias correction method for real-time flood forecasting. In [9], the authors focused on developing five different ANN models for flood forecasting and compared their performance. In [10], the authors use a multilayer perceptron to design a flood prediction model with flow as input-output variables, and the proposed model’s effectiveness was demonstrated through intensive experiments. In [11], the authors designed a flood monitoring system that integrated flow and water level sensors and used a two-class neural network to predict flood status from data stored in the database. Finally, in [12], the authors employed a convolutional neural network (CNN) to predict time series variables such as water level in a flood model, despite CNNs typically being used for two-dimensional image classification with transfer learning.

In addition, in [13], a flood forecasting model that predicted future flood occurrence was designed and evaluated by constructing a hybrid deep learning algorithm called ConvLSTM, which integrated the predictive merits of CNN and a long-term memory network (LSTM). In [14], a fuzzy neural network that used fuzzy numbers to account for uncertainty in the results and model parameters was proposed to predict the peak flow in an urban river. In [15], the potential of the AI computational paradigm for modeling streamflow was explored by developing nine different flood prediction models using all available training algorithms of ANN, fuzzy logic, and adaptive neuro-fuzzy inference systems (ANFIS) algorithms. Lastly, in [16], a deep neural network was used to predict floods as a function of temperature and rainfall intensity, and its accuracy and error were compared with other machine learning models, such as the support vector machine (SVM), K-nearest neighbor (KNN), and Naïve Bayes.

The use of real-time methods based on ANN was also proposed based on neural networks. For example, in [17], a system for predicting flood levels was developed based on real-time sensor data. The system used a multi-layer artificial neural network model created with MATLAB to predict flood levels in advance using data collected from sensors in a real-time monitoring system. In [18], a hybrid river flood forecasting model was presented using time series analysis and artificial neural networks to explain and forecast daily water discharge of the Mohawk River in New York. Multiple linear regression (MLR) models and an ANN model were used to describe each component for predicting the water discharge time series. In [19], five alternative machine learning techniques were used to improve the hydrological model, including linear regression, neural network regression, Bayesian linear regression, and reinforced decision tree regression, with the MIKE-11 hydrologic forecasting model used as a test system. In [20], a machine learning method was presented that uses historical typhoon paths to predict flood hydrographs of a Taiwanese watershed. Finally, in [21], a general framework for probabilistic flood forecasting was introduced, which uses an unaccented Kalman filter (UKF) postprocessing technique to model point forecasts made with a recurrent neural network and their corresponding observations. The methodology was tested using a 6 h long-term time series of the Three Gorges reservoir in China.

In this work, the application of a multivariable NARX model based on neural network for short-term water level forecasting along the Atrato river in Colombia is presented and evaluated. To this end, the present work uses data from two hydrological stations located in the Atrato river, which are monitored by the Institute of Hydrology, Meteorology and Environmental Studies (IDEAM). The data includes measurements of flow, precipitation, and water levels sampled every 12 h over a period of 789 days. The multivariable NARX model is trained to predict the water levels of each station based on the inputs of water level, water flow and water precipitation by considering the inherent dynamic and correlation of the process. The performance of the models is evaluated based on the mean square error of the estimated outputs compared to the actual data. The performance of the proposed approach is compared to a multivariable ARX system. The main contribution of this paper is a general design of a multivariable NARX model structure based on neural networks for short-term water level forecasting. This work is organized as follows; in Section 2 the theoretical framework where the hydrological variables and the NARX model for their corresponding dynamic approximation is proposed. In Section 3 the experimental setup and the estimation results are shown, and finally in Section 4 the conclusions and final remarks are presented.

2. Theoretical Framework

2.1. Hydrological Variables

In order to perform water level forecasting based on the dynamic of a river, two hydrological stations are located on a river in two different positions. In order to consider the correlation among all the variables of the system and their corresponding nonlinearities, a nonlinear dynamical model is proposed. In (1) the inputs and outputs of the proposed model are shown.

$$\begin{array}{c}\hfill y\left[k\right]=\left[\begin{array}{c}{y}_{L_1}\left[k\right]\\ y_{L_2}\left[k\right]\end{array}\right],u\left[k\right]=\left[\begin{array}{c}{u}_{F_1}\left[k\right]\\ u_{P{T}_1}\left[k\right]\\ u_{F_2}\left[k\right]\\ u_{P{T}_2}\left[k\right]\end{array}\right]\end{array}$$

(1)

where $y_{L_1}\left[k\right]$, $y_{L_2}\left[k\right]$ correspond to the two outputs of the level variable of the two stations, $u_{F_1}\left[k\right]$, $u_{F_2}\left[k\right]$, $u_{P{T}_1}\left[k\right]$, $u_{P{T}_2}\left[k\right]$ are the four inputs of the neural network system corresponding to the two stations of the multivariable system, i.e., the $u_{F_j}\left[k\right]$ represents the j-th two inputs of the flow variable of the two stations and $u_{P{T}_j}\left[k\right]$ represents j-th; two more inputs of the rainfall variable of the two stations already mentioned, thus obtaining a multivariable system with four inputs and two outputs.

The dynamic of the hydrological variables is defined by considering a Nonlinear function with an Auto-Regressive and exogenous inputs (NARX) as follows:

$$\begin{array}{c}\hfill y\left[k\right]=f\left(y\right[k-1],\dots ,y[k-n],u[k-1],\dots ,u[k-n\left]\right)+\eta \left[k\right]\end{array}$$

(2)

being n the order of the NARX model and $f(.)$ the nonlinear function, and $\eta \left[k\right]$ the additive noise at time instant k.

2.2. Narx Based Neural Network Structure

In order to consider the NARX model of (2), the inputs are selected as $u[k-j]$ and $y[k-j]$, with $j=1,\dots ,n$, which correspond to a $n\text{-}th$ order model In this work a $4\text{-}$th order model ($n=4$) is considered according to [22] where an analysis of the order selection is performed and the lowest estimations error is obtained for the 3-rd order model or higher. Therefore, by considering the variables described in (1), the proposed NARX model consists of 24 inputs and 2 outputs.

In order to approximate the nonlinear function of (2), the nonlinear function $f(.)$ is approximated by using a neural network structure $f^{*}(.)$, as depicted in Figure 1.

• where the NARX model can be defined as follows:

  $$\begin{array}{c}\hfill y\left[k\right]=f^{*}(y[k-1],\dots ,y[k-n],u[k-1],\dots ,u[k-n])+\eta \left[k\right]\end{array}$$

  (3)

To this end, 24 input activation functions with one hidden layer and 2 outputs are considered. A feed-forward network is selected as a candidate for the NARX model in order to speed-up the training process. The training of the NARX model is performed offline by considering the data sample.

A linear ARX structure can be obtained by neglecting the hidden layer as depicted in Figure 2.

• where the ARX model can be defined as follows:

  $$\begin{array}{c}\hfill y\left[k\right]=-\sum _{j=1}^4{a}_{j}y[k-j]+\sum _{j=1}^4{b}_{j}u[k-j]\end{array}$$

  (4)

  where $A_j\in {\mathbb{R}}^{m\times m}$ and $B_j\in {\mathbb{R}}^{m\times m}$ are the parameters of the model matrix, where y are the outputs and u are the inputs; with $j=1,$ being p the order of the system, $e\left[k\right]$ the noise with m, the number of outputs and inputs of the system, $y\left[k\right]\in {\mathbb{R}}^{m\times 1}$ and $u\left[k\right]\in {\mathbb{R}}^{m\times 1}$.

3. Results

3.1. Experimental Setup

In order to validate the proposed approach, real measurements from two hydrological stations located at the Atrato river are considered. The measured variables are: water level, water flow, and water precipitation. The Atrato river is located in Colombia (South America) and has a total length of 750 km with a variable width in a range of 150 m to 500 m. In addition, the depth of the river has a variability in the range of 31m to 38 m. A total amount of 789 days of data with a sample time of 12 h is considered, with initial date 1 January 2021.

In

Table 1. Location of the hydrological stations.

	Station 1 (E1)	Station 2 (E2)
Longitude	76°40${}^{\prime }$10.75${}^{″}$ W	76°39${}^{\prime }$44.13${}^{″}$ W
Latitude	5°45${}^{\prime }$53.38${}^{″}$ N	5°41${}^{\prime }$52.77${}^{″}$ N
Altitude	20.579 MASL	20.83 MASL
City	Belén de Bajirá	Quibdó

the geographical positions of the hydrological stations are shown.

In addition, it is worth noting that the distance in kilometers between the two stations E1 and E2 is $447.1$ km.

In order to validate the proposed approach, a comparison analysis of the proposed NARX approach based on neural networks (3) is performed with a multivariable ARX model (4). A visual comparison of the real and estimated signals is presented for the ARX and NARX methods and also a quantitative evaluation based on the mean squared error is performed. Both systems (ARX and NARX) are trained offline by considering the measurement data. An evaluation in terms of the training error is also presented. The feed-forward network structure for the NARX approach considers one hidden layer with 256 units. The Rectified Linear Unit (ReLU) Activation Function is selected for the proposed approach, where the ReLU is a piecewise linear function that will output the input directly if it is positive, or zero otherwise.

The implementation of the proposed NARX model based on neural networks and also the ARX model is performed in Python by using Tensorflow, which is an open-source machine learning library developed by Google.

3.2. Estimation Results

In this subsection the forecasting results for the two considered methods are presented: The proposed multivariable NARX approach, and the multivariable ARX.

In Figure 3 the estimation results for the ARX method for each of the two water level outputs are presented. In Figure 3a the short-term estimation of the first water level output and also the real measurements are shown. In Figure 3b the short-term water level forecasting of the second output as well as the real corresponding measurements are depicted.

In Figure 3a,b shows that the real measurements are adequately estimated by using the multivariable ARX model.

In

Table 2. Mean squared Estimation error for several nodes configurations.

NARX Hidden Layer Nodes	Level 1	Level 2	Total
2	$1.6867$	$1.6859$	$3.3726$
4	$0.0266$	$0.0275$	$0.0541$
8	$0.0254$	$0.0272$	$0.0526$
16	$0.0227$	$0.0257$	$0.0484$
32	$0.0232$	$0.0201$	$0.0433$
64	$0.0235$	$0.0180$	$0.0415$
128	$0.0176$	$0.0133$	$0.0309$
256	$0.0085$	$0.0108$	$0.0193$
512	0.0078	$0.0101$	$0.0179$

an analysis for the nonlinear neural network NARX in terms of the number of nodes in the hidden layer and their corresponding mean squared estimation error is shown.

From Table 2 it can be seen that the total estimation error is reduced by increasing the number of nodes in the hidden layer. It is noticeable that there is no significant reduction in the total estimation error between 256 and 512 nodes. Therefore, in this work, 256 nodes in the hidden layer are used for evaluation of the NARX models.

In Figure 4 the estimation results for the proposed multivariable NARX method for each of the two water level outputs are presented, by using 256 nodes at the hidden layer according to the results presented in Table 2. In Figure 4a the short-term estimation of the first water level output and also the real measurements are presented. In Figure 4b the short-term water level forecasting of the second output as well as the real corresponding measurements are presented.

It is worth noting that in Figure 4a,b it is shown that also for the multivariable NARX model, the real measurements are adequately estimated.

By considering the forecasting results presented in Figure 3 and Figure 4, the proposed multivariable NARX and ARX approaches show an adequate performance by visual inspection. In order to determine which approach tracks the dynamics of the hydrology variables more adequately, a quantitative evaluation is performed. To this end, the mean squared error is computed in order to compare the real measurements and their corresponding forecasting for each of the considered methods. As a result, in

Table 3. Mean squared Estimation error.

Neural Network Model	Level 1	Level 2	Total
$ARX$	$0.0280$	$0.0263$	$0.0543$
$NARX$	$0.0085$	$0.0108$	$0.0193$

the mean squared error for the proposed multivariable NARX approach and also the multivariable ARX method are presented. It can be seen that the estimation error for the proposed NARX model is lower than the ARX model. It is worth noting that the reduction of estimation error for the NARX approach in comparison to the ARX approach is over the $50\%$ for all variables.

In Figure 5 the estimation errors during training for each of the considered methods are shown. It can be seen that the multivariable ARX approach converge faster than the proposed multivariable NARX approach. However, the training error was lower for the NARX approach in comparison to the ARX approach. This behaviour validated the fact that there are nonlinear dynamics inherent to the measured hydrological variables and therefore the proposed NARX model forecast the data behaviour more adequately.

4. Discussions and Conclusions

In this work a novel application for multivariable forecasting method for hydrological variables based on multivariable NARX model is presented. To this end, the nonlinear function of the NARX model has been approximated by using neural networks. The proposed multivariable NARX approach is compared to a multivariable ARX approach, where the proposed NARX approach shows a lower estimation error (a reduction over a $50\%$ error as shown in Table 3). By considering the training errors, it can be seen that the training error is lower for the NARX approach in comparison to the ARX approach due to the nonlinear dynamics inherent to the measured hydrological variables. In summary, the proposed multivariable NARX model based on neural networks is an effective tool for water level forecasting by considering the correlation among several hydrological variables and several stations. It is worth noting that the main contribution of the proposed approach is the design of a general structure for modeling that can be extended to several hydrological systems with more stations and variables. In future work, online training, white-box or more complex nonlinear structures can be used to describe the nonlinear behavior of the system.