Statistical Analysis on Random Matrices of Echo State Network in PEMFC System’s Lifetime Prediction

Abstract

The data-driven method of echo state network (ESN) has been successfully used in the proton exchange membrane fuel cell (PEMFC) system’s lifetime prediction area. Nevertheless, the uncertainties of the randomly generated input and internal weight matrices in ESN have not been reported yet. In view of this, an ensemble ESN structure is proposed in this paper to analyze the effects of random matrices from a statistical point of view. For each ESN, the particle swarm optimization (PSO) method is utilized to optimize the hyperparameters of the leaking rate, spectral radius, and regularization coefficient. The statistical results of each ensemble ESN are analyzed from 100 repeated tests whose weight matrices are generated randomly. The mean value of the ensemble ESN and a confidence interval (CI) of 95% are given during the long-term lifetime prediction. The effects of two different distribution shapes, i.e., uniform distribution and Gaussian distribution, are fully compared. Finally, the effects of the ensemble structure and two different distribution shapes are tested by three experimental datasets under steady-state, quasi-dynamic, and full dynamic operating conditions.

1. Introduction

Energy and environment are the focal points attracting the attention of the international community due to the depletion of fossil fuels and the increasing emissions of carbon dioxide [1,2]. Hydrogen (H₂) is considered to be a promising energy carrier of sustainable clean energy in the 21st century. The proton exchange membrane fuel cells (PEMFC) system can transform the chemical energy of H₂ into electricity directly, and the by-products during the electrochemical process are heat and water [3,4]. It is a promising power conversion device because of its satisfactory characteristics except for the high cost and limited lifetime. Predicting the lifetime of the PEMFC system can help the users to take action in advance to extend its working life.

Model-based and data-driven methods are two typical classifications of existing lifetime prediction methods [5,6]. Data-driven methods, which are mainly based on machine learning (ML) theory, construct the nonlinear degradation relations [7,8]. One of the patterns of ML, the recurrent neural network (RNN), has shown great power in nonlinear time series prediction. Nevertheless, the RNN has some weaknesses such as slow convergence rate and the existence of bifurcation. To overcome the weaknesses of RNN, the echo state network (ESN) was proposed by Prof. Jaeger et al. in [9]. There are two characteristics of ESN: one is a large, randomly generated reservoir that is used to replace the hidden layers of RNN; the other is the weight matrices (input and internal) that are generated randomly. The ESN has a faster convergence rate and lower computational training cost than the traditional RNN. Nevertheless, the hyperparameters of ESN (leaking rate, spectral radius, regularization coefficient, reservoir neuron number, reservoir connectivity, etc.) need to be carefully tuned from human expertise [10].

For the PEMFC system’s lifetime prediction application, the ESN has first been applied in [11] to estimate the steady-state cell voltage by the single-step ahead prediction pattern. Four kinds of reservoirs are compared in [12], and the delay line reservoir (DLR) and the simple cycle reservoir (SCR) have the best precision values. A cycle reservoir with a jump (CRJ) model has been proposed in [13], and this structure is validated on the steady-state datasets. The CRJ model can speed up the linear fitting process and improve prediction accuracy. In ref. [14], the steady-state cell voltages are divided into the approximation part and detail part by wavelet transform, and then double ESNs with different dynamic characteristics are utilized to deal with the voltage signal separately. This double ESN structure could improve prediction accuracy. The hyperparameters of ESN in [11,12,13,14] are set by the trial-and-error method. Inspired by the separation concept in [14], the wavelet transform and multiple ESNs are proposed in [15] to deal with the multi-timescale degradation features under the dynamic operating conditions in the long-term prediction horizon. In ref. [15], the hyperparameters of the spectral radius and leaking rate are optimized by the grid-search method, and the prediction accuracy has been improved. To improve the prediction efficiency, the original degradation data prediction is replaced by the shortened coefficients in the discrete wavelet transform process [16]. The prediction data points can be shortened by a factor of 8, and the hyperparameters of ESN (spectral radius, leaking rate, and regularization coefficient) are optimized by the genetic algorithms (GA) method. In ref. [17], the analysis of variance (ANOVA) method is applied to analyze the four hyperparameters of the reservoir neuron number, reservoir connectivity, spectral radius, and leaking rate under the steady-state dataset. Results show that the spectral radius has the biggest effects on the prediction performance. Additionally, based on the ANOVA method, the effects of three hyperparameters (i.e., leaking rate, spectral radius, and regularization coefficient) have been analyzed under the dynamic operating conditions [18]. Results show that the interaction of the spectral radius and regularization parameter has the biggest effect, and the effects of the spectral radius rank second.

Based on the ANOVA analysis results of [17], a multi-reservoir ESN structure is proposed in [19], and each reservoir has a different spectral radius. This multi-reservoir ESN can avoid the optimization of the spectral radius, and its effectiveness is validated under the steady-state cell voltage prediction condition. As an extension of the work in [19], the ESN and Markov chains are combined to predict the cell voltage under the dynamic micro-combined heat and power (μ-CHP) load profile [20]. The RUL prediction by the combined structure can be realized with a mean relative accuracy below 17%. An ensemble ESN network has recently been proposed to avoid hyperparameter optimization and improve the adaptiveness of lifetime prediction [21]. A dynamic health indicator of “virtual steady-state voltage (VSV)” is proposed based on linear parameter varying (LPV) models. Then, a number of ESNs with different combinations of hyperparameters (leaking rate and spectral radius) are used for the VSV prediction. Finally, the long-term RUL prediction and a confidence interval of 95% confidence level are given in detail. In the past two years, the multi-input ESN structure is proposed to improve the prediction accuracy of the RUL [18,22,23,24]. The effects of the operating parameters (i.e., current, temperature, and pressure) are fully analyzed under the steady-state and dynamic operating conditions in [18,22] separately. The multi-input ESN structure could improve the prediction accuracy regarding the health indicator and the scheduled current as the double inputs are more meaningful in practice. In ref. [23], the ambient temperature and stack voltage are used as the double inputs of ESN to improve the RUL prediction accuracy. The normalized root mean square error is equal to 0.098 with a prediction over 2000 h. In ref. [24], the input parameters of ESN are evaluated and selected by the least absolute shrinkage and selection operator (LASSO), and then the degradation-related parameters are used for the RUL prediction. Results show that the ESN with eight input parameters is the optimal choice. Both [23] and [24] are validated by the steady-state datasets.

In the current state-of-art, the developments of ESN for the PEMFC system’s lifetime prediction are focused on the ESN structure (reservoir forms, ensemble ESN, multi-input ESN, etc.) and hyperparameter optimization. In the practical implementation process of ESN, the input and internal weight matrices (W_in and W) remain unchanged after they have been drawn from a random distribution. Nevertheless, the effects of the distribution shape of W_in and W on the lifetime prediction accuracy have not been reported yet. There is abundant evidence that the ensemble structure can improve the prediction robustness and stability such as the ensemble neural network model [25], particle filtering [26], and extreme learning machine [27]. The ensemble ESN is proposed in [21], and the effects of the leaking rate and spectral radius have been fully analyzed. Nevertheless, the statistical property of the ESN, e.g., the uncertainty caused by the random matrices, has not been analyzed yet. Based on these two motivations, the contributions of this paper are summarized as follows:

• The effects of two distribution shapes (uniform and Gaussian distribution) of W_in and W on the prediction accuracy are explored;
• The metaheuristic technique of particle swarm optimization (PSO) is utilized to optimize the hyperparameters of the leaking rate, spectral radius, and regularization coefficient;
• The uncertainty of the ensemble ESN caused by the random matrices is statistically analyzed under three different operating conditions.

2. Mathematical Backgrounds

2.1. Echo State Network

Similar to the structure of RNN, there are three parts in the ESN: input layer, reservoir, and output layer. The different parts are connected by the weight matrix elements of input weight matrix W_in$ϵ{ℝ}^{N\times K}$, internal weight matrix W$ϵ{ℝ}^{N\times N}$, feedback weight matrix W_fb$ϵ{ℝ}^{N\times L}$, and output weight matrix W_out$ϵ{ℝ}^{L\times \left(N+K\right)}$. K is the dimension of the input signal, N is the number of neurons in the reservoir, and L is the dimension of the output signal. The spectral radius $\rho$ is the maximum eigenvalue of W. The W_fb is set to be 0 in this task. At step n, the neuron update in the reservoir is determined by

$$\tilde{x}(n)=f(W_{in}\cdot u(n)+W\cdot x(n))$$

(1)

where $\tilde{x}\left(n\right)$ is the neuron update, $u\left(n\right)$ is the input signal at step n, $x\left(n\right)$ is the neuron state at step n, and f is the activation function. Then, the neuron state at step (n + 1) can be expressed as

$$x(n+1)=(1-\alpha )\cdot x(n)+\alpha \cdot \tilde{x}(n)$$

(2)

where α is the leaking rate of the reservoir. The ESN output is represented by

$$y(n+1)=W_{out}\cdot (x(n+1);u(n))$$

(3)

where $y\left(n+1\right)$ is the output signal. In the training part, the W_out is calculated to minimize the root mean square error (RMSE) between the predicted value ${\mathrm{y}}^{pre}\left(n\right)$ and target value ${\mathrm{y}}^{tar}\left(n\right)$

$$W_{out}=\arg \min \sqrt{\frac{1}{m}{\displaystyle \sum _{n=1}^m{(y^{pre}(n)-y^{tar}(n))}^2}}$$

(4)

where m is the total data points in the training part. When the ridge regression is utilized, the W_out can be calculated by linear regression

$$W_{out}=Y^{tar}{X}^{\mathrm{T}}{(X{X}^{\mathrm{T}}+\gamma I)}^{-1}$$

(5)

where $X$ represents the neuron state and inputs (x(n + 1);u(n)), $Y^{tar}$ is the target value in the training part, $\gamma$ is the regularization coefficient, and I is the identity matrix. In this task, the reservoir neuron number N is 400, the bounds of leaking rate α, spectral radius $\rho$, and the regularization coefficient $\gamma$ are (0, 1), (0, 2), and (1$\times {10}^{-3}$, 9 $\times {10}^{-3}$), respectively. The implementation time of ESN without hyperparameter optimization takes about 20 s in Matlab 2018, Intel-i7 CPU (2.6 GHz), and 16 G RAM environment.

The W_in and W are generated randomly and sparsely. For simplicity, they are usually generated in the same distribution form. Based on the practical guide to applying ESN [28], the uniform distribution and Gaussian distribution are two popular distribution shapes. The uniform distribution is first performed for its continuity of values and boundedness. The distribution interval is [−0.5, 0.5], and the variance D(x) is calculated as 1/12. When the Gaussian distribution has the same variance as uniform distribution, the weight matrix follows the normal distribution N (0, 1/12) [29]. The probability density functions of uniform distribution and Gaussian distribution are shown in Figure 1.

2.2. Particle Swarm Optimization

The hyperparameters of the ESN need to be carefully tuned, including the leaking rate α, spectral radius $\rho$, regularization coefficient $\gamma$, reservoir neuron number N, reservoir connectivity c, etc. Some straightforward hyperparameter setting methods such as trial-and-error [18], grid-search [15], and stochastic gradient descent [10] are computationally complex. The effectiveness of some metaheuristic methods such as PSO [30], GA [31], and big bang–big crunch (BB–BC) [20] have been proven in different tasks. Optimizing all hyperparameters is time-consuming, so three key hyperparameters of the ESN (α, $\rho$, and $\gamma$), which have the biggest effects on the prediction results, are selected to be optimized [16].

The PSO is a satisfactory global search method that is easier to be implemented, has fewer parameters, is computationally less expensive, and can converge more quickly [32]. A population of particles is used for searching, and each particle indicates a candidate solution to the problem. The velocity v_i and position x_i updating of the particle at time step k are shown as

$$\{\begin{array}{l}{v}_i^{k+1}=w{v}_i^k+c_1{r}_1(p_{ib}-x_i^k)+c_2{r}_2(g_{ib}-x_i^k)\hfill \\ x_i^{k+1}=x_i^k+v_i^{k+1}\hfill \end{array}$$

(6)

where p_ib and g_ib are the best personal and global positions, w is the inertial weight, c₁ and c₂ are the learning factors, and r₁ and r₂ are the random variables from 0 to 1. The objective function is to minimize the RMSE between the predicted value ${\mathrm{y}}^{pre}\left(n\right)$ and target value ${\mathrm{y}}^{tar}\left(n\right)$, as shown in Equation (4). In this case study, the swarm size is 30, the maximum number of iterations is 600, the function tolerance is 1$\times {10}^{-6}$, both learning factors are 1.49, and the inertial weight is 0.8.

2.3. Implementation of Ensemble ESN

The implementation process of the ensemble ESN is shown in Figure 2. There are four steps in total:

Step 1 is the training data preparation. Some pretreatment procedures are necessary, such as filtering, resampling, and normalization after the data (voltage, current, etc.) acquisition. In this task, the moving average filtering method is utilized, and the length of the moving window is empirically set to 31. The resampling time interval between two adjacent points is 0.5 h. The data are normalized to the same interval [0, 1] by Equation (7)

$$z^{\prime }=\frac{z-z_{\min }}{z_{\max }-z_{\min }}$$

(7)

where z’ is the normalized value, z is the original value, and z_max and z_min are the maximum and minimum values, respectively. The health indicators can be extracted, i.e., voltage for steady-state and quasi-dynamic operating conditions and the relative power loss rate (RPLR) for the dynamic operating condition.

Step 2 is the degradation prediction based on individual ESNs. The randomly generated procedure of W_in and W are similar in the ESN, so the distribution shapes of these two matrices are the same for convenience. For each ESN_i (1 ≤ i ≤ p), the uniform distribution and Gaussian distribution shapes of these two matrices are explored separately. The hyperparameters of the ESN are optimized by PSO in the training part, and the optimized parameter combination is used in the prediction part. Thus, each ESN would have two long-term prediction results under two random matrices distribution shapes. The total number of ensemble ESNs is set to be 100 (p = 100) for statistical analysis.

Step 3 is the ensemble of prediction results. For a certain distribution shape, 100 times prediction results are assembled to form a single new model to statistically analyze the randomness. The weight of each ESN is taken to be the same, i.e., each ESN has the same contribution during the RUL prediction. Thus, the mean value of 100 ESNs at each time is calculated as the final prediction value.

Step 4 is the result analysis. Each ESN could realize an individual prediction, and its RMSE between the predicted signal and target signal can be calculated by

$$\mathrm{RMSE}=\sqrt{\frac{1}{m}{\displaystyle \sum _{t=1}^m{(y^{pre}(t)-y^{tar}(t))}^2}}$$

(8)

At a certain time step t (1 ≤ t ≤ m), the mean and the variance of the predicted value at time step (s + t) can be represented as

$$\{\begin{array}{l}\overline{y}(s+t)=\frac{1}{p}{\displaystyle \sum _{i=1}^p{\widehat{y}}_i(s+t)}\hfill \\ D(s+t)=\frac{1}{p-1}{\displaystyle \sum _{i=1}^p{(\overline{y}(s+t)-{\widehat{y}}_i(s+t))}^2}\hfill \end{array}$$

(9)

where p is the total number of ensemble ESNs, ${\widehat{y}}_i\left(s+t\right)$ is the predicted value, $\overline{y}\left(s+t\right)$ is mean value, and $D\left(s+t\right)$ is the variance. Assuming the predicted value follows the standard Gaussian distribution, the 95% confidence interval (CI) of the results at time step (s + t) can be expressed as

$$\mathrm{CI}(s+t)=[\overline{y}(s+t)-1.96\sqrt{D(s+t)},\overline{y}(s+t)+1.96\sqrt{D(s+t)}]$$

(10)

3. Experimental Results

Three degradation datasets under different operation conditions are used for exploring the prediction effects of two distribution shapes of weight matrices, testing the performance improvements of the ensemble ESN, and statistically analyzing the uncertainty of random matrices. The steady-state and quasi-dynamic datasets come from the open data of the 2014 Prognostic and Health Management Data Challenge [22]. Two PEMFC stacks are used for the 1000 h duration test under the steady-state and quasi-dynamic operating conditions. There are 5 cells for both stacks, and their active area is 100 cm², the operating temperature is 80$℃$, and the back pressure is 0.2 MPa. For the steady-state test, the constant current is 70 A. For the quasi-dynamic test, the current ripples (7 A) at different frequencies (from 50 mHz to 10 kHz) are superimposed to the constant current (70 A) for the electrochemical impedance spectroscopy (EIS) measurement. The third dataset comes from a PEMFC stack for 382 hours’ duration test under the dynamic μ-CHP load profile. The current has a cycle between 170 A and 85 A. The stack consists of 8 cells, and the active area is 220 cm², the operating temperature is 80$℃$, and the back pressure is 0.15 MPa [18]. The load profiles of the three tests are shown in Figure 3.

3.1. Steady-State Operating Condition

In the steady-state operating condition, the stack output power is used as the degradation health indicator. Half of the data (0–500 h) is used for training the ESN, and the rest of the data (501–1000 h) is used for prediction. During the prediction procedure, 100 individual ESNs have 100 groups of prediction results. Then, the mean value, variance, and 95% CI of the predicted results can be calculated at each time step. When W_in and W follow the uniform distribution, the RUL prediction results are shown in Figure 4. Taking the first 10 groups of ESNs as an example, their hyperparameters and RMSE are shown in

Table 1. Steady-state results of ESN₁-ESN₁₀ when W_in and W follow the uniform distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.80000	0.24477	0.80000	0.80000	0.10000	0.61598	0.55070	0.28000	0.10000	0.10000
		0.38125	0.69227	0.79748	0.93218	0.20756	0.57188	0.56348	0.72581	0.18374	0.17359
		0.00100	0.00900	0.00900	0.00900	0.00164	0.00363	0.00100	0.00900	0.00136	0.00340
RMSE		0.58050	0.59487	0.59647	0.60032	0.59554	0.57550	0.57331	0.57092	0.60588	0.63377

.

After changing the distribution shapes of W_in and W to Gaussian distribution, the RUL prediction results under the steady-state operating condition are shown in Figure 5. The hyperparameters and RMSE of the first 10 groups of ESNs are shown in

Table 2. Steady-state results of ESN₁-ESN₁₀ when W_in and W follow the Gaussian distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.90000	0.90000	0.90000	0.85000	0.39040	0.90000	0.90000	0.71572	0.90000	0.90000
		0.19560	0.33440	0.44590	0.10000	0.86250	0.10000	0.31760	0.10000	1.41060	0.10000
		0.00100	0.00320	0.00650	0.00660	0.00350	0.00110	0.00100	0.00100	0.00100	0.00120
RMSE		0.56508	0.57730	0.57151	0.56970	0.56199	0.56915	0.56801	0.58255	0.57192	0.56398

. To analyze the effects of the two matrices’ distribution shapes of W_in and W on the prediction accuracy, the RMSE of 100 ESNs is shown in Figure 6.

The average RMSE under the Gaussian distribution (0.57488) is lower than the average RMSE under the uniform distribution (0.58602) for the steady-state test. The mean prediction results of the ensemble ESN are calculated, i.e., 0.57841 for uniform distribution and 0.56966 for Gaussian distribution.

3.2. Quasi-Dynamic Operating Condition

In the quasi-dynamic operating condition, the stack output power is used as the degradation-related health indicator. Similar to the steady-state operating condition, half of the data (0–500 h) is used for training the ESN, and the rest of the data (501–1000 h) is used for prediction. When W_in and W follow the uniform distribution, the RUL prediction results are shown in Figure 7. Taking the first 10 groups of ESNs as an example, their hyperparameters and RMSE are shown in

Table 3. Quasi-dynamic results of ESN₁-ESN₁₀ when W_in and W follow the uniform distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.29901	0.10970	0.10820	0.13740	0.10000	0.34770	0.10000	0.41590	0.10800	0.12280
		1.90000	1.80230	1.74110	1.90000	1.62237	1.55670	1.90000	1.42740	1.42460	1.61780
		0.00299	0.00710	0.00860	0.00150	0.00900	0.00110	0.00900	0.00890	0.00900	0.00100
RMSE		1.07394	1.01290	1.02710	1.10380	1.41801	1.03080	1.88830	1.28040	1.09540	1.49140

.

After changing the distribution shapes of W_in and W to Gaussian distribution, the RUL prediction results under the quasi-dynamic operating condition are shown in Figure 8. The hyperparameters and RMSE of the first 10 groups of ESNs are shown in

Table 4. Quasi-dynamic results of ESN₁-ESN₁₀ when W_in and W follow the Gaussian distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.87000	0.35830	0.38420	0.10110	0.87420	0.54400	0.84620	0.82820	0.10000	0.10900
		1.36500	1.80570	1.58110	1.51860	1.47940	1.90000	1.83080	1.43060	1.76060	1.86000
		0.00790	0.00830	0.00900	0.00900	0.00330	0.00900	0.00880	0.00100	0.00100	0.00580
RMSE		1.03890	0.93920	1.28170	0.89560	0.96380	1.37308	1.05020	0.97690	1.13970	0.90080

.

To analyze the effects of the two matrices’ distribution shapes of W_in and W on the prediction accuracy, the RMSE of 100 groups of ESNs is shown in Figure 9. The average RMSE under the Gaussian distribution (1.19634) is lower than the average RMSE under the uniform distribution (1.27562) for the quasi-dynamic test. The mean prediction results of the ensemble ESN are calculated, i.e., 0.95657 for uniform distribution and 0.90618 for Gaussian distribution.

3.3. Dynamic Operating Condition

In the dynamic operating condition, the RPLR is used as the degradation health indicator, which has been described in detail in our previous work [18]. First, the polarization curve of the PEMFC stack at the beginning of life (BoL) is measured. The BoL power at different currents can be calculated to form a table. Secondly, the power at time step t can be calculated by the measured current (I_t) and voltage (U_t). Finally, the RPLR at time step t is calculated by

$$\mathrm{RPLR}=(P_t-P_0)/P_t$$

(11)

where P_t is the power at time step t and P₀ is the BoL power, which is determined by the look-up table method. The P_t and P₀ are at the same current level.

Half of the data (0–191 h) is used for training the ESN, and the rest of the data (192–382 h) is used for prediction. When W_in and W follow the uniform distribution, the RUL prediction results are shown in Figure 10. Taking the first 10 groups of ESNs as an example, their hyperparameters and RMSE are shown in

Table 5. Dynamic results of ESN₁-ESN₁₀ when W_in and W follow the uniform distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.12483	0.25671	0.15844	0.11213	0.34245	0.23291	0.23923	0.14834	0.26963	0.36064
		1.43205	1.74959	1.35257	1.53465	1.18192	1.33707	1.90000	1.23153	1.37328	1.25512
		0.00218	0.00524	0.00900	0.00900	0.00129	0.00405	0.00100	0.00900	0.00165	0.00253
RMSE		0.00476	0.00527	0.00483	0.00482	0.00578	0.00523	0.00508	0.00485	0.00479	0.00501

.

After changing the distribution shapes of W_in and W to Gaussian distribution, the RUL prediction results under the dynamic operating condition are shown in Figure 11. The hyperparameters and RMSE of the first 10 groups of ESNs are shown in

Table 6. Dynamic results of ESN₁-ESN₁₀ when W_in and W follow the Gaussian distribution.

Test Group		ESN1	ESN2	ESN3	ESN4	ESN5	ESN6	ESN7	ESN8	ESN9	ESN10
Hyper-para.	α $\rho$ $\gamma$	0.30376	0.47659	0.54372	0.50078	0.13709	0.57584	0.32100	0.28035	0.11564	0.39271
		1.64620	1.29902	1.55139	1.71862	1.38232	1.44123	1.43160	1.59497	1.50729	1.90000
		0.00634	0.00900	0.00586	0.00900	0.00900	0.00638	0.00157	0.00625	0.00900	0.00900
RMSE		0.00511	0.00527	0.00525	0.00481	0.00413	0.00512	0.00509	0.00473	0.00569	0.00485

.

The RMSE of 100 groups of ESNs is shown in Figure 12. The average RMSE under the Gaussian distribution (0.00489) is lower than the average RMSE under the uniform distribution (0.00502) for the dynamic test. The mean prediction results of the ensemble ESN are calculated, i.e., 0.00473 for uniform distribution and 0.00464 for Gaussian distribution.

Among these three different operating conditions, each random generation of W_in and W produces a different prediction result. The essential reason is that the randomness of the matrices changes the weight values of the connected neurons, and the random weight leads to different neuron behaviors, which are important for the nonlinear relationship construction ability of the reservoir. The numerous repeated tests also illustrate the uncertainty properties of random matrices. The ensemble ESN structure is utilized to statistically analyze the effects of random characteristics and the bounds of uncertainties are obtained. Taking into account these uncertainties, the robustness of the prediction can be further improved in practical applications.

4. Conclusions

The data-driven method of echo state network (ESN) is further explored in the long-term lifetime prediction of the PEMFC system. During the implementation process of the ESN, the input and the internal weight matrices (W_in and W) are randomly generated. The traditional method keeps these two matrices unchanged once they are generated. Nevertheless, the random characteristics of these matrices will affect the performance of the ESN. To statistically analyze the effects of random characteristics, an ensemble ESN is proposed and tested under three different operating conditions. The conclusions are as follows:

1.

The random characteristics of W_in and W affect the lifetime prediction results. The prediction results are presented in the statistical form by the ensemble computing technique, and this helps the user to analyze the uncertainties of the randomness. After the data analysis, a 95% confidence interval (CI) is given, which better qualifies the reliability of the result.

2.

Based on the uniform and Gaussian distribution shapes of W_in and W, the prediction performances are fully compared. To analyze the effects of random matrices, the PSO method is used to optimize the hyperparameters of the ESN. Based on the comparison results, the Gaussian distribution of W_in and W can decrease the prediction error slightly when compared to the uniform distribution. However, the effects of two different distribution shapes on the prediction results are rather insignificant.

3.

Combining the ESN with other methods to improve the prediction accuracy and exploring the prognostic methods to realize the online lifetime prediction will be the focuses of our future work.