Machine Learning-Based Model Predictive Control of Two-Time-Scale Systems

Abstract

In this study, we present a general form of nonlinear two-time-scale systems, where singular perturbation analysis is used to separate the dynamics of the slow and fast subsystems. Machine learning techniques are utilized to approximate the dynamics of both subsystems. Specifically, a recurrent neural network (RNN) and a feedforward neural network (FNN) are used to predict the slow and fast state vectors, respectively. Moreover, we investigate the generalization error bounds for these machine learning models approximating the dynamics of two-time-scale systems. Next, under the assumption that the fast states are asymptotically stable, our focus shifts toward designing a Lyapunov-based model predictive control (LMPC) scheme that exclusively employs the RNN to predict the dynamics of the slow states. Additionally, we derive sufficient conditions to guarantee the closed-loop stability of the system under the sample-and-hold implementation of the controller. A nonlinear chemical process example is used to demonstrate the theory. In particular, two RNN models are constructed: one to model the full two-time-scale system and the other to predict solely the slow state vector. Both models are integrated within the LMPC scheme, and we compare their closed-loop performance while assessing the computational time required to execute the LMPC optimization problem.

1. Introduction

Various applications in the field of chemical engineering involve systems that exhibit different time scales. Instances of such systems include biochemical processes, catalytic reactors, and distillation columns. Furthermore, other scientific sectors, such as power electronics, communication networks, and biological systems, also feature systems with dynamics evolving in disparate time scales. Specifically, in the context of chemical engineering, researchers have utilized the method of singular perturbation to decouple two-time-scale systems into reduced-order subsystems, each associated with a distinct time scale (e.g., [1]). This approach simplifies the analysis of the ordinary differential equations governing the system, allowing for the design of a suitable well-conditioned control law that stabilizes the system. In the context of control system design, model predictive control (MPC) is widely recognized as one of the leading and most convenient approaches employed in stabilizing different types of nonlinear systems. MPC is essentially an optimization problem formulated with a well-defined objective function aimed at enhancing the performance of the system while meeting constraints associated with the system’s physical structure and closed-loop stability. The fact that MPC accounts for multiple inputs, outputs, and constraints within its framework is what makes it a suitable choice for designing control systems to stabilize chemical processes. However, in singularly perturbed systems, the presence of distinct time scales can lead to challenges when employing MPC without accounting for the state evolution in the different time scales. Neglecting this aspect can result in degradation of the closed-loop performance, long time consumption, or, in more critical situations, instability and stiffness in the control system due to issues with the controller’s effectiveness and the presence of ill-conditioning [2].

However, if the time-scale multiplicity is accounted for, MPC has proven to be an efficient control strategy when it comes to situations involving time-scale separation. For instance, in [3], the design of a composite control system was presented, which included two MPC designs: one for the slow subsystem and one for the fast subsystem. Using stability analysis and the theory of singular perturbations, the authors investigated and guaranteed the closed-loop stability for the full, nonlinear two-time-scale system under Lyapunov-based tracking MPC. To additionally consider process economics, a similar strategy was proposed in [4], which focused on the design of a composite controller involving two MPC problems. Specifically, for the stability of the fast subsystem (and full two-time-scale system as a result), a Lyapunov-based tracking MPC was used for the fast states based on the error between the fast states and the slow manifold using a quadratic cost function, while a Lyapunov-based economic MPC was used for the slow subsystem to optimize the economic considerations and achieve any other desirable closed-loop stability properties using a non-quadratic, economic cost function. The authors of [3,4] focused on first-principles-based controllers, and in [5], a two-time-scale system was separated into slow and fast subsystems using a singular perturbation strategy and then modeled using only data from the system via the method of sparse identification for nonlinear dynamics. Subsequently, an MPC was designed based on the reduced-order slow subsystem modeled using sparse identification, and closed-loop stability guarantees were derived. Furthermore, numerical simulations were used to demonstrate the reduced computational cost of the reduced-order sparse-identified model.

Due to the vast advancements in the chemical industry and the multitude of complex reactions occurring in real-life chemical process applications, engineers encounter challenges when constructing first-principles models for these chemical processes. As a result, in recent years, machine learning (ML) models have been utilized to approximate the dynamics of chemical processes. These models are highly beneficial when designed carefully, as they offer a reliable and efficient alternative to classical first-principles models and can then be incorporated into MPC frameworks. Many works have been conducted in this field. For instance, the authors of [6] offered essential insights and a fundamental theory regarding the integration of machine learning techniques into MPC schemes. Their work focused on designing a recurrent neural network (RNN) model to approximate the nominal nonlinear system and integrated the model within an MPC framework to stabilize the system. In [7], polynomial nonlinear autoregressive with exogenous inputs (NARX) models were used to build nonlinear MPC, with the relevant polynomial terms to retain selected via sparse regression. The proposed MPC was applied to a multi-input multi-output chemical reactor, and an algebraic modeling language was used to reduce computational time.

Another research direction termed “approximate MPC” involves the replacement of the MPC optimization problem with a closed-form expression that approximates the MPC control actions without having to solve an optimization problem. This is meant to address the computational challenges inherent in classical MPC at the expense of lower accuracy and generalizability. In [8], an approximate MPC was designed for heating, ventilation, and air-conditioning (HVAC) systems due to the stringent computational resources available for building control systems. Specifically, the MPC optimization problem was imitated using a recurrent neural network with a structure of a nonlinear autoregressive network with exogenous inputs trained with data from 30 days of operation in a closed loop under MPC. While both MPC and approximate MPC greatly reduced the cooling consumption of the HVAC system when tested, approximate MPC solved for the control actions over 100 times faster throughout the testing period, with only a 12% degradation in performance compared to MPC. An alternate research direction to reduce the computational burden of MPC is the design of “explicit MPC”, in which the optimal control actions are computed offline using multiparametric programming methods, allowing the online component of MPC to only have to search a table of linear gains and compute a single function evaluation. A summary of studies investigating explicit MPC is provided in [9]. Machine learning has also been investigated in the context of explicit MPC. In [10], for example, an explicit control law was learned for discrete-time linear time-invariant systems using machine learning. Specifically, the key challenge addressed was that of high dimensionality, which has been attempted to be solved in the existing literature by identifying suboptimal polytope partitions of the state space and designing control laws based on such regions. Ref. [10] focused on tackling high dimensionality in a similar manner but by using reinforcement learning and additionally investigated constraint satisfaction and feasibility guarantees for explicit MPC. Specifically, by using deep neural networks with rectified linear units as the activation function and a modified policy gradient algorithm based on prior knowledge, the objectives were met and demonstrated via three numerical examples of varying levels of dimensionality and complexity. A potential limitation of the approximate and explicit MPC approaches is that they are practically applicable to only smaller-scale systems, even though they are applicable to systems with very fast sampling times [9]. However, for general nonlinear systems and nonlinear MPC, due to the highly nonconvex optimization problem, methods such as the use of suboptimal polytopes [10] may not be readily extended to the nonlinear case.

While ML techniques have shown great success when incorporating them into MPC schemes, the practical application of machine learning-based MPC schemes to real-life processes poses significant challenges. The fact that the ML model is developed using a finite number of data samples makes it challenging to assess the accuracy of the model when considering generalized scenarios. Therefore, researchers have developed methods to quantify the error of the process model used in MPC and study its effects on closed-loop performance and stability. For example, ref. [11] proposed the use of Bayesian neural networks (BNN) to quantify the plant-model mismatch and subsequently create adaptive scenarios in real time for scenario-based MPC using the BNN. Based on the simulation results from a cold atmospheric plasma system, the proposed approach outperformed scenario-based MPC without adaptive scenarios and adaptive scenario-based MPC with Gaussian process regression. On more theoretical fronts, researchers have also studied the concept of generalization error bounds, which are crucial to study to ensure the construction of reliable models that are capable of performing accurate predictions on unseen or new data. Specifically, a low generalization error bound guarantees the effectiveness and high performance of the ML model and, consequently, the MPC scheme utilizing the designed ML model. To address this, ref. [12] derived a generalization error bound for a fully connected RNN (FCRNN) model, discussed the main factors that affect the bound, and integrated the FCRNN model into an MPC scheme to stabilize a nonlinear process. Moreover, the authors of [13] focused on deriving a generalization error bound for a feedforward neural network (FNN) that was used to construct a control Lyapunov-barrier function. The authors of [14] studied the generalization error bound for both a partially connected RNN (PCRNN) and an LSTM-RNN and subsequently carried out a comparison study between the generalization performance of an FCRNN and a PCRNN. Additionally, all the aforementioned works incorporated the designed machine learning models into Lyapunov-based MPC schemes to demonstrate the ability of the LMPCs to stabilize a chemical process. To the best of our knowledge, the application of generalization error bounds to neural networks modeling two-time-scale systems has not yet been investigated.

In light of the above considerations, in this article, we use the theory of statistical machine learning to construct the generalization error bounds for neural networks modeling two-time-scale systems. Moreover, the computational time for a neural network-based MPC is known to be an important practical consideration. However, the computational time for an RNN-based MPC with and without decomposition into lower-order subsystems has not yet been studied. Therefore, we introduce two LMPC frameworks, one designed based on an RNN model that predicts the slow state vector and the other designed based on an RNN model that models the full two-time-scale system, and compare the two LMPC schemes in terms of the closed-loop stability and computational time. The rest of this article is structured as follows. Section 2 introduces the essential preliminaries, as well as the general class of two-time-scale systems considered in this work. Section 3 discusses the generalization error bounds of neural networks modeling two-time-scale systems. In Section 4, we present a machine learning-based Lyapunov-based MPC using an RNN that predicts the slow state vector, followed by a closed-loop stability analysis and the corresponding results. In Section 5, a chemical process example is used to demonstrate the effectiveness of the designed controller. Section 6 summarizes the key findings obtained in this study.

2. Preliminaries

2.1. Notations

The Euclidean norm of a vector is denoted by $|·|$. The transpose of the vector x is given by $x^T$. Given a matrix B, its Frobenius norm is denoted as ${\parallel B\parallel }_F$. Moreover, the infinity norm of the 1-norms of the columns of B is denoted as ${\parallel B\parallel }_{1,\infty }={max}_j{\sum }_i\left|B_{i,j}\right|$. The expression $L_{F}V\left(x\right)$ denotes the standard Lie derivative $L_{F}V\left(x\right):=\frac{\partial V\left(x\right)}{x}f\left(x\right)$. Set subtraction is denoted by `∖’, i.e., $A\setminus B:=\left\{x\in {\mathbb{R}}^n|x\in A,x\notin B\right\}$. A function $f\left(x\right)$ is of class ${\mathcal{C}}^1$ if it is continuously differentiable in its domain. A continuous function $\alpha :[0,a)\to [0,\infty )$ belongs to class $\mathcal{K}$ if it is strictly increasing and is zero only when evaluated at zero. A function $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is said to be L-Lipschitz continuous if there exists $L\ge 0$, such that $\left|f\right(a)-f(b\left)\right|\le L|a-b|$ for all $a,b\in {\mathbb{R}}^n$. A continuous function $\beta :[0,a)\times [0,\infty )\to [0,\infty )$ belongs to class $\mathcal{KL}$ if, for each constant value of t, the function $\beta (·,t)$ is of class $\mathcal{K}$, and for each constant value of s, the function $\beta (s,·)$ is decreasing and approaches zero as $s\to \infty$. The probability that the event A will occur is denoted as $\mathbb{P}\left(A\right)$. Additionally, the expected value of a random variable X is denoted as $\mathbb{E}\left[X\right]$.

2.2. Class of Systems

The following family of ordinary differential equations describes the general class of two-time-scale continuous-time nonlinear systems with k states considered in this work:

$$\begin{array}{cc}\hfill \dot{x}& =f_1(x,z,u,ϵ)\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill ϵ\dot{z}& =f_2(x,z,ϵ)\hfill \end{array}$$

(1b)

where $x\in {\mathbb{R}}^n$ is the slow state vector, $z\in {\mathbb{R}}^p$ is the fast state vector, and $n+p=k$. $u\in {\mathbb{R}}^{q_1}$ is the bounded manipulated input vector. The input vector u is constrained by $u\in U:=\left\{\right|u_i|\le u_{1_{max,i}},i=1,\dots ,q_1\}$. We assume the vector functions $f_1(x,z,u,ϵ)$ and $f_2(x,z,ϵ)$ to be sufficiently smooth vector functions in ${\mathbb{R}}^n$ and ${\mathbb{R}}^p$, respectively. Moreover, we assume that the closed-loop stability region of the nonlinear system in Equation (1) is defined by the region ${\Omega }_{{\rho }_{{}_F}}$, where ${\Omega }_{{\rho }_{{}_F}}:=\{x\in D|V(x,z)\le {\rho }_{{}_F}\}$, where ${\rho }_{{}_F}>0$ and D is an open neighborhood around the origin. The speed ratio of the slow to the fast dynamics of the system is represented by the small positive parameter $ϵ$. In Equation (1b), we observe that the speed ratio $ϵ$ pre-multiplies the derivative of the fast state vector z, which enables us to utilize a well-known strategy called “reduced-order modeling” via singular perturbations. In this approach, we will be able to decompose the system in Equation (1) into two different reduced-order subsystems. We follow the strategy thoroughly illustrated in [2].

The following equations are obtained by setting $ϵ=0$ in Equation (1):

$$\begin{array}{cc}\hfill \dot{\overline{x}}& =f_1(\overline{x},\overline{z},\overline{u},0)\hfill \end{array}$$

(2a)

$$\begin{array}{cc}\hfill 0& =f_2(\overline{x},\overline{z},0)\hfill \end{array}$$

(2b)

where $\overline{x}$ and $\overline{z}$ are the slow state vector and the fast state vector associated with the system in Equation (1) under the case where $ϵ=0$, respectively. Additionally, the following assumption is considered to be fundamental in the theory of singular perturbations.

Assumption 1. 

Equation (2b) has an isolated solution, which is determined by

$$\overline{z}={\overline{f}}_2\left(\overline{x}\right)$$

(3)

where $\overline{z}$ is a quasi-steady state for the fast state z, and the function ${\overline{f}}_2:{\mathbb{R}}^n\to {\mathbb{R}}^p$ is continuously differentiable.

We substitute Equation (3) into Equation (2a) to obtain the reduced-order slow subsystem,

$$\dot{\overline{x}}=f_1(\overline{x},{\overline{f}}_2\left(\overline{x}\right),\overline{u},0)$$

(4)

Furthermore, to obtain the dynamics of the reduced-order fast subsystem, we introduce a fast timescale $\tau =t/ϵ$ and a new coordinate $\overline{\overline{z}}=z-\overline{z}$ to re-write Equation (1b) with respect to the newly introduced variables $\overline{\overline{z}}$ and $\tau$. Then, the fast subsystem can be expressed as the derivative of $\overline{\overline{z}}$ with respect to $\tau$ while setting $ϵ=0$,

$$\frac{\mathrm{d}\overline{\overline{z}}}{\mathrm{d}\tau }=f_2(\overline{x},\overline{\overline{z}}+{\overline{f}}_2\left(\overline{x}\right),0)$$

(5)

Starting at $t_0$, the initial conditions of the state vectors x and z are given by the vectors $x_0$ and $z_0$, respectively. Based on Equations (1)–(5), and by utilizing the basic theoretical concepts of two-time-scale systems in [2], the following equation will hold for all $t\in [t_p,T]$, where $t_p>t_0$

$$z\left(t\right)=\overline{z}\left(t\right)+\mathcal{O}\left(ϵ\right).$$

(6)

where $\overline{z}\left(t\right)$ reflects the slow transient of z, and $\mathcal{O}\left(ϵ\right)$ is an error of order epsilon. A variable, $z\left(t\right)$, is $\mathcal{O}\left(ϵ\right)$, where $ϵ$ is a positive constant if there exists a positive constant $\overline{k}$ (independent of $ϵ$), such that $\left|z\left(t\right)\right|\le \overline{k}ϵ$. Furthermore, if we were to consider the time interval where $t\in [t_0,T]$, Equation (6) would be slightly modified, such that the approximation of the state z is given by the following:

$$z\left(t\right)=\overline{z}\left(t\right)+\overline{\overline{z}}\left(t\right)+\mathcal{O}\left(ϵ\right),$$

(7)

where $\overline{z}\left(t\right)$ and $\overline{\overline{z}}\left(t\right)$ are the slow and fast transients of z, respectively. Additionally, the approximation of state x by $\overline{x}$ is given by the following equation for all $t\in [t_0,T]$

$$x\left(t\right)=\overline{x}\left(t\right)+\mathcal{O}\left(ϵ\right).$$

(8)

The fast subsystem in Equation (5) needs to satisfy certain stability properties for the above closeness of the solution estimates to hold true, which are described by the following assumption.

Assumption 2. 

The equilibrium $\overline{\overline{z}}\left(\tau \right)=0$ in Equation (5) demonstrates uniform asymptotic stability in $x_0$ and $t_0$. In addition, $z_0-\overline{z}\left(t_0\right)$ resides within its domain of attraction. As a result, for all $\tau \ge 0$, $\overline{\overline{z}}\left(\tau \right)$ exists.

If Assumption 2 holds true, then

$$\underset{\tau \to \infty }{lim}\overline{\overline{z}}\left(\tau \right)=0,$$

(9)

This implies that, at some time $t_p>t_0$, z will approach close to its quasi-steady state $\overline{z}$. The local stability of the equilibrium of the fast subsystem holds if the following verifiable condition holds.

Assumption 3. 

All the eigenvalues of $\frac{\partial f_2}{\partial z}$, computed for $ϵ=0$, along $\overline{x}\left(t\right),\overline{z}\left(t\right)$, exhibit real parts less than a constant negative value, i.e.,

$$\Re \lambda \left\{\frac{\partial f_2}{\partial z}\right\}\le -c<0.$$

(10)

Utilizing the aforementioned assumptions, we establish the well-known “Tikhonov’s theorem” in the following theorem.

Theorem 1 

(c.f. Theorem 3.1 in [2]). Consider that Assumptions 2 and 3 hold true. Then, for all $t\in [t_0,T]$, Equations (7) and (8) are valid. In addition, there exists a specific time instant $t_p\ge t_0$, such that Equation (6) is valid for all $t\in [t_p,T]$.

Several sources such as [15] are useful for analyzing slightly varied formulations of Theorem 1 and reviewing its proof. At this point, we have investigated several essential concepts of two-time-scale systems. We will assume that the error between x and $\overline{x}$ is not greater than $\mathcal{O}\left(ϵ\right)$. This assumption allows us to simplify the analysis by approximating $\overline{x}$ as x. Similarly, the same principle applies to z and $\overline{z}$, with our focus on capturing the slow transient of the fast state dynamics z, that is, $\overline{z}$. Therefore, based on this assumption, we express the reduced-order slow subsystem in Equation (4) as the following throughout this manuscript,

$$\dot{x}=F(x,u):=f\left(x\right)+g\left(x\right)u,x\left(t_0\right)=x_0,$$

(11)

where $f(·)$ and $g(·)$ are sufficiently smooth vector functions of dimensions $n\times 1$ and $n\times q_1$, respectively. Without loss of generality, we assume that the initial time $t_0=0$ and that $f\left(0\right)=0$; hence, the steady state of the nonlinear system in Equation (11) is the origin. Finally, we will assume that the fast subsystem is globally asymptotically stable, which is necessary for establishing the stability of the closed-loop system under model predictive control using machine learning models in Section 4.

Assumption 4. 

The origin of the closed-loop fast subsystem described by Equation (5) exhibits global asymptotic stability uniformly in x. This implies that there exists a function ${\beta }_{\overline{\overline{z}}}$ of class $\mathcal{K}\mathcal{L}$, such that for any $\overline{\overline{z}}\left(0\right)\in {\mathbb{R}}^p$,

$$|\overline{\overline{z}}\left(t\right)|\le {\beta }_{\overline{\overline{z}}}\left(|\overline{\overline{z}}\left(0\right)|,\frac{t}{ϵ}\right)\forall t\ge 0$$

(12)

2.3. Stabilizability Assumption via Control Lyapunov Function

Regarding the dynamics of the slow subsystem described in Equation (11), we assume that there exists a locally Lipschitz feedback controller, $\mathsf{\Phi }\left(x\right)\in U$, which renders the origin of the slow subsystem in Equation (11) asymptotically stable, i.e., there exists a ${\mathcal{C}}^1$ Lyapunov function $V\left(x\right)$ that is continuously differentiable and meets the following set of inequalities:

$$a_1\left(\right|x\left|\right)\le V\left(x\right)\le a_2\left(\right|x\left|\right),$$

(13a)

$$\frac{\partial V\left(x\right)}{\partial x}F(x,\mathsf{\Phi }\left(x\right))\le -a_3\left(\right|x\left|\right),$$

(13b)

$$\left|\frac{\partial V\left(x\right)}{\partial x}\right|\le a_4\left(\right|x\left|\right)$$

(13c)

where $a_1,a_2,a_3$, and $a_4$ are class $\mathcal{K}$ functions for all $x\in {\mathbb{R}}^n\subset D$. Additionally, given that the system $F(x,u)$ has a Lipschitz property and the input u is constrained by the set U, then there exist positive constants M, $L_x$, and $L_x^{{}^{\prime }}$, such that the subsequent inequalities hold for all $x,x^{\prime }\in D$, and $u\in U$:

$$\begin{array}{cc}\hfill & \left|F\right(x,u\left)\right|\le M\hfill \end{array}$$

(14a)

$$\begin{array}{cc}\hfill & |F(x,u)-F(x^{\prime },u)|\le L_x|x-x^{\prime }|\hfill \end{array}$$

(14b)

$$\begin{array}{cc}\hfill & \left|\frac{\partial V\left(x\right)}{\partial x}F(x,u)-\frac{\partial V\left(x^{\prime }\right)}{\partial x}F(x^{\prime },u)\right|\le L_x^{{}^{\prime }}|x-x^{\prime }|\hfill \end{array}$$

(14c)

Moreover, the region ${\Omega }_{\rho }:=\{x\in D|V\left(x\right)\le \rho \},\rho >0$, is defined as the stability region for the slow subsystem in Equation (11).

Now we discuss the incorporation of machine learning models, particularly neural networks, in process systems engineering. The first step in developing any type of neural network model is generating a comprehensive data set that effectively captures the input–output relation. This data set is used to train the network and enables it to learn patterns present in the data and make accurate predictions. To generate a data set that captures the dynamics of the nonlinear system in Equation (1) within the region ${\Omega }_{\rho }$, we follow the data generation technique described in [6]. The first step is to carry out various open-loop simulations of the nonlinear system in Equation (1), covering a wide range of initial conditions (i.e., $x_0\in {\Omega }_{\rho }$ and $z_0\in {\Omega }_{\rho }$) and valid input signals under sample-and-hold implementation (i.e., the input is applied to the system as piecewise constant functions, $u=u\left(t_k\right)\in U,\forall t\in [t_k,t_{k+1})$, where $t_{k+1}:=t_k+\Delta$, $\Delta$ is the sampling period). To integrate the nonlinear system in Equation (1), the well-established forward Euler approach is used with a sufficiently small integration time step $h_c\ll \Delta$. As a result, an extensive data set containing the dynamics of the system is generated, which is then utilized to train a neural network employing the Keras library.

2.4. Recurrent Neural Networks

Consider designing a recurrent neural network (RNN) model that predicts only the slow state vector x in Equation (1), or in other words, to approximate the dynamics of the slow subsystem in Equation (11). The network is trained using m data points $({\mathbf{x}}_{i,t},{\mathbf{y}}_{i,t})$ of T-time length, where $i=1,\dots ,m$ and $t=1,\dots ,T$. ${\mathbf{x}}_{i,t}\in {\mathbb{R}}^{d_x}$ and ${\mathbf{y}}_{i,t}\in {\mathbb{R}}^{d_y}$ are the RNN input and output, respectively. Additionally, $d_x$ and $d_y$ are the dimensions of the RNN input and output, respectively. It is worth mentioning that the notations used for the RNN inputs and outputs are represented in bold to establish a clear distinction between the RNN notations and the notations used to describe the nonlinear two-time-scale system in Equation (1). The RNN input ${\mathbf{x}}_{i,t}$ comprises the current slow state measurement, along with the manipulated inputs applied through the time steps $t=1,\dots ,T$, where the manipulated inputs are generated randomly within the set U. The RNN output ${\mathbf{y}}_{i,t}$ comprises the slow state predicted through the time steps $t=1,\dots ,T$. As a result, the RNN model is designed to make predictions of the slow states over one sampling period with $T=\frac{\Delta }{h_c}$ time steps. The data set utilized to develop the RNN model is constructed by generating m independent data sequences obtained from an underlying distribution over ${\mathbb{R}}^{d_x\times T}\times {\mathbb{R}}^{d_y\times T}$.

With the aim of simplification, we introduce a single-hidden-layer RNN model, given that ${\mathbf{h}}_i\in {\mathbb{R}}^{d_h}$ are the hidden states and can be evaluated as follows:

$${\mathbf{h}}_{i,t}={\sigma }_h(U{\mathbf{h}}_{i,t-1}+W{\mathbf{x}}_{i,t})$$

(15)

where the element-wise nonlinear activation function is denoted by ${\sigma }_h$. $U\in {\mathbb{R}}^{d_h\times d_h}$ is the weight matrix associated with the hidden states, whereas $W\in {\mathbb{R}}^{d_h\times d_x}$ is the weight matrix associated with the input vector. Furthermore, the following equation evaluates the output layer $y_{i,t}$ of the RNN model:

$${\mathbf{y}}_{i,t}={\sigma }_y\left(\mathcal{V}{\mathbf{h}}_{i,t}\right)$$

(16)

where the element-wise activation function of the output layer is denoted by ${\sigma }_y$, and $\mathcal{V}\in {\mathbb{R}}^{d_y\times d_h}$ is the weight matrix associated with the output layer. In this particular application, the input of the RNN model will be the current slow state measurement, $x\in {\mathbb{R}}^n$, as well as the manipulated input $u\in {\mathbb{R}}^{q_1}$ for the next sampling period. The output of the RNN model will be the predicted slow states x for at least one sampling period ahead. The basic structure of the RNN is represented in Figure 1.

Let $\breve{\mathbf{y}}$ be the predicted value and $\mathbf{y}$ be the actual value. We can then define the loss function $L(\mathbf{y},\breve{\mathbf{y}})$ that computes the squared difference between the actual and predicted values. More precisely, we will consider the $L_2$ (i.e., mean squared error) loss function. Without loss of generality, we establish the following standard assumptions that are required for the study of the generalization error bounds and are reviewed for completeness:

Assumption 5. 

The inputs of the RNN are bounded (i.e., $\parallel {\mathbf{x}}_{i,t}\parallel \le B_X$ for all $i=1,\dots ,m$ and $t=1,\dots ,T$).

Assumption 6. 

The Frobenius norms of the weight matrices are bounded (i.e., ${∥\mathcal{V}∥}_F\le B_{\mathcal{V},F},$${∥W∥}_F\le B_{W,F},{∥U∥}_F\le B_{U,F}$).

Assumption 7. 

${\sigma }_h$ is a positive homogeneous and 1-Lipschitz continuous nonlinear activation function (i.e., ${\sigma }_h\left(\alpha z\right)=\alpha {\sigma }_h\left(z\right)$ holds $\forall \alpha \ge 0$ and $z\in \mathbb{R}$).

Assumption 8. 

The data sets used for training, testing, and validation are drawn from the same distribution.

Remark 1. 

Although the RNN model is developed to approximate the dynamics of the slow subsystem described by Equation (11), we emphasize the fact that the data used to train this RNN model are constructed using the nonlinear two-time-scale system in Equation (1). The reason is that in most practical scenarios, engineers are limited to working with data obtained from the original nonlinear two-time-scale system in Equation (1). Additionally, it is possible to design an RNN model that predicts the full two-time-scale dynamics of Equation (1) using full-state measurements.

Remark 2. 

Assumption 5 takes into account the bounded nature of the RNN inputs. This aligns with the observation that the states x and the inputs u are restricted within certain limits, where $x\in {\Omega }_{\rho }$ and $u\in U$. Assumption 6 requires that the weight matrices of the RNN are bounded. This requirement can be met while training the RNN since the search for the optimal RNN parameters is limited to a finite class of neural network hypotheses. Assumption 8 indicates that the RNN model constructed using data derived from industrial operations will be utilized in the exact same process under the condition that the data distribution remains unchanged. It is important to point out that in the context of assessing the generalization performance for machine learning models, Assumption 8 is regarded as essential. Several well-known activation functions can be used to construct the RNN model that satisfy Assumption 7; the rectified linear unit (ReLu) activation function is one such example.

2.5. Feedforward Neural Networks

Using the RNN model described in Section 2.4, we are able to predict the slow state vector x. However, it is important to devise a methodology to predict the values of the fast states. In line with the framework proposed in [5], we consider the design of a feedforward neural network (FNN) that predicts the fast states using the previously predicted slow states from the RNN model. More precisely, the FNN input will be the slow states x, and its output will be the fast states z. The FNN model is trained using m independent data points from an underlying distribution over ${\mathbb{R}}^{d_x^F}\times {\mathbb{R}}^{d_y^F}$, where $d_x^F$ and $d_y^F$ are the dimensions of the FNN input and output, respectively. In this particular application, given that the FNN input is the slow state, $x\in {\mathbb{R}}^n$, and its output is the fast state $z\in {\mathbb{R}}^p$, we have $d_x^F=n$ and $d_y^F=p$. The general form of an FNN model can be formulated as follows:

$${\mathbf{y}}^F={\sigma }_d\left(Q_d{\sigma }_{d-1}\left(Q_{d-1}{\sigma }_{d-2}(\dots {\sigma }_1\left(Q_1{\mathbf{x}}^F\right))\right)\right)$$

(17)

where d is the total number of FNN layers, ${\mathbf{y}}^F\in {\mathbb{R}}^{d_y^F}$ is the predicted output of the FNN, and ${\mathbf{x}}^F\in {\mathbb{R}}^{d_x^F}$ is the FNN’s input. For each FNN layer l, where $l=1,\dots ,d$, the weight parameter matrix is denoted as $Q_l$, and the activation function is represented as ${\sigma }_l$. The depth of the network is represented by the number of layers d. However, the width of the network is the maximal number of neurons in a hidden layer and is represented by $h_{max}$. Given that $h_l$ denotes the number of neurons in the l^th layer, the width of the network can be expressed as $h_{max}={max}_{l=1,\dots ,d}\left\{h_l\right\}$. The general FNN structure is shown in Figure 2.

If ${\breve{\mathbf{y}}}^F$ is the predicted value and ${\mathbf{y}}^F$ is the actual value, we can define the loss function $L({\mathbf{y}}^F,{\breve{\mathbf{y}}}^F)$ that computes the squared difference between the actual and the predicted values. More precisely, we will consider the $L_2$ (i.e., mean squared error) loss function. Similar to the discussion on RNNs, we establish the following assumptions for the developed FNN model:

Assumption 9. 

The inputs of the FNN are bounded (i.e., $\parallel {\mathbf{x}}_i^F\parallel \le B_X$ for all $i=1,\dots ,m$).

Assumption 10. 

The norms of the weight matrices are bounded in the sense that the maximal 1-norm of the rows of the weight matrices in the hidden layers and output is bounded (i.e., ${∥Q∥}_{1,\infty }\le B_Q$).

Assumption 11. 

The activation function ${\sigma }_l$ is 1-Lipschitz continuous and satisfies ${\sigma }_l\left(0\right)=0$, where $l=1,\dots ,d$ (e.g., $tanh(·)$).

Assumption 9 is an implication of the assumption that the RNN output (slow states) is bounded. Assumptions 9–11 follow the same reasoning as Assumptions 5–7, as explained in Remark 2. Building upon the previous assumptions, it should be noted that Assumption 8 remains applicable to the FNN model.

3. Generalization Error Bounds of Neural Networks Modeling Two-Time-Scale Systems

The primary goal of constructing any neural network model is to make accurate predictions. Therefore, analyzing the generalization error bounds for various types of neural networks modeling different types of nonlinear systems enables us to improve the construction and design of these network models. Typically, the assessment of neural networks is conducted using a finite set of training samples. Hence, it is essential to investigate the generalization error bound for the neural network model, which allows us to evaluate the performance of the model in accurately predicting outcomes for new, unseen data. In other words, considering new data from the same distribution, the generalization error bound quantifies the ability of a neural network model to make accurate predictions for unseen data that the neural network has not encountered before.

In this work, we consider an RNN that predicts the dynamics of the slow states. Then, an FNN is utilized to predict the fast states using the predicted slow states as the input for the FNN. The generalization error bounds of the neural networks modeling two-time-scale systems are investigated. Particularly, in this section, we discuss the generalization error bounds of both the RNN (which predicts the slow states’ dynamics) and the FNN (which predicts the fast states). The generalization error bound is derived by utilizing the basic concepts of statistical machine learning theory. Hence, in the upcoming subsection, we outline the fundamental concepts and definitions used to derive the generalization error bounds. Additionally, it is worth mentioning that in the following subsection, the same principles that apply to $\mathbf{x}$,$\mathbf{y}$, and $\breve{\mathbf{y}}$ also extend to ${\mathbf{x}}^F$,${\mathbf{y}}^F$, and ${\breve{\mathbf{y}}}^F$, respectively. In the same vein, the principles that apply to $d_x$ and $d_y$ are also true for $d_x^F$ and $d_y^F$, respectively. Let $\mathcal{H}$ be the hypothesis class of RNN functions $h(·)$ that map the $d_x$-dimensional input $\mathbf{x}\in {\mathbb{R}}^{d_x}$ to the $d_y$-dimensional output $\breve{\mathbf{y}}\in {\mathbb{R}}^{d_y}$. Similarly, let ${\mathcal{H}}^F$ be the hypothesis class of FNN functions $h^F(·)$ that map the $d_x$-dimensional input ${\mathbf{x}}^F\in {\mathbb{R}}^{d_x}$ to the $d_y$-dimensional output ${\breve{\mathbf{y}}}^F\in {\mathbb{R}}^{d_y}$. We also note that in the following subsection, the principles that apply to $\mathcal{H}$ and $h(·)$ are also valid for ${\mathcal{H}}^F$ and $h^F(·)$, respectively. It should be noted that the dimensions $d_x$ and $d_y$ differ based on whether the network is an RNN or an FNN.

3.1. Generalization Error Bound Preliminaries

Definition 1. 

Consider a function, h, which makes predictions of the output $\mathbf{y}$ corresponding to the input $\mathbf{x}$, along with an underlying distribution D. The generalization error or the expected loss is formulated as follows:

$$\mathbb{E}\left[L(h\left(\mathbf{x}\right),\mathbf{y})\right]={\int }_{X\times Y}L(h\left(\mathbf{x}\right),\mathbf{y})\rho (\mathbf{x},\mathbf{y})\mathrm{d}\mathbf{x}\mathrm{d}\mathbf{y}.$$

(18)

where $\rho (\mathbf{x},\mathbf{y})$ represents the joint probability distribution for $\mathbf{x}$ and $\mathbf{y}$, whereas X and Y represent the vector space for all possible inputs and outputs, respectively, and L denotes the loss function.

We introduce the following definition of the empirical error as an approximation of the expected loss due to the fact that the joint probability distribution $\rho$ is unknown in many scenarios.

Definition 2. 

Considering a data set consisting of m data samples $S=(s_1,\dots ,s_m)$, with each $s_i=({\mathbf{x}}_i,{\mathbf{y}}_i)$, the empirical error or risk can be expressed as

$${\hat{\mathbb{E}}}_S\left[L(h\left(\mathbf{x}\right),\mathbf{y})\right]=\frac{1}{m}\sum _{i=1}^{m}L(h\left({\mathbf{x}}_i\right),{\mathbf{y}}_i)$$

(19)

We note that the m data samples are gathered from the same data distribution. In this work, the loss function $L(\mathbf{y},\breve{\mathbf{y}})$ is chosen as the mean squared error (i.e., $L_2$ loss function). Furthermore, the generalization error bounds are derived using “Rademacher complexity”, a widely recognized concept in the field of machine learning theory that is used to determine the complexity and richness of a class of functions. The Rademacher complexity is defined as follows:

Definition 3. 

Given a data set $S=\{s_1,\dots ,s_m\}$ with m samples, and a hypothesis class $\mathcal{F}$ of real-valued functions, the empirical Rademacher complexity of $\mathcal{F}$ can be defined as follows:

$${\mathcal{R}}_S\left(\mathcal{F}\right)={\mathbb{E}}_{ϵ}\left[\underset{f\in \mathcal{F}}{sup}\frac{1}{m}\sum _{i=1}^m{ϵ}_{i}f\left(s_i\right)\right]$$

(20)

where $ϵ={({ϵ}_1,\dots ,{ϵ}_m)}^T$ consists of the Rademacher random variables ${ϵ}_i$. These variables are independent and identically distributed (i.i.d.), satisfying the condition that $\mathbb{P}({ϵ}_i=-1)=\mathbb{P}({ϵ}_i=1)=0.5$.

Given that ${\mathcal{G}}_t$ is the class of loss functions associated with the function class $\mathcal{H}$ and is defined as follows

$${\mathcal{G}}_t=\{g_t:(\mathbf{x},\mathbf{y})\to L(h\left(\mathbf{x}\right),\mathbf{y}),h\in \mathcal{H}\},$$

(21)

where $\mathbf{x}$ and $h\left(\mathbf{x}\right)$ are the model’s input and output, respectively, whereas $\mathbf{y}$ denotes the actual value of the output, the upper bound of the generalization error can be computed using the Rademacher complexity ${\mathcal{R}}_S\left({\mathcal{G}}_t\right)$, as shown in the following Lemma.

Lemma 1 

(c.f. Theorem 3.3 in [16]). Given a data set $S=\left(s_i\right)$, $i=1,\dots ,m$, that consists of m i.i.d. data samples, where $s_i={({\mathbf{x}}_{i,t},{\mathbf{y}}_{i,t})}_{t=1}^T$ for an RNN and $s_i=({\mathbf{x}}_i,{\mathbf{y}}_i)$ for an FNN, with probability $1-\delta$, the following inequality holds for all $g_t\in {\mathcal{G}}_t$ over the data samples S:

$$\mathbb{E}\left[g_t(\mathbf{x},\mathbf{y})\right]\le \frac{1}{m}\sum _{i=1}^m{g}_t({\mathbf{x}}_i,{\mathbf{y}}_i)+2{\mathcal{R}}_S\left({\mathcal{G}}_t\right)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}$$

(22)

For a comprehensive proof of Lemma 1, interested readers can refer to [12,16]. By observing Equation (22), clearly, the upper bound of the generalization error relies on three terms. The first term is the empirical loss ($\frac{1}{m}{\sum }_{i=1}^m{g}_t({\mathbf{x}}_i,{\mathbf{y}}_i)$). The second term is the Rademacher complexity ($2{\mathcal{R}}_S\left({\mathcal{G}}_t\right)$), whereas the third term depends on the data set size m, along with the confidence level $\delta$. At this stage, the goal is to be able to effectively quantify the generalization error bound using predefined and measurable values. This can be achieved by imposing a further upper bound on the Rademacher complexity term. Having established the background and basic concepts of the generalization error bound, our next discussion focuses on examining the generalization error bounds for neural networks modeling two-time-scale systems.

3.2. RNN Generalization Error Bound

In this subsection, we investigate the generalization error bound for the RNN model that predicts the slow dynamics of the two-time-scale system defined in Equation (1). We start by introducing the following Lemma, which upper-bounds the Rademacher complexity in terms of the RNN parameters.

Lemma 2. 

Let ${\mathcal{H}}_{k,t}$, $k=1,\dots ,d_y$ represent the class of real-valued functions associated with the k^th component of the RNN output at the t^th time step, with the activation functions and weight matrices satisfying Assumptions 5–8. Given a data set $S={({\mathbf{x}}_{i,t},{\mathbf{y}}_{i,t})}_{t=1}^T,i=1,\dots ,m$, consisting of m i.i.d. data samples, the following inequality holds for the Rademacher complexity:

$${\mathcal{R}}_S\left({\mathcal{H}}_{k,t}\right)\le \frac{M(\sqrt{2log\left(2\right)t}+1)B_X}{\sqrt{m}}$$

(23)

where $M=B_{\mathcal{V},F}{B}_{W,F}\frac{B_{U,F}^t-1}{B_{U,F}^{-1}}$, and $B_X$ is the upper bound for the RNN inputs.

For a comprehensive and detailed proof of Lemma 2, interested readers may refer to [12]. By applying the results derived and proven in [12], the following Lemma specifically addresses the generalization error bound for the RNN model that predicts the slow dynamics of the two-time-scale system defined in Equation (1).

Lemma 3. 

Consider the general class of the two-time-scale continuous-time nonlinear systems described in Equation (1) under the assumption that the slow and fast states in Equation (1) are stable and the perturbation parameter ϵ is sufficiently small. An RNN satisfying Assumptions 5–8 is constructed to predict the slow states of the system at the t^th time step. Given the $L_r$–Lipschitz loss function that belongs to the family of loss functions ${\mathcal{G}}_t$ associated with the RNN function of class ${\mathcal{H}}_t$ and a data set $S={({\mathbf{x}}_{i,t},{\mathbf{y}}_{i,t})}_{t=1}^T$, $i=1,\dots ,m$, with m i.i.d. data samples, the following inequality holds true with a probability of at least $1-\delta$ over S for all $t\in [t_0,T]$:

$$\mathbb{E}\left[g_t^{RNN}(\breve{x},x)\right]\le \frac{1}{m}\sum _{i=1}^m{g}_t({\breve{x}}_i,x_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{M{B}_X(1+\sqrt{2log\left(2\right)t})}{\sqrt{m}}\right)$$

(24)

where $M=B_{\mathcal{V},F}{B}_{W,F}\frac{B_{U,F}^t-1}{B_{U,F}^{-1}}$, and $B_X$ is the upper bound for the RNN inputs.

Equation (24) indicates that the generalization error bound of the RNN that predicts the slow state vector x depends on a number of factors, such as the number of training samples m, the time length t of the RNN’s inputs, the upper bound on the input vector $B_X$, and the complexity hypothesis class in terms of the weight matrices M, as well as the empirical loss ($\frac{1}{m}{\sum }_{i=1}^m{g}_t({\breve{x}}_i,x_i)$). We emphasize that the actual slow state is denoted as x, and the slow state predicted by the RNN model is $\breve{x}$. Additionally, based on Equation (8), the generalization error bound in Equation (24) holds for all $t\in [t_0,T]$ under the assumption that the slow and fast states are stable, and the perturbation parameter $ϵ$ is sufficiently small.

Now we discuss the implementation of the RNN’s generalization error bound for a specific loss function. A locally Lipschitz continuous loss function is used to optimize the RNN’s weights and biases. To be more precise, we employ the MSE loss function of the form,

$$L=\frac{1}{m}\sum _{i=1}^m{({\breve{\mathbf{y}}}_i,{\mathbf{y}}_i)}^2,$$

(25)

where $\breve{\mathbf{y}}$ and $\mathbf{y}$ are the predicted and actual values, respectively. Since the loss function L is locally Lipschitz continuous, it satisfies the following inequality,

$$|L(\mathbf{y},{\breve{\mathbf{y}}}_{\mathbf{2}})-L(\mathbf{y},{\breve{\mathbf{y}}}_{\mathbf{1}})|\le L_r|{\breve{\mathbf{y}}}_{\mathbf{2}}-{\breve{\mathbf{y}}}_{\mathbf{1}}|$$

(26)

where $L_r$ is the local Lipschitz constant for the loss function L. Since the RNN model predicts the slow states of Equation (1), the loss function is computed over the actual and predicted slow states, x and $\breve{x}$, respectively. Moreover, the expected loss of L is upper-bounded by the following inequality, with a probability of at least $1-\delta$:

$$\mathbb{E}\left[L(\breve{x},x)\right]\le \frac{1}{m}\sum _{i=1}^{m}L({\breve{x}}_i,x_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{M{B}_X(1+\sqrt{2log\left(2\right)t})}{\sqrt{m}}\right)$$

(27)

Given that the MSE loss function L computes the error between the RNN’s output x and the RNN’s predicted output $\breve{x}$, the upper bound on $|\breve{x}-x|$ can be written as follows:

$$|\breve{x}-x|\le \sqrt{\frac{1}{m}{\sum }_{i=1}^{m}L({\breve{x}}_i,x_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{M{B}_X(1+\sqrt{2log\left(2\right)t})}{\sqrt{m}}\right)}$$

(28)

3.3. FNN Generalization Error Bound

In this subsection, we investigate the generalization error bound for the FNN model that predicts the fast states of the two-time-scale system defined in Equation (1). We start by introducing the following Lemma, which upper-bounds the Rademacher complexity in terms of the FNN parameters.

Lemma 4 

(c.f. Theorem 2 in [17]). Given a class of scalar-valued functions ${\mathcal{H}}_k^F$ and a neural network with depth d, where ${∥Q_1∥}_{1,\infty }\le B_Q$ for all $l=1,\dots ,d$ and Assumptions 8–11 are satisfied, the following inequality holds for the Rademacher complexity:

$${\mathcal{R}}_S\left({\mathcal{H}}_k^F\right)\le \frac{2B_X{\left(B_Q\right)}^d\sqrt{d+1+log\left(d_x\right)}}{\sqrt{m}}$$

(29)

For a comprehensive and detailed proof of Lemma 4, interested readers may refer to [17]. By applying the results derived and proven in [13], the following Lemma specifically addresses the generalization error bound for the FNN model that predicts the fast states using the slow states of the two-time-scale system defined in Equation (1).

Lemma 5. 

Consider the general class of two-time-scale continuous-time nonlinear systems described in Equation (1) under the assumption that the slow and fast states are stable, and the perturbation parameter ϵ is sufficiently small. An FNN satisfying Assumptions 8–11 is constructed to predict the fast states using the slow states. Given the $L_r$–Lipschitz loss functions associated with the vector-valued FNN hypothesis class ${\mathcal{H}}^F$ and a data set S consisting of m i.i.d. data samples, the following inequality holds true, with a probability of at least $1-\delta$ over S for all $t\in [t_p,T]$, where $t_p$ is defined in Theorem 1:

$$\mathbb{E}\left[g_t^{FNN}(\breve{z},z)\right]\le \frac{1}{m}\sum _{i=1}^m{g}_t({\breve{z}}_i,z_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{B_X{\left(B_Q\right)}^d\sqrt{d+1+log\left(d_x\right)}}{\sqrt{m}}\right)$$

(30)

Equation (30) indicates that the generalization error bound of the FNN modeling the fast subsystem depends on several factors, such as the number of training samples m, the upper bound on the input vector $B_X$, the complexity hypothesis class in terms of the upper bound of the weight matrices $B_Q$, and the number of layers d, as well as the empirical loss ($\frac{1}{m}{\sum }_{i=1}^{m}L({\breve{z}}_i,z_i)$). We emphasize that the actual fast state is denoted as z, and the fast state predicted by the FNN model is $\breve{z}$. Therefore, the generalization error bound of Equation (30) holds for all $t\in [t_p,T]$ under the assumption that the slow and fast states are stable, and the perturbation parameter $ϵ$ is sufficiently small.

Now we discuss the implementation of the FNN’s generalization error bound specifically on the MSE loss function. A locally Lipschitz continuous loss function such as the MSE is used to optimize the FNN’s weights and biases. The MSE loss function satisfies the inequality of Equation (26). Since the FNN model predicts the fast states of Equation (1), the loss function is computed over the actual and predicted fast states, z and $\breve{z}$, respectively. Moreover, the expected loss of L is upper-bounded by the following inequality, with a probability of at least $1-\delta$:

$$\mathbb{E}\left[L(\breve{z},z)\right]\le \frac{1}{m}\sum _{i=1}^{m}L({\breve{z}}_i,z_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{B_X{\left(B_Q\right)}^d\sqrt{d+1+log\left(d_x\right)}}{\sqrt{m}}\right)$$

(31)

Given that the MSE loss function L computes the error between the RNN’s output z and the RNN’s predicted output $\breve{z}$, the upper bound on $|\breve{z}-z|$ can be written as follows:

$$|\breve{z}-z|\le \sqrt{\frac{1}{m}{\sum }_{i=1}^{m}L({\breve{z}}_i,z_i)+3\sqrt{\frac{log\left(\frac{2}{\delta }\right)}{2m}}+\mathcal{O}\left(L_r{d}_y\frac{B_X{\left(B_Q\right)}^d\sqrt{d+1+log\left(d_x\right)}}{\sqrt{m}}\right)}$$

(32)

4. Machine Learning-Based LMPC Using an RNN That Approximates the Slow Subsystem

Having investigated the generalization error bounds for neural networks modeling two-time-scale systems, the rest of this manuscript focuses on the design and performance of LMPC using RNN models with and without time-scale decomposition. Specifically, the objective is to design an RNN to predict only the slow state vector x, following the approach outlined in Section 2.4, and then incorporate the designed RNN model within an LMPC scheme to show that it is sufficient to stabilize the full two-time-scale system in Equation (1). In this section, we present a stability analysis for the proposed LMPC framework. For simplicity, in this section, we present the RNN model that predicts the slow state vector x as a continuous-time nonlinear system of the following form:

$$\dot{\hat{x}}=F_{nn}(\hat{x},u):=A\hat{x}+{\mathsf{\Theta }}^T\hat{z}$$

(33)

where $\widehat{x}\in {\mathbb{R}}^n$ is the RNN’s state vector and $u\in {\mathbb{R}}^{q_1}$ is the manipulated input vector. The weight matrices are denoted as A and $\mathsf{\Theta }$. The diagonal coefficient matrix A has negative diagonal entries. $\mathsf{\Theta }$ is defined as $\mathsf{\Theta }=[{\theta }_1,\dots ,{\theta }_n]\in {\mathbb{R}}^{(q_1+n)\times n}$, with entries ${\theta }_i=b_i[w_{i1},\dots ,w_{i(q_1+n)}],i=1,\dots ,n$, given that $w_{ij}$ represents the weight associated with the connection between the ith neuron and the jth output, with i ranging from 1 to n and j ranging from 1 to $(q_1+n)$. The vector $\widehat{z}=[{\widehat{z}}_1,\dots ,{\widehat{z}}_n,{\widehat{z}}_{n+1},\dots ,{\widehat{z}}_{n+q_1}]=[\sigma \left({\widehat{x}}_1\right)\dots \sigma \left({\widehat{x}}_n\right),u\dots u_{q_1}]\in {\mathbb{R}}^{n+q_1}$ consists of both the network states $\widehat{x}$ and inputs u. Additionally, a nonlinear activation function $\sigma (·)$ is applied to the network states $\widehat{x}$ when constructing the vector $\widehat{z}$. It is worth mentioning that the weight matrices and activation functions satisfy Assumptions 5–8. For simplicity, we assume a single-hidden-layer RNN model represented by Equation (33), which does not explicitly include bias terms. Nevertheless, it should be emphasized that the results obtained in this section are not limited to single-hidden-layer RNN models, but can be generalized to include deep RNN models that consist of multiple hidden layers.

4.1. Lyapunov-Based Control Using an RNN Model

Taking into account the RNN model designed to predict the values of the slow states, we assume that there exists a stabilizing control law ${\mathsf{\Phi }}_{nn}\left(x\right)\in U$ (e.g., a P-controller, a Lyapunov-based controller) that can render the origin of the RNN model in Equation (33) asymptotically stable in an open neighborhood around the origin denoted as $\widehat{D}$. Hence, there exists a continuously differentiable Lyapunov function $\widehat{V}:{\mathbb{R}}^n\to {\mathbb{R}}_{\ge 0}$ that satisfies the following inequalities:

$${\widehat{a}}_1\left(\right|x\left|\right)\le \widehat{V}\left(x\right)\le {\widehat{a}}_2\left(\right|x\left|\right)$$

(34a)

$$\frac{\partial \widehat{V}\left(x\right)}{\partial x}{F}_{nn}(x,{\mathsf{\Phi }}_{nn}\left(x\right))\le -{\widehat{a}}_3\left(\right|x\left|\right)$$

(34b)

$$\left|\frac{\partial \widehat{V}\left(x\right)}{\partial x}\right|\le {\widehat{a}}_4\left(\right|x\left|\right)$$

(34c)

where ${\widehat{a}}_i$ are class $\mathcal{K}$ functions for all $x\in {\mathbb{R}}^n\subset \widehat{D}$. The Lyapunov function $\widehat{V}$ can be designed using a quadratic formulation, which satisfies the required criteria, or, alternately, using learning-based approaches, as conducted using a linear parameter-varying (LPV) framework in [18]. In the simulation example presented in our work below, a quadratic Lyapunov function is used, and extensive open-loop and closed-loop simulations are conducted to verify that the asymptotic stability assumption is met. The Lyapunov level set that ensures the stability of the RNN model is defined as ${\Omega }_{\widehat{\rho }}:=\{x\in \widehat{D}|\widehat{V}\left(x\right)\le \widehat{\rho }\}$, where $\widehat{\rho }>0$. This implies that the closed-loop stability region of the RNN model in Equation (33) is denoted as ${\Omega }_{\widehat{\rho }}$. Moreover, there exist positive constants $M_{nn}$ and $L_{nn}$, such that the following inequalities are satisfied for all $x,x^{\prime }\in {\Omega }_{\widehat{\rho }}$, and $u\in U$:

$$|F_{nn}(x,u)|\le M_{nn}$$

(35a)

$$\left|\frac{\partial \widehat{V}\left(x\right)}{\partial x}{F}_{nn}(x,u)-\frac{\partial \widehat{V}\left(x^{\prime }\right)}{\partial x}{F}_{nn}(x^{\prime },u)\right|\le L_{nn}|x-x^{\prime }|$$

(35b)

The feedback controller $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$ can stabilize the dynamics of the slow subsystem in Equation (11) under a sufficiently small modeling error between the nominal slow subsystem and the RNN model. This result is shown in the following proposition.

Proposition 1. 

Consider the RNN model in Equation (33) that satisfies the stabilizability conditions of Equation (34) and is rendered asymptotically stable around the origin under the control law $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$ for all $x\in {\Omega }_{\widehat{\rho }}$. Then, the origin of the slow subsystem in Equation (11) is asymptotically stable if there exists a positive real number ${\nu }_m$, where ${\nu }_m<{\widehat{a}}_3\left(\right|x\left|\right)/{\widehat{a}}_4\left(\right|x\left|\right)$, such that the modeling error between the slow subsystem in Equation (11) and the RNN model in Equation (33) (i.e., $\nu =|F(x,u)-F_{nn}(x,u)|$) is upper-bounded by ${\nu }_m$ for all $x\in {\Omega }_{\widehat{\rho }}$.

Proof. 

We follow the strategy outlined in [5] to prove Proposition 1. The main objective is to show that, under the control law $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$, the slow subsystem in Equation (11) is asymptotically stable around the origin for all $x\in {\Omega }_{\widehat{\rho }}$ if ${\mathsf{\Phi }}_{nn}\left(x\right)$ renders the RNN designed to approximate the slow subsystem asymptotically stable with a bounded modeling error between the nominal slow subsystem and the RNN model. In other words, we show that $\dot{\widehat{V}}\left(x\right)\le 0$ for the slow subsystem in Equation (11) under the controller $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$ based on the RNN model. We utilize the inequalities in Equations (34b) and (34c) and compute $\dot{\widehat{V}}\left(x\right)$ as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}& =\frac{\partial \widehat{V}\left(x\right)}{\partial x}F(x,{\mathsf{\Phi }}_{nn}\left(x\right))\hfill \\ \hfill & =\frac{\partial \widehat{V}}{\partial x}(F_{nn}(x,{\mathsf{\Phi }}_{nn}\left(x\right))+F(x,{\mathsf{\Phi }}_{nn}\left(x\right))-F_{nn}(x,{\mathsf{\Phi }}_{nn}\left(x\right))\hfill \\ \hfill & \le -{\widehat{a}}_3\left(\right|x\left|\right)+{\widehat{a}}_4\left(\right|x\left|\right)(F(x,{\mathsf{\Phi }}_{nn}\left(x\right))-F_{nn}(x,{\mathsf{\Phi }}_{nn}\left(x\right)))\hfill \\ \hfill & \le -{\widehat{a}}_3\left(\right|x\left|\right)+{\nu }_m{\widehat{a}}_4\left(\right|x\left|\right)\hfill \end{array}$$

(36)

By assigning ${\nu }_m<\frac{{\widehat{a}}_3\left(\right|x\left|\right)}{{\widehat{a}}_4\left(\right|x\left|\right)}$, we obtain $\dot{\widehat{V}}\left(x\right)\le -{\tilde{a}}_3\left(\right|x\left|\right)\le 0$, where ${\tilde{a}}_3\left(\right|x\left|\right)=-{\widehat{a}}_3\left(\right|x\left|\right)+{\nu }_m{\widehat{a}}_4\left(\right|x\left|\right)>0$ since ${\widehat{a}}_3$ and ${\widehat{a}}_4$ are known functions chosen in such a way as to ensure that the above result holds. To show that if the general class $\mathcal{K}$ functions are chosen as ${\widehat{a}}_3=a_3\left|x\right|$ and ${\widehat{a}}_4=a_4\left|x\right|$, where $a_3$ and $a_4$ are constants, then ${\nu }_m=\frac{a_3}{a_4}$. Hence, we guarantee the closed-loop stability of the slow subsystem in Equation (11) around the origin under the control law ${\mathsf{\Phi }}_{nn}\left(x\right)\in U$ for all $x\in {\Omega }_{\widehat{\rho }}$. □

The RNN model designed in Equation (33) is incorporated into an LMPC scheme, where the control actions computed by the LMPC are implemented in a sample-and-hold fashion. Moreover, this sample-and-hold implementation of the LMPC control law $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$ establishes certain properties. These properties are studied in the following two propositions. Specifically, the following proposition shows that the error between the state of the slow subsystem in Equation (11) and the state predicted by the RNN model in Equation (33) is bounded.

Proposition 2. 

Consider the RNN model in Equation (33) and the slow subsystem in Equation (11), starting with the same initial condition $x_0={\widehat{x}}_0\in {\Omega }_{\widehat{\rho }}$. There exists a class $\mathcal{K}$ function $f_w(·)$ and a positive constant κ, such that the following inequalities are satisfied for all $x,\widehat{x}\in {\Omega }_{\widehat{\rho }}$:

$$\begin{array}{cc}\hfill & |x\left(t\right)-\widehat{x}\left(t\right)|\le f_w\left(t\right):=\frac{{\nu }_m}{L_x}\left(e^{L_{x}t}-1\right)\hfill \end{array}$$

(37a)

$$\begin{array}{cc}\hfill & \widehat{V}\left(x\right)\le \widehat{V}\left(\widehat{x}\right)+{\widehat{a}}_4\left({\widehat{a}}_1^{-1}\left(\widehat{\rho }\right)\right)|x-\widehat{x}|+\kappa |x-\widehat{x}{|}^2\hfill \end{array}$$

(37b)

Proof. 

Considering that $e\left(t\right)=x\left(t\right)-\widehat{x}\left(t\right)$ represents the error vector between the state of the slow subsystem in Equation (11) and the state of the RNN model in Equation (33), the bound for the time derivative of $e\left(t\right)$ is given as follows:

$$\begin{array}{cc}\hfill |\dot{e}\left(t\right)|& =|F(x,u)-F_{nn}(\widehat{x},u)|\hfill \\ \hfill & \le |F(x,u)-F(\widehat{x},u)|+|F(\widehat{x},u)-F_{nn}(\widehat{x},u)|\hfill \end{array}$$

(38)

Using Equation (14b), for all $x,\widehat{x}\in {\Omega }_{\widehat{\rho }}$, the first term in Equation (38) can be bounded as follows:

$$\begin{array}{cc}\hfill |F(x,u)-F(\widehat{x},u)|& \le L_x|x\left(t\right)-\widehat{x}\left(t\right)|\hfill \\ \hfill & \le L_x|x\left(t\right)-\widehat{x}\left(t\right)|\hfill \end{array}$$

(39)

The term $|F(\widehat{x},u)-F_{nn}(\widehat{x},u)|$ in Equation (39) denotes the modeling error, and it is upper-bounded by ${\nu }_m$ for all $\widehat{x}\in {\Omega }_{\widehat{\rho }}$. Hence, the term $\dot{e}\left(t\right)$ can be further bounded by utilizing the bound of the modeling error and the bound of Equation (39), as follows:

$$\begin{array}{cc}\hfill |\dot{e}\left(t\right)|& \le L_x|x\left(t\right)-\widehat{x}\left(t\right)|+{\nu }_m\hfill \\ \hfill & \le L_x\left|e\left(t\right)\right|+{\nu }_m\hfill \end{array}$$

(40)

Taking into account the zero initial condition (i.e., $e\left(0\right)=0$), we integrate the inequality in Equation (40) and obtain the following upper bound for the error vector for all $x,\widehat{x}\in {\Omega }_{\widehat{\rho }}$:

$$\left|e\left(t\right)\right|=|x\left(t\right)-\widehat{x}\left(t\right)|\le \frac{{\nu }_m}{L_x}\left(e^{L_{x}t}-1\right)$$

(41)

Equation (37b) can be derived using the Taylor series expansion of $\widehat{V}\left(x\right)$ around $\widehat{x}$ for all $x,\widehat{x}\in {\Omega }_{\widehat{\rho }}$ as follows:

$$\widehat{V}\left(x\right)\le \widehat{V}\left(\widehat{x}\right)+\frac{\partial \widehat{V}\left(\widehat{x}\right)}{\partial x}|x-\widehat{x}|+\kappa |x-\widehat{x}{|}^2$$

(42)

where $\kappa$ is a positive real number. Furthermore, Equations (34a) and (34b) are used to further upper-bound $\widehat{V}\left(x\right)$ as follows:

$$\widehat{V}\left(x\right)\le \widehat{V}\left(\widehat{x}\right)+{\widehat{a}}_4\left({\widehat{a}}_1^{-1}\left(\widehat{\rho }\right)\right)|x-\widehat{x}|+\kappa |x-\widehat{x}{|}^2$$

(43)

□

The following proposition demonstrates that the closed-loop state of the slow subsystem described in Equation (11) is maintained within the stability region ${\Omega }_{\widehat{\rho }}$ at all times. Additionally, utilizing the Lyapunov-based controller $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$ through sample-and-hold implementation ensures that the closed-loop state of the slow subsystem can be ultimately bounded within a small region around the origin ${\Omega }_{{\rho }_{min}}$.

Proposition 3 

(c.f Proposition 3 in [5]). Consider the slow subsystem in Equation (11) under the controller ${\mathsf{\Phi }}_{nn}\left(\widehat{x}\right)\in U$ that satisfies the conditions in Equation (34) and is implemented in a sample-and-hold fashion (i.e., ${\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right),\forall t\in [t_k,t_{k+1})$, where $t_{k+1}:=t_k+\Delta$) to stabilize the RNN model in Equation (33). Then, there exist ${ϵ}_w>0$, $\Delta >0$ and $\widehat{\rho }>{\rho }_{min}>{\rho }_{nn}>{\rho }_s$ that satisfy

$$\begin{array}{cc}\hfill & -{\widehat{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_{nn}{M}_{nn}\Delta \le -{ϵ}_s\hfill \end{array}$$

(44a)

$$\begin{array}{cc}\hfill & -{\tilde{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_x^{{}^{\prime }}{M}_F\Delta \le -{ϵ}_w\hfill \end{array}$$

(44b)

and

$$\begin{array}{cc}\hfill {\rho }_{nn}& :=max\left\{\widehat{V}\left(\widehat{x}(t+\Delta )\right)\right|\widehat{x}\left(t\right)\in {\Omega }_{{\rho }_s},u\in U\}\hfill \end{array}$$

(45a)

$$\begin{array}{cc}\hfill {\rho }_{min}& \ge {\rho }_{nn}+-{\widehat{a}}_4\left({\widehat{a}}_1^{-1}\left(\widehat{\rho }\right)\right)f_w(\Delta )+\kappa {\left(f_w(\Delta )\right)}^2\hfill \end{array}$$

(45b)

such that, for any $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$, there exists a class $\mathcal{KL}$ function ${\beta }_x$ and a class $\mathcal{K}$ function $\overline{\gamma }$, such that the following inequality is satisfied:

$$\left|x\left(t\right)\right|\le {\beta }_x\left(\right|x\left(0\right)|,t)+\overline{\gamma }\left({\rho }_{min}\right)$$

(46)

and the state $x\left(t\right)$ of the nominal slow subsystem in Equation (11) is bounded in ${\Omega }_{\widehat{\rho }}$ for all times and ultimately bounded in ${\Omega }_{{\rho }_{min}}$.

Proof. 

Assuming that $x\left(t_k\right)=\widehat{x}\left(t_k\right)$, the first part of establishing this proof involves showing that $\widehat{V}\left(\widehat{x}\right)$ is decreasing under the control law $u\left(t\right)={\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right)\in U$ for $t\in [t_k,t_{k+1})$, where $x\left(t_k\right)$ and $\widehat{x}\left(t_k\right)$ are the state of the slow subsystem in Equation (11) and the state of the RNN model in Equation (33), respectively. The time derivative of $\widehat{V}\left(\widehat{x}\right)$ for all $t\in [t_k,t_{k+1})$ is calculated as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(\widehat{x}\left(t\right)\right)& =\frac{\partial \widehat{V}\left(\widehat{x}\left(t\right)\right)}{\partial \widehat{x}}{F}_{nn}(\widehat{x}\left(t\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right))\hfill \\ \hfill & =\frac{\partial \widehat{V}\left(\widehat{x}\left(t_k\right)\right)}{\partial \widehat{x}}{F}_{nn}\left(\widehat{x}\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right)\right)\hfill \\ \hfill & +\frac{\partial \widehat{V}\left(\widehat{x}\left(t\right)\right)}{\partial \widehat{x}}{F}_{nn}\left(\widehat{x}\left(t\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right)\right)\hfill \\ \hfill & -\frac{\partial \widehat{V}\left(\widehat{x}\left(t_k\right)\right)}{\partial \widehat{x}}{F}_{nn}\left(\widehat{x}\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right)\right)\hfill \end{array}$$

(47)

By utilizing the inequalities in Equations (34a) and (34b), we further bound $\dot{\widehat{V}}\left(\widehat{x}\left(t\right)\right)$ as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(\widehat{x}\left(t\right)\right)\le & -{\widehat{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+\frac{\partial \widehat{V}\left(\widehat{x}\left(t\right)\right)}{\partial \widehat{x}}{F}_{nn}(\widehat{x}\left(t\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right))\hfill \\ \hfill & -\frac{\partial \widehat{V}\left(\widehat{x}\left(t_k\right)\right)}{\partial \widehat{x}}{F}_{nn}(\widehat{x}\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(\widehat{x}\left(t_k\right)\right))\hfill \end{array}$$

(48)

By applying the Lipschitz inequalities in Equation (35), we proceed with bounding $\dot{\widehat{V}}\left(\widehat{x}\left(t\right)\right)$ as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(\widehat{x}\left(t\right)\right)\le & -{\widehat{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_{nn}|\widehat{x}\left(t\right)-\widehat{x}\left(t_k\right)|\hfill \\ \hfill \le & -{\widehat{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_{nn}{M}_{nn}\Delta \hfill \end{array}$$

(49)

Therefore, if Equation (44a) is satisfied, the following inequality holds for all $\widehat{x}\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$ and $t\in [t_k,t_{k+1})$:

$$\dot{\widehat{V}}\left(x\left(t\right)\right)\le -{ϵ}_s$$

(50)

Upon integrating the aforementioned differential equation over the time interval $t\in [t_k,t_{k+1})$, it is derived that $\widehat{V}\left(\widehat{x}\left(t_{k+1}\right)\right)\le \widehat{V}\left(\widehat{x}\left(t_k\right)\right)-{ϵ}_s\Delta$. Therefore, when Equation (44a) is satisfied, this leads to the fact that $\dot{\widehat{V}}\left(x\left(t\right)\right)$ is negative for any $\widehat{x}\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$. As a result, utilizing the control law $u={\mathsf{\Phi }}_{nn}\left(\widehat{x}\right)$ in a sample-and-hold fashion ensures that the closed-loop state of the RNN model in Equation (33) is bounded within the region ${\Omega }_{\widehat{\rho }}$ and moves toward the origin. Nevertheless, it should be observed that Equation (50) may not hold true in cases where $x\left(t_k\right)=\widehat{x}\left(t_k\right)\in {\Omega }_{{\rho }_s}$, which indicates that the state $\widehat{x}\left(t_k\right)$ may leave the region ${\Omega }_{{\rho }_s}$ within one sampling period. Hence, we introduce the region ${\Omega }_{{\rho }_{nn}}$ in Equation (45a) to guarantee that the closed-loop state $\widehat{x}\left(t_k\right)$ of the RNN model will be bounded in the region ${\Omega }_{{\rho }_{nn}}$ within one sampling period for all $t\in [t_k,t_{k+1})$, $u\in U$ and $\widehat{x}\left(t_k\right)\in {\Omega }_{{\rho }_s}$. Consider the case where $\widehat{x}\left(t_{k+1}\right)$ leaves the region ${\Omega }_{{\rho }_s}$. Then, the controller $u={\mathsf{\Phi }}_{nn}\left(x\left(t_{k+1}\right)\right)$ reactivates to derive the states into the region ${\Omega }_{{\rho }_s}$, and Equation (50) will be satisfied again at $t=t_{k+1}$. Up to this point, it can be concluded that the state of the RNN system in Equation (33) is ultimately bounded in ${\Omega }_{{\rho }_{nn}}$ for all $x_0\in {\Omega }_{\widehat{\rho }}$.

The second part of this proof is to demonstrate that the controller $u={\mathsf{\Phi }}_{nn}\left(x\right)\in U$, implemented in a sample-and-hold fashion, effectively bounds the states of the slow subsystem in Equation (11) in ${\Omega }_{\widehat{\rho }}$ and can ultimately derive the states to a small region around the origin. This requires showing that $\widehat{V}\left(x\right)$ for the slow subsystem in Equation (11) is decreasing under the control law $u\left(t\right)={\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right)$ for $t\in [t_k,t_{k+1})$ and $x\left(t_k\right)=\widehat{x}\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$. The derivative of $\widehat{V}\left(x\left(t\right)\right)$ with respect to time is computed as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(x\left(t\right)\right)& =\frac{\partial \widehat{V}\left(x\left(t\right)\right)}{\partial x}F(x\left(t\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right))\hfill \\ \hfill & =\frac{\partial \widehat{V}\left(x\left(t_k\right)\right)}{\partial x}F(x\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right))\hfill \\ \hfill & +\frac{\partial \widehat{V}\left(x\left(t\right)\right)}{\partial x}F(x\left(t\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right))\hfill \\ \hfill & -\frac{\partial \widehat{V}\left(x\left(t_k\right)\right)}{\partial x}F(x\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right))\hfill \end{array}$$

(51)

Using the inequality in Equation (36), which holds for all $x\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$, the first term in Equation (51) can be further bounded as follows:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(x\left(t\right)\right)\le & -{\tilde{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+\frac{\partial \widehat{V}\left(x\left(t\right)\right)}{\partial x}F(x\left(t\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right),\xi )\hfill \\ \hfill & -\frac{\partial \widehat{V}\left(x\left(t_k\right)\right)}{\partial x}F(x\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right),0)\hfill \end{array}$$

(52)

where ${\tilde{a}}_3(·)$ was previously defined in the proof of Proposition 1. By using the Lipschitz condition stated in Equation (14), we derive the following upper bound for $\dot{\widehat{V}}\left(x\left(t\right)\right)$:

$$\begin{array}{cc}\hfill \dot{\widehat{V}}\left(x\left(t\right)\right)\le & {\tilde{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_x^{{}^{\prime }}|x\left(t\right)-x\left(t_k\right)|\hfill \\ \hfill \le & {\tilde{a}}_3\left({\widehat{a}}_2^{-1}\left({\rho }_s\right)\right)+L_x^{{}^{\prime }}{M}_F\Delta \hfill \end{array}$$

(53)

Therefore, if Equation (44b) holds true, the following inequality is satisfied for all $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_s}$ and for all $t\in [t_k,t_{k+1})$:

$$\dot{\widehat{V}}\left(x\left(t\right)\right)\le -{ϵ}_w$$

(54)

By integrating the above differential equation over the time interval $t\in [t_k,t_{k+1})$ between any two arbitrary points within the previous time interval, it can be shown that for all $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_S}$, the following results are obtained:

$$\widehat{V}\left(x\left(t_{k+1}\right)\right)\le V\left(x\left(t_k\right)\right)-{ϵ}_w\Delta$$

(55)

$$\widehat{V}\left(x\left(t\right)\right)\le \widehat{V}\left(x\left(t_k\right)\right),\forall t\in [t_k,t_{k+1})$$

(56)

Based on Equation (54), it can be observed that $\dot{\widehat{V}}\left(x\left(t\right)\right)$ is negative for all $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_S}$. As a result, the state of the slow subsystem in Equation (11) remains continuously bounded within the region ${\Omega }_{\widehat{\rho }}$ and can be ultimately driven toward the origin in each sampling period by implementing the control law $u={\mathsf{\Phi }}_{nn}\left(x\right)$. In addition, if $x\left(t_k\right)\in {\Omega }_{{\rho }_s}$, the state of the RNN model in Equation (33) is bounded within the region ${\Omega }_{{\rho }_{nn}}$ for one sampling period. This outcome has been demonstrated in the first part of the proof. Taking onto account the bounded modeling error between the state of the RNN, as described in Equation (33), and the state of the slow subsystem in Equation (11), a compact set ${\Omega }_{{\rho }_{min}}\supset {\Omega }_{{\rho }_{nn}}$ exists and satisfies Equation (45b). The set ${\Omega }_{{\rho }_{min}}$ is introduced to ensure that the state of the slow subsystem in Equation (11) is bounded within the region ${\Omega }_{{\rho }_{min}}$ during one sampling period, provided that the state of the RNN represented by Equation (33) is bounded within the region ${\Omega }_{{\rho }_{nn}}$. In the case where the state $x\left(t\right)$ enters the set ${\Omega }_{{\rho }_{min}}\setminus {\Omega }_{{\rho }_S}$, it has been shown that Equation (56) is satisfied. Therefore, under the control law $u={\mathsf{\Phi }}_{nn}\left(x\right)$, the state $x\left(t\right)$ will be driven toward the origin in the next sampling period, ultimately bounding the state of the slow subsystem within a small region around the origin ${\Omega }_{{\rho }_{min}}$. Hence, considering the continuity property of the Lyapunov function $\widehat{V}$, it follows that there exists a class $\mathcal{KL}$ function ${\beta }_x$ and a class $\mathcal{K}$ function $\overline{\gamma }$, such that if $x_0\in {\Omega }_{\widehat{\rho }}$, then $x\left(t\right)\in {\Omega }_{\widehat{\rho }}$ for all $t\le t_0$:

$$\left|x\left(t\right)\right|\le {\beta }_x\left(\right|x\left(0\right)|,t)+\overline{\gamma }\left({\rho }_{min}\right)$$

(57)

□

4.2. Machine Learning-Based LMPC Formulation

In this subsection, we introduce the machine learning-based LMPC formulation that utilizes the RNN model designed in Equation (33) to predict future states over a certain future horizon. Furthermore, the LMPC is essentially an optimization problem consisting of an objective function and a number of constraints. This optimization problem is solved repeatedly to compute the optimal input trajectory. The following shows the formulation of the machine learning-based LMPC optimization problem:

$$\begin{array}{cc}\hfill \mathcal{J}=& \underset{u\in S(\Delta )}{\min }{\int }_{t_k}^{t_{k+N}}{L}_{MPC}(\tilde{x}\left(t\right),u\left(t\right))dt\hfill \end{array}$$

(58a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& \dot{\tilde{x}}\left(t\right)=F_{nn}(\tilde{x}\left(t\right),u\left(t\right))\hfill \end{array}$$

(58b)

$$\begin{array}{cc}\hfill & u\left(t\right)\in U,\forall t\in [t_k,t_{k+N})\hfill \end{array}$$

(58c)

$$\begin{array}{cc}\hfill & \tilde{x}\left(t_k\right)=x\left(t_k\right)\hfill \end{array}$$

(58d)

$$\begin{array}{cc}\hfill & \dot{\widehat{V}}(x\left(t_k\right),u)\le \dot{\widehat{V}}(x\left(t_k\right),{\mathsf{\Phi }}_{nn}\left(x\left(t_k\right)\right),ifx\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_{nn}}\hfill \end{array}$$

(58e)

$$\begin{array}{cc}\hfill & \widehat{V}\left(\tilde{x}\left(t\right)\right)\le {\rho }_{nn},\forall t\in [t_k,t_{k+N}),ifx\left(t_k\right)\in {\Omega }_{{\rho }_{nn}}\hfill \end{array}$$

(58f)

where the predicted slow state trajectory is denoted as $\tilde{x}$. The set of piecewise constant functions with period $\Delta$ is denoted as $S(\Delta )$, and the number of sampling periods in the prediction horizon is denoted by N. The optimal control action $u^{*}\left(t\right)$ is calculated by repeatedly solving the machine learning-based LMPC optimization problem over the entire prediction horizon $t\in [t_k,t_{k+N})$. Then, the optimal control action $u^{*}\left(t_k\right)$ computed at the first sampling period is transmitted by the controller to be applied to the process. Subsequently, the resulting real-time state, $x\left(t_k\right)$, is fed back to the machine learning-based LMPC optimization problem to compute the optimal input trajectory for the next sampling time. The optimization problem aims to minimize the time integral of $L_{MPC}(\tilde{x}\left(t\right),u\left(t\right))$ over the prediction horizon, which is represented in the cost function in Equation (58a). As for the constraints of the optimization problem, the first constraint demonstrated in Equation (58b) is essentially the RNN model utilized to approximate the states of the slow subsystem’s dynamics. Equation (58c) represents the input constraints that may be applied to the process over the entire prediction horizon. The initial condition needed to solve Equation (58b) is essentially the slow state measurement at $t=t_k$, which is specified in the constraint in Equation (58d). In the case where $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_{nn}}$, the constraint Equation (58e) ensures that the closed-loop state converges toward the origin. Furthermore, if the state $x\left(t_k\right)$ enters the region ${\Omega }_{{\rho }_{nn}}$, it is necessary to guarantee that the states predicted by the RNN model remain bounded within the region ${\Omega }_{{\rho }_{nn}}$ throughout the entire prediction horizon. This is accomplished by employing Equation (58f).

4.3. Closed-Loop Stability

Assuming that certain conditions are met, the following theorem establishes the closed-loop stability of a singularly perturbed system described by Equation (1) when the machine learning-based LMPC of Equation (58) is implemented.

Theorem 2. 

Consider the singularly perturbed system in Equation (1) in a closed loop, with the optimal control action $u^{*}$ calculated using the machine learning-based LMPC in Equation (58) based on the controller ${\mathsf{\Phi }}_{nn}\left(x\right)$ that meets the conditions of Equation (34). Additionally, assuming that Propositions 1–3 and Assumptions 1 and 4 hold true, then there exist class $\mathcal{KL}$ functions ${\beta }_x$ and ${\beta }_z$, a pair of positive real numbers $(\delta ,d)$, and ${ϵ}^{*}>0$, such that if $max\left\{\right|x\left(0\right),\left|z\right(0\left)\right|\}\le \delta$ and $ϵ\in (0,{ϵ}^{*}]$, then, for all $t\ge 0$,

$$\left|x\left(t\right)\right|\le {\beta }_x\left(\left|x\right(0\left)\right|,t\right)+\overline{\gamma }\left({\rho }_{min}\right)+d$$

(59)

$$\left|z\left(t\right)\right|\le {\beta }_z\left(\left|z\left(0\right)\right|,\frac{t}{ϵ}\right)+d$$

(60)

Proof. 

We follow the proof analysis in [5]. Moreover, we consider that by substituting the optimal control action $u^{*}$ into Equation (1), the closed-loop system can be represented as follows:

$$\begin{array}{cc}\hfill \dot{x}& =f_1(x,z,u^{*},ϵ)\hfill \end{array}$$

(61a)

$$\begin{array}{cc}\hfill ϵ\dot{z}& =f_2(x,z,ϵ)\hfill \end{array}$$

(61b)

We set $ϵ=0$ and obtain the following,

$$\begin{array}{cc}\hfill \dot{x}& =f_1(x,z,u^{*},0)\hfill \end{array}$$

(62a)

$$\begin{array}{cc}\hfill 0& =f_2(x,z,0)\hfill \end{array}$$

(62b)

where the assumption that the error between z and $\overline{z}$ is not greater than $\mathcal{O}\left(ϵ\right)$ enables us to simplify the analysis by approximating $\overline{z}$ as z. By solving Equation (62b) for z, we obtain a unique, isolated root $z=\overline{f}\left(x\right)$. We substitute the root into Equation (62a) and obtain the following for the reduced-order slow subsystem,

$$\dot{x}=f_1(x,{\overline{f}}_2\left(x\right),u^{*},0)$$

(63)

By observing the LMPC equation in Equation (58), in the case where $x\left(t_k\right)\in {\Omega }_{\widehat{\rho }}\setminus {\Omega }_{{\rho }_{nn}}$, Equation (58e) is activated, such that the Lyapunov function $\widehat{V}$ under the calculated control law u is decreased. According to the finding in Proposition 3 and considering a sufficiently small modeling error, the state of the slow subsystem in Equation (61a) will gradually converge to the origin and enter the region ${\Omega }_{{\rho }_{nn}}$ within a finite number of sampling time steps. Equation (58f) is activated once the state $x\left(t_k\right)$ enters the region ${\Omega }_{{\rho }_{nn}}$, where it ensures that the predicted state is bounded within ${\Omega }_{{\rho }_{nn}}$ throughout the prediction horizon. Additionally, by ensuring that the modeling error and the sampling time are sufficiently small, Proposition 3 establishes that the actual state in Equation (61a) can be maintained within a slightly expanded set ${\Omega }_{{\rho }_{min}}$, encompassing ${\Omega }_{{\rho }_{nn}}$. Hence, LMPC guarantees that for any initial state $x_0\in {\Omega }_{\widehat{\rho }}$, the state $x\left(t\right)$ of the closed-loop slow subsystem described by Equation (61a) is maintained bounded within the region ${\Omega }_{\widehat{\rho }}$ at all times and fulfills the bound of Equation (57) that was established in Proposition 3.

Additionally, by introducing a fast time scale $\tau =t/ϵ$, a new coordinate $\overline{\overline{z}}=z-{\overline{f}}_2\left(x\right)$, and considering $ϵ=0$, the closed-loop fast subsystem is given by:

$$\frac{d\overline{\overline{z}}}{d\tau }=f_2(x,\overline{\overline{z}}+{\overline{f}}_2\left(x\right),0)$$

(64)

According to Assumption 4, global asymptotic stability is achieved for the origin of the closed-loop fast subsystem in Equation (64), satisfying Equation (12) for any $\overline{\overline{z}}\left(0\right)\in {\mathbb{R}}^p$. As a result, the closed-loop system in Equation (61) meets all the assumptions required for Theorem 1 in [19] to hold true. Hence, there exist class $\mathcal{KL}$ functions ${\beta }_x$ and ${\beta }_z$, a pair of positive real numbers $(\delta ,d)$, and ${ϵ}^{*}>0$, such that if $max\left\{\right|x\left(0\right),\left|z\right(0\left)\right|\}\le \delta$ and $ϵ\in (0,{ϵ}^{*}]$, the closed-loop states of the slow and fast systems are bounded by Equations (59) and (60). □

Remark 3. 

It is possible to define the general class of two-time-scale systems with an additional input associated with the fast subsystem as follows:

$$\begin{array}{cc}\hfill \dot{x}& =f_1(x,z,u,ϵ)\hfill \end{array}$$

(65a)

$$\begin{array}{cc}\hfill ϵ\dot{z}& =f_2(x,z,u_2,ϵ)\hfill \end{array}$$

(65b)

where $u_2\in {\mathbb{R}}^{q_2}$ is the bounded manipulated input vector associated with the fast subsystem. The input vector $u_2$ is constrained by $u_2\in U_2:=\left\{\right|u_i|\le u_{2_{max,i}},i=1,\dots ,q_2$}. However, if there exists a stabilizing control law $u_2=h_{stable}(x,z)$ that stabilizes the dynamics of the fast subsystem, then the general class of two-time-scale systems defined by Equation (65) can be reduced to the class of systems presented in Equation (1), which is the primary focus of this work. Furthermore, there are relevant works that have addressed the stability analysis of classes of systems defined in Equation (65), which share similar concepts but incorporate certain modifications. Interested readers may refer to [3] to further explore this subject.

5. Example of Application to a Chemical Process

In this section, we apply the proposed reduced-order machine learning strategy to a chemical process example. Specifically, we build two ML models—a reduced-order ML model to approximate the slow subsystem, and another ML model to capture the dynamics of the full two-time-scale system. Subsequently, each ML model is incorporated into an LMPC scheme, and the performance of the controllers is compared in terms of the closed-loop properties and computational efficiencies.

We consider a perfectly mixed, non-isothermal CSTR, in which an irreversible and endothermic reaction that transforms chemical A to product B$(A\to B)$ occurs. Figure 3 depicts the CSTR, where $C_A$ represents the concentration of chemical A, and $T_r$ represents the temperature of the reactor’s contents. $C_{A0}$ denotes the inlet molar concentration of chemical A, which is fed into the reactor at flow rate F and temperature $T_{A0}$. Assuming that the vessel maintains a constant holdup, $V_r$ denotes the volume of liquid present in the reactor. Due to the occurrence of an endothermic reaction within the reactor, a heating jacket with volume $V_j$ is used to provide the energy as required. $T_{j0}$ is the inlet temperature of the heat-transfer fluid provided to the jacket with a flow rate $F_j$. The reactor’s contents and the heat-transfer fluid maintain constant densities, denoted as ${\rho }_m$ and ${\rho }_j$, respectively. In addition, they both have constant heat capacities, denoted as $c_{p,m}$ and $c_{p,j}$, respectively. The enthalpy of the reaction is $\Delta H_r$, the heat-transfer coefficient is U, and the heat-transfer contact area between the reactor and the jacket is $A_r$. Given that $k_0$ is the pre-exponential constant, R is the ideal gas constant, and E is the activation energy of the reaction, the time-varying rate constant of the reaction is denoted by k and is given by the following equation:

$$k=k_{0}exp\left(\frac{-E}{R{T}_r}\right)$$

(66)

The first-principles equations of the CSTR are described by the following dynamic material and energy balance equations:

$$\begin{array}{cc}\hfill V_r\frac{\mathrm{d}{C}_A}{\mathrm{d}t}& =F_r(C_{A0}-C_A)-k_{0}exp\left(\frac{-E}{R{T}_r}\right)C_A{V}_r\hfill \end{array}$$

(67a)

$$\begin{array}{cc}\hfill V_r\frac{\mathrm{d}{T}_r}{\mathrm{d}t}& =F_r(T_{A0}-T_r)+\frac{-\Delta H_r}{{\rho }_m{C}_{p,m}}{k}_{0}exp\left(\frac{-E}{R{T}_r}\right)C_A{V}_r+\frac{U{A}_r}{{\rho }_m{C}_{p,m}}(T_j-T_r)\hfill \end{array}$$

(67b)

$$\begin{array}{cc}\hfill V_j\frac{\mathrm{d}{T}_j}{\mathrm{d}t}& =F_j(T_{j0}-T_j)-\frac{U{A}_r}{{\rho }_j{C}_{p,j}}(T_j-T_r)\hfill \end{array}$$

(67c)

where the values of the process parameters are listed in

Table 1. Notations and parameter values of the CSTR.

$V_r=1.0{\mathrm{m}}^3$	$E=8.0\times {10}^3\mathrm{kcal}{\mathrm{kg}}^{-1}$
$V_j=0.08{\mathrm{m}}^3$	$U=1000.0\mathrm{kcal}{\mathrm{h}}^{-1}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$
$A_r=6.0{\mathrm{m}}^3$	$k_0=3.36\times {10}^6{\mathrm{h}}^{-1}$
$C_{A0s}=3.75\mathrm{kmol}{\mathrm{m}}^{-3}$	${\rho }_m=900.0\mathrm{kg}{\mathrm{m}}^{-3}$
$C_{As}=2.54\mathrm{kmol}{\mathrm{m}}^{-3}$	${\rho }_j=800.0\mathrm{kg}{\mathrm{m}}^{-3}$
$R=1.987\mathrm{kcal}{\mathrm{kmol}}^{-1}{\mathrm{K}}^{-1}$	$c_{p,m}=0.231\mathrm{kcal}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$F_r=3.0{\mathrm{m}}^3{\mathrm{h}}^{-1}$	$c_{p,j}=0.200\mathrm{kcal}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$F_j=20.0{\mathrm{m}}^3{\mathrm{h}}^{-1}$	$\Delta H_r=5.4\times {10}^4\mathrm{kcal}{\mathrm{mol}}^{-1}$
$T_{rs}=274.4\mathrm{K}$	$T_{j0}=357.5\mathrm{K}$
$T_{js}=303.3\mathrm{K}$	$T_{A0}=310.0\mathrm{K}$

. Additionally, we consider the constant $ϵ=\frac{V_j}{V_r}$ to be the singular perturbation parameter. Hence, the system of Equation (67) can be written in the standard singularly perturbed form of Equation (1), which implies that the concentration of chemical A ($C_A$) and the temperature of the reactor ($T_r$) are the slow states, whereas the temperature of the heating jacket ($T_j$) can be considered as the fast state. The input of the system in Equation (67) is the feed concentration of chemical A ($C_{A0}$). The control objective is to drive the system to its stable steady state $(C_{As},T_{rs},T_{js})=(2.54\mathrm{kmol}{\mathrm{m}}^{-3},274.4\mathrm{K},303.3\mathrm{K})$, corresponding to the steady-state input value of $C_{A0s}=3.75\mathrm{kmol}{\mathrm{m}}^{-3}$, while the input is permitted to vary within the range $\Delta C_{A0}=[-3.5,3.5]\mathrm{kmol}{\mathrm{m}}^{-3}$. To shift the steady state of the system in Equation (67) to the origin, we express the states and the input in terms of deviation variables and rewrite the system in Equation (67). Specifically, the deviation variables are defined as follows:

$$\begin{array}{cc}\hfill \Delta C_A& =C_A-C_{As}\hfill \end{array}$$

(68a)

$$\begin{array}{cc}\hfill \Delta T_r& =T_r-T_{rs}\hfill \end{array}$$

(68b)

$$\begin{array}{cc}\hfill \Delta T_j& =T_j-T_{js}\hfill \end{array}$$

(68c)

$$\begin{array}{cc}\hfill \Delta C_{A0}& =C_{A0}-C_{A0s}\hfill \end{array}$$

(68d)

Therefore, to write the CSTR system of Equation (67) in the standard form of Equation (1), we define $x^T=[\Delta C_A\Delta T_r]$, $z=\Delta T_j$, and the input $u=\Delta C_{A0}$. The explicit Euler method is utilized to simulate the CSTR system, i.e., Equation (67) is numerically integrated with a sufficiently small time step $h_c={10}^{-4}$ h. The open-source software package IPOPT [20] is employed to solve the ML-based LMPC optimization problem in Equation (58), with a sampling time of $\Delta =0.03$ h.

5.1. Data Generation and Development of RNN Models

The first step in building the RNN models was to generate a comprehensive data set that captured the dynamics of the system in Equation (67). To implement the explicit Euler algorithm, we first needed to initialize the process model using a set of initial conditions $(C_A\left(0\right),T_r\left(0\right),T_j\left(0\right))$, where $C_A\left(0\right)\in [0,9]{\mathrm{mol}/\mathrm{m}}^3$, $T_r\left(0\right)\in [280,370]$ K and $T_j\left(0\right)\in [300,390]$ K. We conducted open-loop simulations by integrating the CSTR system defined by Equation (67) using a sufficiently small time step $h_c$ for ${10}^6$ random initial conditions within the specified ranges. We applied random input signals $C_{A0}\in [0.5,7.5]{\mathrm{mol}/\mathrm{m}}^3$ over a period of one sampling period $\Delta$, which yielded a data set of ${10}^6$ trajectories, each with a length equal to $\Delta$. The data set was then divided into three sets—training, validation, and testing. The training of the models was performed using the open-source library Keras [21]. The first RNN model was designed to predict the dynamics of the full two-time-scale system in Equation (67) (i.e, it predicts the dynamics of the slow and fast states). In other words, it utilizes the current state measurements $x\left(t_k\right)$ and $z\left(t_k\right)$, and the manipulated input for the subsequent sampling period $u\left(t\right)\in [t_k,t_{k+1})$ to predict the slow and fast state measurements of $x\left(t\right)$ and $z\left(t\right)\forall t\in [t_k,t_{k+1}]$. For simplicity, we denote the model that predicts the dynamics of the full two-time-scale system in Equation (67) as $RN{N}_F$, where the subscript “F” denotes “full” and not “fast”. This model was designed using a single layer of 20 long short-term memory recurrent neural network (LSTM-RNN) units, and the validation and testing errors achieved were $1.434\times {10}^{-6}$ and $1.432\times {10}^{-6}$, respectively. In addition, the total number of learning parameters for training this model was 2063. We note that the network was designed in a step-wise manner. Initially, we started with a simple structure, consisting of a single-layer network with one LSTM-RNN unit but the validation error was high. Gradually, we increased the network’s complexity by adding more LSTM-RNN units. This progression was continued until we achieved a satisfactory level of validation loss. In this particular case, the optimal outcome was achieved using a single-layer network with 20 LSTM-RNN units. Further increasing the number of LSTM-RNN units beyond 20 resulted in an increase in the validation loss. Therefore, $RN{N}_F$ was constructed using 20 LSTM-RNN units. On the other hand, the second RNN, denoted as $RN{N}_S$, was designed to predict the dynamics of only the slow states in Equation (67). Specifically, this RNN utilizes the current slow state measurement $x\left(t_k\right)$ and the manipulated input for the subsequent sampling period $u\left(t\right)\in [t_k,t_{k+1})$ to predict the slow state measurements of $x\left(t\right)\forall t\in [t_k,t_{k+1}]$. This model was designed using a single layer of five LSTM-RNN units corresponding to a total of 192 learning parameters following the same step-wise manner strategy used to design $RN{N}_F$, with a validation error of $2.67\times {10}^{-4}$ and a testing error of $2.68\times {10}^{-4}$. The results of both models are summarized in

Table 2. Specifications of the constructed recurrent neural network models.

RNN Model	${\mathit{RNN}}_{\mathit{F}}$	${\mathit{RNN}}_{\mathit{S}}$
Number of units used	20	5
Testing error	$1.432\times {10}^{-6}$	$2.68\times {10}^{-4}$
Validation error	$1.434\times {10}^{-6}$	$2.67\times {10}^{-4}$
Number of learning parameters	2063	192

. The low testing errors for both models indicate that the modeling error, as defined in Proposition 1, is sufficiently small over a wide range of open-loop trajectories. Due to having fewer learning parameters, the $RN{N}_S$ model is significantly less complex compared to the $RN{N}_F$ model, possibly leading to a lower computational cost and making it a more practical choice for control applications. Therefore, our aim is to investigate whether the LMPC scheme utilizing $RN{N}_S$ can sufficiently meet the desired control objective more efficiently compared to the LMPC scheme using $RN{N}_F$, which is demonstrated via closed-loop simulations of the CSTR in Equation (67) under both LMPC designs. However, before proceeding, it is essential to define the Lyapunov and objective functions used in both LMPC schemes. For the $RN{N}_F$-based LMPC, since the controller design was based on the dynamics of the full two-time-scale system in Equation (67), the Lyapunov function is defined as:

$$V_F(x,z)={\left[\begin{array}{c}x-x_s\\ z-z_s\end{array}\right]}^T{P}_F\left[\begin{array}{c}x-x_s\\ z-z_s\end{array}\right]$$

(69)

where the matrix $P_F$ is given by

$$P_F=\left[\begin{array}{ccc}91.6& 2.9& 4.3\\ 2.9& 0.8& 0.2\\ 4.3& 0.2& 0.7\end{array}\right]$$

The objective function for the $RN{N}_F$-based LMPC is given by $L_F(x,z,u)={\left|x\right|}_{Q_{1F}}^2+{\left|z\right|}_{Q_{2F}}^2+{\left|u\right|}_{Q_{3F}}^2$, where $Q_{1F}=\left[\begin{array}{cc}20& 0\\ 0& 0.05\end{array}\right]$, $Q_{2F}=6$, and $Q_{3F}=0.1$. On the other hand, the $RN{N}_S$-based LMPC was designed based on the dynamics of the slow subsystem of the CSTR, specifically Equations (67a) and (67b) expressed in the standard form. Therefore, the Lyapunov function of the slow subsystem used in the $RN{N}_S$-based LMPC is given by:

$$V\left(x\right)={(x-x_s)}^{T}P(x-x_s)$$

(70)

where the matrix P is defined as

$$P=\left[\begin{array}{cc}84.9& 1.2\\ 1.2& 0.4\end{array}\right]$$

The objective function is given by $L(x,u)={\left|x\right|}_{Q_1}^2+{\left|u\right|}_{Q_2}^2$, where $Q_1=\left[\begin{array}{cc}20& 0\\ 0& 0.05\end{array}\right]$ and $Q_2=0.05$.

Remark 4. 

In this particular application, we used long short-term memory recurrent neural network (LSTM-RNN) units to construct the models. LSTM-RNNs are a special type of RNN known for their ability to overcome the common problem of vanishing/exploding gradients in RNNs, resulting in generally better performance. For more information about LSTM-RNNs, interested readers may refer to [22]. However, we can always utilize classical RNNs that are well-suited for this particular application by choosing appropriate structures and carefully tuning the hyperparameters to obtain the desired objective, as well as an acceptable level of performance.

5.2. Simulation Results

In this subsection, we conduct closed-loop simulations of the CSTR system described by Equation (67) under the two LMPC schemes, one with each RNN model. We first assess the ability of both LMPCs to achieve the control objective adequately compared to LMPCs that utilize the respective first-principles systems as their process models. Subsequently, we compare the RNN-based LMPCs to each other in terms of closed-loop performance and the computational time required for the simulations.

To establish the satisfactory performance of both RNN-based LMPCs, we compared their closed-loop performance to first-principles-based LMPCs, with the latter serving as the baseline for the best possible performance achievable under LMPC. Specifically, the LMPC in Equation (58) was considered in two scenarios, with the process model of Equation (58e) being different between each scenario:

• Scenario 1: The LMPC utilizing $RN{N}_F$ as the process model was compared to an LMPC employing the first-principles model in Equation (67), denoted as $F{P}_F$, as its process model.
• Scenario 2: The LMPC utilizing $RN{N}_S$ as the process model was compared to an LMPC employing the first-principles slow subsystem in Equations (67a) and (67b), denoted as $F{P}_S$, as its process model. In this case, for the first-principles-based LMPC, we note that the full CSTR system in Equation (67) was integrated, but only the slow states $C_A$ and $T_r$ from Equations (67a) and (67b) were used in calculating the LMPC cost function of Equation (58a) and the Lyapunov function V.

For both scenarios, we started the simulations from the initial condition $I{C}_{\mathrm{main}}=(\Delta C_A\left(0\right),$$\Delta T_r\left(0\right),\Delta T_j\left(0\right))=(1,30,40)$. The LMPC prediction horizon was set to $N=3$, and the remaining controller parameters were described in Section 5.1. The state and input trajectories under the LMPCs of scenarios 1 and 2 are shown in Figure 4 and Figure 5, respectively. To quantitatively compare the performance of the different LMPCs, the time integral of the cost function of the LMPC, ${\int }_{t=0}^{t_f}L(x\left(\tau \right),u\left(\tau \right))\mathrm{d}\tau$, was calculated over the entire simulation duration, $t_f=3$ h. For scenario 1, the cost function values were 3458 and 3485 for the $F{P}_F$-based LMPC and the $RN{N}_F$-based LMPC, respectively, whereas for scenario 2, the values of the cost function were 176 and 179 for the $F{P}_S$-based LMPC and the $RN{N}_S$-based LMPC, respectively. For both scenarios, we noticed that the value of the cost function when utilizing the corresponding RNN model closely aligned with that achieved when employing the respective first-principles model, demonstrating the reliable predictive performance of the designed RNN models and their ability to drive the system to the steady state when incorporated into an LMPC scheme.

Next, we demonstrated the computational efficiency of the $RN{N}_S$-based LMPC due to its lower complexity while still achieving the desired controller performance. For $I{C}_{\mathrm{main}}$, the computational time for the $RN{N}_F$-based LMPC shown in Figure 4 was 5578 s, whereas, for the $RN{N}_S$-based LMPC shown in Figure 5, it was significantly lower at 2170 s, which was $61\%$ lower. This substantial difference in computational time can make $RN{N}_S$ a more practical choice for real-time application of MPC, where computations must be completed within a sampling period to be sent to the actuator. Hence, to rigorously investigate the impact of integrating $RN{N}_F$ and $RN{N}_S$ into an LMPC framework in terms of stability and computational demand, we conducted closed-loop simulations from several different initial conditions in addition to $I{C}_{\mathrm{main}}$.

The initial conditions chosen for further investigation are outlined in

Table 3. Computational times for the $RN{N}_F$-based LMPC and the $RN{N}_S$-based LMPC over the simulation duration of $t_f=3$ h starting from various initial conditions within the operating region.

Index	Initial Condition	Computational Time (s)
	$\mathbf{(}\Delta {\mathit{C}}_{\mathit{A}}\mathbf{\left(}\mathbf{0}\mathbf{\right)}\mathbf{,}\mathbf{\Delta }{\mathit{T}}_{\mathit{r}}\mathbf{\left(}\mathbf{0}\mathbf{\right)}\mathbf{,}\mathbf{\Delta }{\mathit{T}}_{\mathit{j}}\mathbf{\left(}\mathbf{0}\mathbf{\right)}\mathbf{)}$	${\mathit{RNN}}_{\mathit{F}}$-Based LMPC	${\mathit{RNN}}_{\mathit{S}}$-Based LMPC
$I{C}_{main}$	(1, 30, 40)	5578	2170
$I{C}_1$	(−1, 50, 40)	16,059	1807
$I{C}_2$	(−1, −10, −3)	4801	2667
$I{C}_3$	(−3, 30, 5)	31,884	2417
$I{C}_4$	(−2, −10, 100)	17,896	1921
$I{C}_5$	(1, 90, 10)	6078	1990
$I{C}_6$	(1, 20, 60)	5795	2404
$I{C}_7$	(3, −6, 20)	21,231	2411
$I{C}_8$	(1, 50, 50)	5161	2225
$I{C}_9$	(−2, 30, 60)	10,275	2194
$I{C}_{10}$	(−1, 10, 80)	18,146	2106

. Ten different initial conditions ($I{C}_i$ where $i=1,\dots ,10$) within the operating region were selected, along with $I{C}_{\mathrm{main}}$. For each initial condition, we simulated the CSTR system described in Equation (67) under the $RN{N}_F$-based LMPC, as well as the $RN{N}_S$-based LMPC. The state and input profiles under each LMPC for four representative initial conditions are depicted in Figure 6, Figure 7, Figure 8 and Figure 9, which indicate that both LMPC schemes successfully drove the process in Equation (67) to its steady state. The closed-loop behavior was similar for the remaining six initial conditions in terms of convergence to the steady state. Additionally, Table 3 provides the computational times required in terms of CPU time for the entire simulation duration of $t_f=3$ h when applying either LMPC scheme for the 11 different initial conditions. For all the tested initial conditions, the computational time required to execute the $RN{N}_S$-based LMPC was less than that required for the $RN{N}_F$-based LMPC. Among all the initial conditions in Table 3, the largest relative difference in computational time was observed for $I{C}_3$, with a percentage difference of $92\%$ between the two LMPC schemes. Hence, the incorporation of $RN{N}_S$ in the LMPC framework has been demonstrated to be more practical and computationally efficient compared to employing $RN{N}_F$, without compromising stability and closed-loop performance.

The computational time of an MPC optimization problem can be influenced by several factors. As mentioned in [23], MPC computational times are generally affected by the horizon length N, the number of constraints in the MPC optimization problem, and the number of states and control actions of the system. The computational time is also directly related to the complexity of the process model. Hence, given the lower computational times across all 11 initial conditions tested, incorporating $RN{N}_S$ into an LMPC framework is a more practical approach compared to the use of $RN{N}_F$. In the study of [24], it was observed that the CPU time of the IPOPT optimization problem increased as the complexity of the problem increased. Additionally, several works have discussed computational complexity evaluations of neural networks. For instance, the authors of [25] offered a comprehensive and systematic analysis for quantifying and comparing the computational complexity of neural network layers. The authors proposed and computed three computational metrics per layer for different types of neural networks and presented the exact mathematical expressions and derivations, providing a deep understanding of various networks’ complexities. Upon analyzing the results derived in [25], it is evident that a neural network’s complexity is proportional to its hyperparameters. For example, factors like the number of neurons, the number of hidden units, the input time sequence size, the number of output neurons, and other parameters associated with the structure, size, and design of the network, all of which affect the complexity and performance of a network, as well as the time required to solve the ML-based MPC optimization problem. As $RN{N}_S$ consisted of only 5 LSTM-RNN units while $RN{N}_F$ consisted of 20 LSTM-RNN units, the complexity of $RN{N}_F$ was much higher than that of $RN{N}_S$, leading to a correspondingly complex MPC optimization problem. Hence, over the entire simulation duration of $t_f=3$ h, the total CPU time required to solve all 100 MPC optimization problems over the 100 sampling periods was significantly less for the $RN{N}_S$-based LMPC than the $RN{N}_F$-based LMPC.

Remark 5. 

Although the state and input trajectories in Figure 6 have not completely settled, the process is stabilized under both the $RN{N}_F$-based LMPC and the $RN{N}_S$-based LMPC. However, the process requires simulation for a longer duration, exceeding 3 h, for the states and input to ultimately converge to their steady-state values and remain close to the origin. The reason for limiting the simulation time is to ensure fair and clear comparisons between the computational times for both controllers from various initial conditions, as listed in Table 3.

Remark 6. 

In this particular application, $RN{N}_S$ was designed to utilize the current slow state measurement and the manipulated input for the next sampling period. As a result, the output of $RN{N}_S$ will be the predicted slow states x for only one sampling period ahead ($1\Delta$). Consequently, in this application, since the prediction horizon was chosen to be an integer multiple of the sampling time Δ (i.e., $N=3$ implies that $t_{k+N}=t_k+3\Delta$ in Equation (58a)), based on the design of $RN{N}_S$, $RN{N}_S$ is required to be invoked at least three times to be able to make predictions for the entire prediction horizon. Additionally, an alternative method involves designing an RNN model, denoted as $RN{N}_{new}$, with the same goal of predicting the slow states. However, $RN{N}_{new}$ utilizes not only the current slow state measurements and the manipulated input for the next sampling period but also the current fast state measurements to predict the slow states one sampling period Δ ahead, i.e., $x\left(t_{k+1}\right)$. To obtain predictions for the entire prediction horizon ($N=3$), an FNN takes as its input the output of $RN{N}_{new}$ and predicts the fast state z at the next sampling period, i.e., the FNN predicts $z\left(t_{k+1}\right)$ from $x\left(t_{k+1}\right)$, following the design described in Section 2.5. Subsequently, for the next sampling period, the outputs of both $RN{N}_{new}$ and the FNN are given as inputs to $RN{N}_{new}$, enabling it to predict the slow states $x\left(t_{k+2}\right)$ for the next sampling period in the prediction horizon.

6. Conclusions

In this work, we introduced a general class of nonlinear two-time-scale systems, where the two distinct time scales can be decoupled into slow subsystem and fast subsystem dynamics using singular perturbation analysis. Machine learning was employed to approximate the dynamics of both subsystems. In particular, an RNN was used to predict the slow state vector or, in other words, to approximate the dynamics of the slow subsystem, whereas an FNN was used to approximate the dynamics of the fast subsystem. To draw inferences about the performance of the neural network models on unseen data, the generalization error bounds for both neural networks when modeling two-time-scale systems were derived. Subsequently, the RNN modeling of the slow states ($RN{N}_S$) was incorporated into an LMPC framework and sufficient conditions to guarantee closed-loop stability were derived under the sample-and-hold implementation of the LMPC. Finally, a chemical process example was used to demonstrate the efficiency of the LMPC scheme using an RNN to predict the slow states compared to first-principles-based LMPC schemes and an LMPC scheme using an RNN to model the full two-time-scale system. Through closed-loop simulations, while both RNN-based LMPCs were able to achieve the desired control objective of driving the system to the steady state, due to the significantly lower complexity of the RNN approximating only the slow subsystem, the LMPC based on $RN{N}_S$ required significantly less computational time in all simulations compared to the LMPC using the RNN modeling the full singularly perturbed system, which supported the use of $RN{N}_S$ in real-time MPC applications.