Data-Driven Process Monitoring and Fault Diagnosis: A Comprehensive Survey

Abstract

This paper presents a comprehensive review of the historical development, the current state of the art, and prospects of data-driven approaches for industrial process monitoring. The subject covers a vast and diverse range of works, which are compiled and critically evaluated based on the different perspectives they provide. Data-driven modeling techniques are surveyed and categorized into two main groups: multivariate statistics and machine learning. Representative models, namely principal component analysis, partial least squares and artificial neural networks, are detailed in a didactic manner. Topics not typically covered by other reviews, such as process data exploration and treatment, software and benchmarks availability, and real-world industrial implementations, are thoroughly analyzed. Finally, future research perspectives are discussed, covering aspects related to system performance, the significance and usefulness of the approaches, and the development environment. This work aims to be a reference for practitioners and researchers navigating the extensive literature on data-driven industrial process monitoring.

1. Introduction

The ongoing advancements in society demand continuous improvements in safety, efficiency, and sustainability within process industries. To tackle these challenges, companies must have a comprehensive understanding of process behaviors, enabling them to quickly detect, identify and fix malfunctions. The rise in digitalization has provided access to vast amounts of data collected from various sensors, creating an opportunity for informed decision making. Research on data-driven process monitoring aims to develop methods that enable real-time use of process data in the most effective manner, with the goal of ensuring healthy performance.

Achieving this objective involves various aspects, such as mathematical modeling, data processing, benchmarking methodologies, software development, and practical implementation in industrial settings. The literature on data-driven process monitoring has seen significant growth in recent years, with numerous studies addressing these aspects. In particular, many review articles and overviews have been published, typically focusing on specific topics. For instance, Yu and Zhang [1] discuss the challenges of deep learning modeling; Kong [2] explores the application of latent variable models; Zhao [3] examines non-stationary process monitoring; Peres and Fogliatto [4] review strategies for variable selection; and Melo et al. [5] analyze open benchmarks for comparing methodologies.

Despite the wealth of existing works, to the best of our knowledge, the literature lacks a comprehensive survey encompassing at once the following gaps: (i) compiling and critically analyzing previous reviews related to the topic; (ii) presenting the most popular mathematical models in a didactic manner, ensuring the inclusion of most significant models; (iii) conducting a thorough examination of often overlooked aspects, such as practical implementation and software development; and (iv) covering the historical progression, the current state of the art, and future perspectives of the field.

The primary objective of the present paper is to provide a survey of the research field of process monitoring, specifically addressing the four aforementioned points. By addressing these points in a single document, this review seeks to offer a holistic understanding of the field and its fundamental aspects. The next sections of the article are organized as follows. Section 2 provides a broad overview of the historical development of process monitoring. Detailed discussions on mathematical modeling techniques are presented, specifically multivariate statistical models (Section 3) and machine learning approaches (Section 4). Section 5 focuses on the exploration, characterization, and treatment of process data. Section 6 reviews computational tools and openly available benchmarks in the literature. Section 7 addresses the practical implementation of process monitoring in industrial settings. Section 8 explores future perspectives and challenges in the field. Section 9 concludes the article. Additionally, Appendixes Appendix A and Appendix B include a compilation of relevant resources such as review papers, overviews, comparative studies, books, and commercial tools related to process monitoring in the field of process systems engineering (PSE). These resources can serve as references for further exploration of specific topics.

2. Historical Remarks

Process monitoring is a term widely used in both daily application and engineering practice, encompassing various aspects such as the supervision of production by operators. However, as industries have evolved, specific methodologies have been developed to systematize this procedure.

Shewhart [6,7], during his tenure at Bell Telephone Laboratories, introduced the concept of control charts. These graphical tools, rooted in statistical principles, enable the monitoring of significant process variables to detect events associated with special cause variations, distinct from common cause variations that are inherent and expected during process operation. The development of control charts is widely recognized as a pivotal moment in the establishment of statistical process monitoring [8].

Another significant breakthrough occurred with the introduction of the $T^2$ statistic by Hotelling [9]. This statistical tool allowed for the simultaneous analysis of multiple variables and their interactions, marking the inception of the field of multivariate statistical process monitoring [10]. This pioneering work enabled a comprehensive approach to process monitoring by considering the interconnectedness among various variables.

Despite Hotelling’s work in 1947, the widespread adoption of multivariate monitoring applications occurred primarily in the late 1980s. This can be attributed to two key factors: (i) the increased accessibility and utilization of computers and (ii) the proposal of applying latent variable modeling, a particular class of multivariate statistical methodologies, to process monitoring problems [11]. These methodologies have become the prevailing approach from the 1990s onwards, marking a paradigm shift that has been extensively documented in numerous review and overview articles from that era [12,13,14].

Since the 2000s, the rapid advancement of computational capacity has led to the growing prominence of machine learning approaches for multivariate process monitoring. The ability of machine learning algorithms to process and analyze vast amounts of data has contributed to their increasing relevance in the field of multivariate monitoring, enabling the identification of complex relationships and patterns that were previously challenging to detect [15,16].

In recent years, there has been a growing tendency to classify multivariate statistical techniques as machine learning algorithms [16,17]. However, despite this trend, in the PSE literature, they are conventionally categorized separately [10,15,18,19]. This separation arises from the historical development of multivariate statistical models in fields such as statistics and chemometrics, along with the association of machine learning methodologies with the broader field of computer science. In this study, we present the two approaches in dedicated sections. Section 3 provides an overview of the main techniques developed within the framework of multivariate statistics, while Section 4 focuses on the techniques derived from the machine learning literature. By separating these discussions, we aim to offer a clear and comprehensive examination of each approach within the context of its historical development. However, it is important to keep in mind that nuances and gray areas may emerge: while multivariate techniques can be interpreted as learning algorithms, many machine learning methodologies inherently possess a statistical foundation.

3. Multivariate Statistics

In multivariate statistics, multiple variables and their interactions are analyzed simultaneously. In this context, latent variable modeling stands out as the most popular multivariate statistical approach for applications in process monitoring [16]. This approach involves transforming the raw data’s original variables into a new set of variables that explicitly reveal the underlying, hidden (latent) characteristics present in the data. The choice of technique depends on the specific characteristics one wishes to uncover. Some commonly used techniques are principal component analysis (PCA), partial least squares (PLS) and canonical correlation analysis (CCA) [20,21,22,23]. These methods are widely applied in industrial process monitoring applications because they generate orthogonal latent variables that eliminate the linear correlations between the original variables, a common characteristic in industrial data [24].

Latent variable modeling has gained such historical significance that several works which aimed to present the literature on data-driven process monitoring solely focused on this methodology [14,18,20,25,26,27,28]. These works may suggest two implicit viewpoints: either the entire process monitoring field is exclusively constituted by these methods, or these are the only methods worthy of attention.

Only MacGregor and Cinar [26] provided a justification for this preference:

“In this paper, we focus on the proper use of data-driven (empirical) models for the monitoring, control and optimization of processes. In particular, we focus on latent variable models because, as we will show, they provide the proper structure to allow them to be built from plant data and be used for monitoring, control and optimization”.

[26]

Throughout their article, the authors justified their choice by arguing that latent variable modeling methods are the only ones that can model both the predictor ($\mathbf{X}$) and predicted ($\mathbf{Y}$) portions of a dataset. According to the authors, other regression techniques would solely model the $\mathbf{Y}$ space, which would be equivalent to assuming that the $\mathbf{X}$ space has full rank, a hypothesis rarely met in real-world scenarios. As a consequence, latent variable methods would be the most suitable for handling missing data issues and the only data-driven monitoring methodology that could provide unique, interpretable, and causal models.

Several objections can be made to this viewpoint. The shift to the latent variable space is not the only way to address collinearity in the $\mathbf{X}$ matrix. The nonlinearity of some regression methods can itself mitigate or eliminate this issue, and even in the context of linear methods, other strategies exist, including subset selection and penalization methods such as ridge regression and Lasso [29]. Data reconciliation techniques can also be employed, as they allow for the treatment of fluctuations and uncertainties in both $\mathbf{X}$ and $\mathbf{Y}$ [30,31,32,33]. The aim of these objections is not to diminish the importance of latent variable models but rather to highlight that their prevalence in the process monitoring literature may be partially attributed to historical biases. Recognizing this fact can help foster openness to new opportunities and better understand the shift in discourse towards buzzwords such as machine learning and artificial intelligence, as discussed in Section 4.

In the remaining part of this section, a detailed account of the main techniques used in multivariate statistics is presented from a historical and didactic perspective. The theoretical descriptions emphasize the assumptions that each technique makes regarding the nature of the data. While these assumptions are essential in characterizing the use of the techniques, they are often overlooked by practitioners.

3.1. PCA

3.1.1. Fundamentals

Principal component analysis (PCA) was originally introduced by Pearson [34]; however, the application in the field of process monitoring was proposed much later by Wise et al. [11]. PCA is a technique that produces a set of orthogonal linear combinations of the original variables, with the goal of selecting a subset of these combinations that effectively captures and summarizes the variability of the data. The first assumption of the technique refers to the approach used for deriving the latent variables:

PCA, Assumption 1.

The latent variables can be obtained by performing a linear transformation on the dataset.

Let $\mathbf{X}\in {\mathbb{R}}^{n\times m}$ be a dataset, where each row corresponds to an observation and each column corresponds to a measured variable. The linear transformation performed by PCA can be represented as a change in basis:

$$\mathbf{T}=\mathbf{XP},$$

(1)

where the columns of $\mathbf{P}\in {\mathbb{R}}^{m\times m}$, denoted by {${\mathbf{p}}_{\mathbf{1}}$, …, ${\mathbf{p}}_{\mathbf{m}}$}, are referred to as principal components. These components represent a new basis for expressing each observation (i.e., row) of the matrix $\mathbf{X}$. On the other hand, the columns of $\mathbf{T}\in {\mathbb{R}}^{n\times m}$ correspond to the latent variables and constitute projections of the columns of $\mathbf{X}$ onto the basis {${\mathbf{p}}_1$, …, ${\mathbf{p}}_{\mathbf{m}}$}.

To select the new basis $\mathbf{P}$, it is necessary to assess which characteristics of the transformed matrix $\mathbf{T}$ provide the optimal representation of the data. However, this evaluation can only be performed by making additional assumptions about the nature of the data:

PCA, Assumption 2.

There is a degree of linear dependence between variables, which hinders the description of the essence of the data due to the presence of redundant and unnecessary information.

PCA, Assumption 3.

The data exhibit a high signal-to-noise ratio, indicating that larger variances carry greater relative importance of information.

Hypothesis PCA-A2 indicates that it is desirable for the columns of $\mathbf{T}$ to be linearly independent. In other words, the empirical covariance matrix ${\mathbf{C}}_{\mathbf{T}}$:

$${\mathbf{C}}_{\mathbf{T}}=\frac{{(\mathbf{T}-\overline{\mathbf{T}})}^{\prime }(\mathbf{T}-\overline{\mathbf{T}})}{n-1},$$

(2)

should only contain diagonal elements. In Equation (2), $\overline{\mathbf{T}}$ denotes the vector of column-wise means of $\mathbf{T}$. Before proceeding with the mathematical development, it is necessary to introduce one additional hypothesis:

PCA, Assumption 4.

Each variable in the data matrix has a zero mean, i.e., $\overline{\mathbf{X}}=\overline{\mathbf{T}}=0$.

In practice, the validity of this assumption is ensured through a process called centralization, where the mean of each variable is subtracted. In this case, the covariance matrix of $\mathbf{T}$ can be expressed as:

$${\mathbf{C}}_{\mathbf{T}}=\frac{{\mathbf{T}}^{\prime }\mathbf{T}}{n-1}.$$

(3)

Using Equation (1), Equation (3) can be rewritten in terms of the principal component matrix $\mathbf{P}$ as follows:

$${\mathbf{C}}_{\mathbf{T}}={\mathbf{P}}^{\prime }{\mathbf{C}}_{\mathbf{X}}\mathbf{P}.$$

(4)

According to the Spectral Theorem [35], a symmetric matrix such as ${\mathbf{C}}_{\mathbf{X}}$ can be decomposed into eigenvalues and eigenvectors as follows:

$${\mathbf{C}}_{\mathbf{X}}=\mathbf{E}\mathsf{\Lambda }{\mathbf{E}}^{\prime },$$

(5)

where $\mathbf{E}$ is the matrix containing the eigenvectors of ${\mathbf{C}}_{\mathbf{X}}$ as columns, and $\mathsf{\Lambda }$ is a diagonal matrix containing the eigenvalues of ${\mathbf{C}}_{\mathbf{X}}$ in descending order of magnitude.

To obtain a diagonal matrix ${\mathbf{C}}_{\mathbf{T}}$, the matrix $\mathbf{P}$ can be conveniently chosen as $\mathbf{P}=\mathbf{E}$. This is because substituting Equation (5) into Equation (4) yields:

$${\mathbf{C}}_{\mathbf{T}}=\mathsf{\Lambda }.$$

(6)

If Assumptions PCA-A2 and A3 are satisfied, selecting $\mathbf{P}=\mathbf{E}$ results in a transformed matrix $\mathbf{T}$ that is advantageous for data analysis in two ways. Firstly, it eliminates the redundancy caused by the linear dependence between the variables. Secondly, it organizes the variables in decreasing order of importance according to the variance values present in the diagonal matrix ${\mathbf{C}}_{\mathbf{T}}$.

It is useful to interpret the PCA technique from an optimization perspective. In this context, the first principal component ${\mathbf{p}}_1$ should maximize the variance of the latent variable ${\mathbf{t}}_1={\mathbf{Xp}}_1$:

$$\begin{array}{cc}\hfill {\mathbf{p}}_1& =\underset{\parallel \mathbf{p}{\parallel }_2=1}{\arg \max }\left\{{\left(\mathbf{Xp}\right)}^{\prime }\mathbf{Xp}\right\}\hfill \\ \hfill & =\underset{\parallel \mathbf{p}{\parallel }_2=1}{\arg \max }\left\{{\mathbf{p}}^{\prime }{\mathbf{X}}^{\prime }\mathbf{Xp}\right\}\hfill \\ \hfill & =\underset{\parallel \mathbf{p}{\parallel }_2=1}{\arg \max }\left\{{\mathbf{p}}^{\prime }{\mathbf{C}}_{\mathbf{X}}\mathbf{p}\right\}\hfill \\ \hfill & =\arg \max \left\{\frac{{\mathbf{p}}^{\prime }{\mathbf{C}}_{\mathbf{X}}\mathbf{p}}{{\mathbf{p}}^{\prime }\mathbf{p}}\right\}.\hfill \end{array}$$

(7)

In the last equality, the unit norm constraint is relaxed by dividing the objective function by $\parallel \mathbf{p}{\parallel }_2^2$. The remaining principal components can be found in the successive orthogonal spaces by deflating the matrix ${\mathbf{C}}_{\mathbf{x}}$ with respect to $\mathbf{p}$ [36].

The objective function represented by Equation (7) takes a mathematical form known as the Rayleigh quotient:

$$r(\mathbf{A},\mathbf{B},\mathbf{w})=\frac{{\mathbf{w}}^{\prime }\mathbf{A}\mathbf{w}}{{\mathbf{w}}^{\prime }\mathbf{B}\mathbf{w}}.$$

(8)

It can be shown that the critical points of $r(\mathbf{A},\mathbf{B},\mathbf{w})$ correspond to the solution of the generalized eigenvalue problem [22,37]:

$$\mathbf{Ae}=\lambda \mathbf{Be}.$$

(9)

Since $\mathbf{B}=\mathbf{I}$ in the objective function, Equation (9) simplifies to an eigenvalue problem, which is consistent with the derivation of Equation (6). It can be demonstrated that the eigenvalue decomposition of the covariance matrix ${\mathbf{C}}_{\mathbf{x}}$ is equivalent to the singular value decomposition of the data matrix $\mathbf{X}$ [38]. In fact, the latter approach is computationally more efficient and stable, as it eliminates the need to calculate ${\mathbf{C}}_{\mathbf{x}}$.

3.1.2. Dimensionality Reduction

In order to perform dimensionality reduction, a certain number a of principal components is selected such that they account for a certain percentage of the explained variance (usually 90%), resulting in a reduced matrix ${\mathbf{P}}_{\mathbf{a}}\in {\mathbb{R}}^{m\times a}$. The reduced matrix of latent variables ${\mathbf{T}}_{\mathbf{a}}\in {\mathbb{R}}^{n\times a}$ is obtained through the equation:

$${\mathbf{T}}_{\mathbf{a}}={\mathbf{XP}}_{\mathbf{a}}.$$

(10)

Based on Assumption PCA-A3, the most informative variables are those that exhibit more variability in the multivariate space under study. Therefore, the matrix ${\mathbf{T}}_{\mathbf{a}}$ would be a more effective and suitable representation of the data, as it excludes the variables associated with smaller variances (which, according to the assumption, correspond to measurement noise). This matrix can be projected onto the original variables as follows:

$$\widehat{\mathbf{X}}={\mathbf{T}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime }=\mathbf{X}\left({\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime }\right).$$

(11)

By doing so, the original variables can be reconstructed in the matrix $\widehat{\mathbf{X}}$, which includes only the primary variation in the dataset, corresponding to the a leading principal components. In this sense, the PCA technique can be viewed as a filter, as the transformation from $\mathbf{X}$ to $\widehat{\mathbf{X}}$, in theory, reduces the noise level of the dataset.

The difference between $\mathbf{X}$ and $\widehat{\mathbf{X}}$ is represented by the residual matrix $\mathbf{E}$:

$$\mathbf{E}=\mathbf{X}-\widehat{\mathbf{X}}=\mathbf{X}(\mathbf{I}-{\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime }).$$

(12)

The subspaces generated by $\widehat{\mathbf{X}}$ and $\mathbf{E}$ are known as principal space and residual space, respectively, [20].

Several discussions regarding selection methods for the parameter a can be found in the literature [39,40,41,42,43,44]. In particular, Saccenti and Camacho [43] conducted a comparison of different techniques, including statistical-based methods, cross-validation methods and methods based on numerical approximation. The authors offered recommendations for selecting the most suitable methods based on the specific characteristics of the analyzed data.

Figure 1 illustrates the application of the PCA procedure to a bidimensional dataset. In the first plot, the red arrows depict the two principal components, with the larger arrow indicating the first principal component, associated with the direction of higher explained variance. The second plot demonstrates the linear transformation from $\mathbf{X}$ to $\mathbf{T}$. The third plot demonstrates the dimensionality reduction step, where the bidimensional data is transformed into a unidimensional representation. Finally, the last plot presents the reconstructed data in the original bidimensional space, now with variation only along the direction of the principal component.

3.1.3. Process Monitoring

In the context of process monitoring, the data matrix $\mathbf{X}$ is often derived from a time series. The aim of applying the PCA technique in this scenario is to identify the structure of the principal subspace during a normal operating period and use it as a reference to evaluate new observations in order to determine whether the structure is maintained.

When the process that generates the time series exhibits dynamic behavior, i.e., temporal dependence, the PCA technique may not be able to accurately identify the structure of the principal subspace. In such cases, the latent variables may be autocorrelated and potentially exhibit cross-correlation [45]. Hence, to utilize PCA for process monitoring purposes, the following assumption needs to be considered:

PCA, Assumption 5.

The data matrix is obtained from a steady-state process.

The monitoring procedure utilizing PCA can be carried out in two distinct stages. The first stage, referred to as training, involves applying PCA to a dataset collected from normal operation, which is commonly known as the training set. The outcome of the training process includes two matrices: the projection matrix ${\mathbf{P}}_{\mathbf{a}}$ and the explained variances matrix ${\mathsf{\Lambda }}_{\mathbf{a}}$. In the second stage, known as testing, new observations $\mathbf{x}$ are subjected to monitoring using statistical indices. The most frequently employed indices for monitoring are the $T^2$ and Q indices, which are described below.

In the context of PCA, the $T^2$ statistic is computed by applying the confidence ellipse of the normal distribution to the latent variable ${\mathbf{t}}_{\mathbf{a}}$, which is the projection of the observation vector $\mathbf{x}$ onto the first a principal components [27,46]:

$$T^2={\mathbf{t}}_{\mathbf{a}}^{\prime }{{\mathsf{\Lambda }}_{\mathbf{a}}}^{-1}{\mathbf{t}}_{\mathbf{a}}={\mathbf{x}}^{\prime }{\mathbf{P}}_{\mathbf{a}}{{\mathsf{\Lambda }}_{\mathbf{a}}}^{-1}{\mathbf{P}}_{\mathbf{a}}^{\prime }·\mathbf{x}$$

(13)

The $T^2$ statistic provides a global measure of the variability captured by the PCA model in the first a principal components, which is associated with the systematic variations in the process being monitored [20]. During the monitoring phase, an increase in the $T^2$ value for new observations indicates the occurrence of disturbances with a similar nature to the variations identified by the PCA model, that is, disturbances in the directions of the plane described by the model [47].

The Q statistic is defined as the Euclidean distance between an observation vector $\mathbf{x}$ and its projection onto the principal subspace $\widehat{\mathbf{x}}=\left({\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime }\right)\mathbf{x}$:

$$Q=\left|\right|\mathbf{x}-\widehat{\mathbf{x}}{\left|\right|}^2=\left|\right|(\mathbf{I}-{\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime })\mathbf{x}{\left|\right|}^2={(\mathbf{x}-\widehat{\mathbf{x}})}^{\prime }(\mathbf{x}-\widehat{\mathbf{x}})={\mathbf{x}}^{\prime }(\mathbf{I}-{\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime })\mathbf{x}.$$

(14)

The Q statistic is the squared norm of the prediction errors (residuals) of the PCA model and is often referred to as SPE (square prediction error). It provides a global measure of variability that is not captured by the PCA model, i.e., variability present in the residual space, which is associated with unusual or unexplained variations in the process being monitored [20]. During monitoring, an increase in the Q value for new observations suggests a change in the relationships between the variables, such as a violation in correlations, resulting in a break in the structure captured by the model during training. In other words, the plane described by the PCA model does not include the direction of the disturbance [47]. Due to the form of Equation (14), establishing a detection limit using the Q statistic is equivalent to delimiting a hypersphere as a confidence region in the residual space [48].

The PCA procedure is illustrated in the present paper through the application of the Tennessee Eastman Process (TEP) dataset. This dataset is widely used in the process monitoring literature and originates from a chemical process simulation developed by Eastman Chemical Company [49]. The TEP comprises five units: reactor, condenser, vapor–liquid separator, stripper, and recycle centrifugal compressor. It involves four reactants and one inert being fed into a reactor, which generates two products and one byproduct through exothermic and irreversible reactions. For a comprehensive critical analysis of the TEP dataset, readers are referred to Melo et al. [5].

Figure 2 displays the application of the PCA training procedure to the TEP dataset. The first plot shows the growth in accumulated explained variance as the number of components increases. The second plot presents the control chart for the Q index when normal data are used.

The use of the $T^2$ statistic, which is derived from the normal distribution, requires the formulation of an additional hypothesis:

PCA, Assumption 6.

The dataset$\mathbf{X}$follows a normal distribution.

In some cases, the $T^2$ statistic can be used even when Assumption PCA-A6 is violated, since the latent variables are linear combinations of the original variables. In this case, one may assume that the latent variables approximately follow a normal distribution due to the Central Limit Theorem [50]. This theorem establishes that the sum of random variables asymptotically approaches a normal distribution as the number of variables increases.

Some references consider normality as an intrinsic requirement for applying the PCA technique and not as a requirement for the use of the $T^2$ statistic, as considered in the present text. This happens because these references apply and interpret PCA in the context of statistical (in)dependence, while in the present text, the discussion involved the less rigorous concept of linear (in)dependence (see Assumption PCA-A2). To ensure that linear independence between the latent variables is equivalent to statistical independence, some hypotheses about the probability distribution of the data must be enunciated. It can be shown that assuming that the data follows a normal distribution is a sufficient condition to guarantee this equivalence [51].

This issue is subtle and often a source of conceptual confusion. For instance, Jonathon Shlens included the normality assumption in the early versions of his excellent tutorial on PCA [52] but removed it in later versions [38]. To support the understanding of this topic, it is worth consulting a concise and elucidative comparison between the terms linear independence, orthogonality, and uncorrelatedness presented by Rodgers et al. [53].

The Q and $T^2$ statistics are applicable to the two primary activities conventionally associated with process monitoring, namely fault detection and diagnosis.

3.1.4. Fault Detection

To carry out fault detection, control limits must be established for the statistics $T^2$ and Q. Considering a confidence level $\alpha$, the most commonly used limits are [47,54]:

$$T_{\alpha }^2=\frac{a(n^2-1)}{n(n-a)}{F}_{\alpha (a,n-a)},$$

(15)

$$Q_{\alpha }={\theta }_1{\left[\frac{h_0{c}_{\alpha }\sqrt{2{\theta }_2}}{{\theta }_1}+1+\frac{{\theta }_2{h}_0(h_0-1)}{{\theta }_1^2}\right]}^{1/h_0},$$

(16)

where:

• $F_{\alpha (a,n-a)}$ is the upper limit of the $\alpha$ percentile of the Fisher-Snedecor F distribution, with a and $n-a$ degrees of freedom;
• ${\theta }_i={\sum }_{j=a+1}^m{\lambda }_j^i$;
• $h_0=1-\left({\displaystyle \frac{2{\theta }_1{\theta }_3}{3{\theta }_2^2}}\right)$;
• $c_{\alpha }=1.645$ is the z-score corresponding to the $(1-\alpha )$ percentile of the standard normal distribution.

Equation (15) is derived directly from statistical theory, while Equation (16) is based on the approximation of the distribution of quadratic forms and was first proposed by Jackson and Mudholkar [55]. When the underlying normality assumptions are not met, Equations (15) and (16) may not perform well, and alternative control limits based on specific statistical properties of each dataset need to be developed.

Figure 3 presents the Q control chart for one of the faults in the Tennessee Eastman Process dataset. The fault event is indicated by the vertical line at sample 160. Following the occurrence of the fault, the data points exceed the horizontal dashed line, which represents the $Q_{\alpha }$ control limit. This triggers an alarm, indicating a deviation from the expected behavior of the process.

3.1.5. Fault Identification and Diagnosis

There exists a certain degree of confusion in the process monitoring literature regarding the precise definitions of the terms identification and diagnosis. For instance, Chiang et al. [20] defined identification as the act of indicating the most relevant variables for performing diagnosis, which was in turn defined as the determination of the root cause of the abnormal state. In contrast, Qin [18] proposed a different definition, in which the meanings of the terms are reversed: identification was defined as the process of identifying a fault from a set of possible known faults, while diagnosis referred to the identification of the variables that contributed to the occurrence of the fault. The latter definition is adopted in the present text.

There are several diagnostic methods in the context of PCA, among which the most popular are the contribution methods [20,50,56,57,58]. These methods are based on analyzing the contribution of each variable to the detection index. In order to apply these methods, an additional hypothesis must be assumed:

PCA, Assumption 7.

At the time of fault, the magnitude of the contribution of a variable to the increase in the detection rate is proportional to the influence of the variable on the occurrence of the abnormal event.

The contribution methods used in PCA can be broadly classified into three categories [56]:

• Decomposition contributions: These methods decompose the detection index into the sum of contributions from individual variables.
• Reconstruction-based contributions: These methods consider the contribution of a variable to be the magnitude of the reconstruction of the detection index along the direction of that variable.
• Diagonal contributions: In multiblock problems, similar variables are grouped into blocks based on domain knowledge, which simplifies the scenarios with many variables. Diagonal contributions keep only the block terms in the detection index.

The formulation of the most widespread technique, the method of decomposition contributions, will be presented as an illustration. The indices $T^2$ and Q can be written in the form of a generalized index, $idx$:

$$idx={\mathbf{x}}^{\prime }\mathbf{Mx},$$

(17)

where:

• $\mathbf{M}={\mathbf{P}}_{\mathbf{a}}{{\mathsf{\Lambda }}_{\mathbf{a}}}^{-1}{\mathbf{P}}_{\mathbf{a}}^{\prime }$ for $idx=T^2$;
• $\mathbf{M}=\mathbf{I}-{\mathbf{P}}_{\mathbf{a}}{\mathbf{P}}_{\mathbf{a}}^{\prime }$ for $idx=Q$.

The decomposition contribution of variable i to the generalized index is denoted as $c{d}_i$ and expressed as [56]:

$$c{d}_i={\mathbf{x}}^{\prime }{\mathbf{M}}^{1-\beta }{\mathbf{I}}_{\mathbf{i}}{\mathbf{I}}_i^{\prime }{\mathbf{M}}^{\beta }\mathbf{x},$$

(18)

where ${\mathbf{I}}_i$ constitutes the ith column of the identity matrix and $0\le \beta \le 1$. According to the choice of the $\beta$ parameter, different degrees of decomposition can be defined. In the literature, $\beta =0$ or $\beta =1$ are used under the name “partial contribution” and $\beta =1/2$ is used under the name “complete contribution”.

Figure 4 displays a partial contribution heatmap for a fault in the Tennessee Eastman Process dataset. This heatmap provides insights into the variables that contribute to the detection index at each moment in time, enabling the diagnosis of the variables that are associated with the evolution of the fault.

Despite their widespread use, the contribution methods mentioned above may suffer from a problem known in the literature as the smearing-out effect. This occurs when contributions from variables affected by the fault increase contributions from unaffected variables [59]. A detailed analysis of this effect was reported by Kerkhof et al. [60], who concluded that it is more pronounced along the principal components. The authors recommended applying contribution methods only in cases where there is a clear physical interpretation of the principal components. If such an interpretation is not available, they suggest using the deviation of each variable in relation to the expected value as an alternative for diagnosis (this procedure is equivalent to performing univariate diagnosis). In this context, a methodology to address the smearing-out effect was recently proposed by Alakent [61], which involves the application of a non-linear dimensionality reduction method.

Figure 5 illustrates the flowchart of the complete process monitoring pipeline described in this section, including the following steps: data collection, model training, threshold setting, fault detection, fault diagnosis, fault identification, and process recovery.

3.1.6. Evolution and Recent Trends

Among the multivariate monitoring techniques, PCA has received the most attention in both simulated and real systems. Nevertheless, in its fundamental form, as outlined in earlier sections, it may not be adequate for a multitude of significant industrial applications. As a result, there have been several suggestions for enhancing its efficacy, which can be summarized as follows.

• Batch processes: Since the early 1990s, when PCA was first applied, adaptations have been made to utilize the model for batch processes through the implementation of multi-way PCA [62,63]. This technique transforms the data matrix of dimension $n\times m\times b$ (where the third dimension b refers to multiple batches) into a two-dimensional matrix through dimension reduction, which can be achieved either by time or by batch [18]. To compare the effectiveness of monitoring batch processes using multi-way PCA with other strategies such as parallel factor analysis and trilinear decomposition, it is recommended to refer to comparison articles [64,65,66], as well as review articles [67,68]. A promising technique that has not been discussed in these reviews is 2D-PCA [69,70], which enables the separate monitoring of the dynamics associated with variables and batches.
• Dynamics: When the process being analyzed does not meet the assumption of being in a steady state, exhibiting dynamic characteristics such as temporal dependence and autocorrelation, the confidence regions from the PCA monitoring statistics become too large, resulting in high rates of undetected faults [18]. To address this problem, the data matrix can be expanded by incorporating the lagged values of the original variables [45]. However, this approach increases the number of parameters and principal components, which contradicts the dimensionality reduction benefit of PCA, lacks an explicit representation of the relationship between latent and original variables, and fails to establish a connection between latent variables and the dynamic content of the data [71]. To mitigate these drawbacks, recent methods have been proposed to explicitly create dynamic principal components [71,72,73]. For example, DiPCA (dynamic-inner PCA) [71,74] separates the latent variables into two groups: a set of principal time series and a residual with little or no autocovariance. The latter group can be treated as a static dataset by traditional methods, such as PCA. DiPCA was recently applied to assemble a sequential structure for extracting and modeling dynamic and non-linear characteristics of data [75] and to solve the time series segmentation problem [76], which involves generating subsequences based on changes in the dynamics. Adaptive algorithms are another strategy to deal with dynamics by automatically adjusting to changes in mean, variance, and/or correlation structures in the data [18]. This strategy is especially useful for processes with drifts or changes in operating points. Examples of such algorithms include RPCA (recursive PCA), MW-PCA (moving window PCA), and EWMA-PCA (exponentially weighted moving average PCA) [46,77,78,79,80]. A review of recent advances in monitoring dynamic processes was presented by Aldrich [81].
• Nonlinearity: There are two main strategies for incorporating nonlinearity into PCA. The first involves the use of neural networks, as proposed by several researchers: (i) Kramer [82], who developed an approach based on associative networks; (ii) Dong and McAvoy [83], who combined neural networks using the principal curves algorithm [84]; (iii) Jia et al. [85], who applied a network called IT-net [86] to process monitoring; and (iv) Liu et al. [87], who combined networks with the SVDD (support vector data description) technique, a machine learning method described in Section 4. The second strategy is the use of the kernel-PCA (KPCA) technique, developed by Scholkopf et al. [88], applied in process monitoring by Lee et al. [89] and Choi et al. [90] and further developed in several works [91,92,93,94,95,96,97,98]. A kernel is a function that maps data points into a higher-dimensional space, where nonlinear representations can be captured using linear relationships [99]. In addition to these two main strategies, there are other approaches for adapting PCA to nonlinearity, such as the use of the genetic algorithm [100]. Recent reviews on applications of the kernel technique in process monitoring have been published [99,101,102].
• Multimodality: To address multimodality, clustering algorithms can be utilized to distinguish between operation modes [103]. One straightforward approach involves identifying clusters based on prior knowledge about the process or specialized algorithms. Subsequently, individual PCA models are constructed for each cluster [18]. Another effective approach is the application of Gaussian mixture models, as proposed in several works [104,105,106,107,108], which enable the automatic identification of clusters during the PCA training process. A comprehensive review of multimodal process monitoring has been presented by Quiñones-Grueiro et al. [103].
• Multiple time scales: Bakshi et al. [109] conducted the pioneering work of analyzing process data across different time scales, utilizing the methodology proposed by Rioul and Vetterli [110], which involves the application of wavelets for data processing. Kosanovich and Piovoso [111,112] used this approach to pre-process data before applying PCA for monitoring. Bakshi [113] integrated PCA and wavelets into a single method, called MSPCA (multiscale PCA), which works by generating a PCA model for each of the time scales of interest and then joining the models to create a single multiscale model. Since this work, several applications have been reported using or adapting this approach, involving: extension to the diagnostic problem [114,115], theoretical analysis [116], generation of contribution plots [117], analysis of data with multiresolution characteristics [118], and combination with the kernel technique for modeling nonlinearity [119]. Recent works include the application in multi-unit and batch processes [120], the integration with the generalized likelihood ratio test [121] and the use in conjunction with kernel techniques to deal with non-linearities [122]. Extensive reviews on the application of multiscale methods in process data analysis have been published [123,124].
• Multiple units and/or stages: To address the issue of multiple units and/or stages in a decentralized manner, multiblock methods such as CPCA (consensus PCA) and HPCA (hierarchical PCA) have gained widespread use [18]. These techniques involve dividing the data matrix into several blocks simultaneously to obtain local and global information. An important contribution was the demonstration of the algorithmic equivalence between regular and multiblock methods [125]. Recently, fully data-driven approaches for partitioning units/stages have gained attention [126]. Notable examples of these approaches include: the proposal of constructing sub-blocks along the principal components [127]; the combination of spectral clustering based on mutual information, Bayesian inference, and KPCA to handle non-linearity [128]; the combination of KPCA with SVDD to determine the independent variables of the process [126]; and the development of a hybrid method for variable categorization [129]. Comprehensive reviews on plant-wide/distributed monitoring have been published [130,131].
• Error handling: When errors are not independent and identically distributed, incorporating information on the probability density of variables can enhance the modeling of the residual space and improve effectiveness [132,133]. Probabilistic PCA [134], Bayesian PCA [135], and MLPCA (maximum likelihood PCA) [90,136,137,138,139] are some of the approaches used to address this issue. It is noteworthy that MLPCA [132] or principal component regression [140] can handle heteroskedastic data (i.e., data with variance dependent on time or operating conditions). An interesting proposal in this regard is the integration of PCA methodology with data reconciliation [141]. Ge [142] provided a tutorial review on the application of probabilistic modeling on latent variables for process data analysis.
• Poor data quality: To address problems related to poor data quality, such as missing or spurious observations, a class of techniques called robust PCA has been developed. Pioneering work by Walczak and Massart [143] and Xie et al. [144] laid the foundation for this field, which was further developed with applications and extensions [145,146,147,148,149,150,151]. It is interesting to note that robust PCA is widely used in computer vision, image processing, and video monitoring applications [152]. Severson et al. [153] published a review of PCA applications to process data with missing observations. Zhu et al. [154] provided an informative discussion comparing PCA to other robust methods from a big data perspective.

3.2. PLS/CCA

3.3. Fundamentals

PLS (partial least squares) and CCA (canonical correlation analysis) are two techniques with similar objectives, which are to generate two sets of linear combinations of the original variables (one for each category of variables, predictor and predicted) with the goal of selecting the combinations that best summarize the linear relationship between the two categories [22,36,37]. CCA was introduced by Hotelling [155], while PLS was first proposed by Wold [156]. Although PLS is widely used in chemometrics, which may explain its prevalence in process monitoring research, the preference for PLS over CCA in chemometrics has been recently challenged [157]. It is worth noting that PLS is also popular in other fields, such as the biopharmaceutical sector, where it is the most commonly used data-driven model [158].

The following assumptions about the nature of the data are shared by both techniques:

PLS/CCA, Assumption 1.

The latent variables can be obtained by performing a linear transformation on the dataset.

PLS/CCA, Assumption 2.

The dataset can be pre-classified into two distinct categories, namely the predictor set $\mathbf{X}$ and the predicted set $\mathbf{Y}$.

PLS/CCA, Assumption 3.

Within each set $\mathbf{X}$ and $\mathbf{Y}$, there is a degree of linear dependence among the variables, which hinders the description of the essence of the data due to the presence of redundant and unnecessary information.

PLS/CCA, Assumption 4.

There is a latent linear relationship between the set $\mathbf{X}$ and the set $\mathbf{Y}$, which is of interest to be made explicit.

The difference between PLS and CCA lies in the way of obtaining the latent linear relationship mentioned in PLS/CCA-A4: while PLS seeks the directions that maximize the covariance between $\mathbf{X}$ and $\mathbf{Y}$, CCA seeks the directions that maximize the correlation between $\mathbf{X}$ and $\mathbf{Y}$ [22].

PLS, Assumption 5.

The latent linear relationship stated in Assumption 4 is better represented by the directions that maximize the covariance between $\mathbf{X}$ and $\mathbf{Y}$.

CCA, Assumption 5.

The latent linear relationship stated in Assumption 4 is better represented by the directions that maximize the correlation between $\mathbf{X}$ and $\mathbf{Y}$.

Finally, another shared assumption is that it is necessary for the mathematical development of both techniques:

PLS/CCA, Assumption 6.

Each variable in the data matrices has a zero mean, i.e., $\overline{\mathbf{X}}=\overline{\mathbf{Y}}=0$.

Just like PCA, PLS and CCA problems can be formulated as the maximization of the Rayleigh quotient.

For PLS:

$$\begin{array}{cc}\hfill {\mathbf{p}}_{1,\mathbf{x}},{\mathbf{p}}_{1,\mathbf{y}}& =\underset{\parallel \mathbf{p}{\parallel }_2=1}{\arg \max }\left\{\frac{{\mathbf{p}}_{\mathbf{x}}^{\mathbf{T}}{\mathbf{C}}_{\mathbf{xy}}{\mathbf{p}}_{\mathbf{y}}}{\sqrt{{\mathbf{p}}_{\mathbf{x}}^{\mathbf{T}}{\mathbf{p}}_{\mathbf{x}}}\sqrt{{\mathbf{p}}_{\mathbf{y}}^{\mathbf{T}}{\mathbf{p}}_{\mathbf{y}}}}\right\}.\hfill \end{array}$$

(19)

For CCA:

$$\begin{array}{cc}\hfill {\mathbf{p}}_{1,\mathbf{x}},{\mathbf{p}}_{1,\mathbf{y}}& =\underset{\parallel \mathbf{p}{\parallel }_2=1}{\arg \max }\left\{\frac{{\mathbf{p}}_{\mathbf{x}}^{\mathbf{T}}{\mathbf{C}}_{\mathbf{xy}}{\mathbf{p}}_{\mathbf{y}}}{\sqrt{{\mathbf{p}}_{\mathbf{x}}^{\mathbf{T}}{\mathbf{C}}_{\mathbf{x}}{\mathbf{p}}_{\mathbf{x}}}\sqrt{{\mathbf{p}}_{\mathbf{y}}^{\mathbf{T}}{\mathbf{C}}_{\mathbf{y}}{\mathbf{p}}_{\mathbf{y}}}}\right\}.\hfill \end{array}$$

(20)

This allows the problems to be solved using the framework of generalized eigenvalue problems, as presented in Equation (9) [37]. The monitoring statistics $T^2$ and Q are formulated similarly to those of PCA [20,131].

There is currently no consensus on the appropriate situations to use partial least squares (PLS) or canonical correlation analysis (CCA). Some studies suggest using CCA to model input–output relationships and PLS to monitor quality variables, which are typically more challenging to measure than process variables, resulting in issues such as delays [131,159,160]. Zhang et al. [160] also noted that CCA is typically applied at a local, process-specific level, whereas PCA and PLS are applied at a global level, encompassing the entire plant. In the field of computer science, Taylor and Cristianini [36] argued that PLS is appropriate for establishing relationships between input and output representations, while CCA is suitable for finding relationships between different representations of the same dataset. Another approach for comparing PLS and CCA involves investigating their mathematical relationship. Barker and Rayens [161], for instance, view PLS as a penalized CCA, where the penalties are represented by a PCA model in the space $\mathbf{X}$ and a PCA model in the space $\mathbf{Y}$.

Confusion in the literature exists regarding the nomenclature of these techniques. For example, the acronym PLS can have two meanings: the original name, partial least squares, or a reinterpretation, projection to latent structures [162]. Moreover, there are many variants of PLS [163], and it is often unclear which variant is being referred to when using the acronym. The term canonical correlation analysis (CCA) is used synonymously with canonical variate analysis (CVA) in some references [20,164]. However, CVA is better understood and accepted in the literature as the application of CCA methodology for identifying subspaces to find a linear combination of past inputs and outputs that is more predictive of future outputs [165,166]. Therefore, CVA has a more restricted meaning since nothing in the CCA technique mathematically imposes its application in this specific context. The confusion increases when analyzing literature from other domains. For example, in a book on ecology, Gittins [167] defines CVA as an application of CCA in which variables with arbitrary values are added to the dataset, to which the analyst assigns meanings artificially to include some data characteristic in the analysis (such as information on classes). This definition of the term is entirely unrelated to what is discussed in the fields of chemometrics and process monitoring.

The history and trends of PLS/CCA techniques, especially PLS, follow the same lines as the development of PCA. Works have been developed for applications concerning batch processes [168,169,170,171,172,173,174], dynamics [169,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192], nonlinearity [119,185,193,194,195,196,197,198], multimodality [199,200,201,202], multiple time scales [119,203,204,205], multiple units and/or stages [125,170,193,206,207,208,209,210,211,212,213], error handling [214,215,216,217] and poor data quality [216,218,219,220,221,222]. In addition, new topics have emerged, such as the study of the effect of dataset size on model structure [223]. For a comprehensive review of the CCA technique, the reader is referred to Yang et al. [224].

3.4. Other Techniques

As previously mentioned, PCA and PLS/CCA are multivariate statistical techniques widely used in the field of process monitoring. However, it is important to note that there are other alternative methodologies available, which are worth considering. Some of these methodologies include:

• Fisher discriminant analysis (FDA): Introduced by Fisher [225], this technique assumes the a priori division of variables into two categories and aims to generate an orthogonal set of linear combinations of the original variables that effectively separate these categories. FDA is particularly valuable when data is available for both normal operation and multiple fault cases [18]. Various examples of FDA applications in monitoring problems have been proposed [226,227,228,229,230,231,232,233,234,235,236,237,238,239,240]. For more detailed information on this technique, please refer to Chiang et al. [20].
• Independent component analysis (ICA): Introduced by Jutten and Herault [241], ICA generates a set of linear combinations of the original variables to maximize the independence statistic between these combinations. ICA is considered an alternative to PCA when the data distribution is not assumed to be normal [242]. The application of ICA in process monitoring is relatively recent, with the initial works proposed in the early 2000s [243,244,245,246,247,248]. Subsequent advancements have been proposed in the recent literature [249,250,251,252,253,254,255,256,257,258,259]. Notably, Zhang et al. [260] proposed an interesting method that automatically selects between the exclusive or mixed application of PCA and ICA techniques, with or without using kernel functions to incorporate nonlinearity. For more detailed information on the technique, please refer to Tharwat [261] or Comon [262]. For a comprehensive review of ICA’s application in process monitoring, it is recommended to consult Palla and Pani [263].
• Slow feature analysis (SFA): Proposed by Wiskott and Sejnowski [264], this technique is based on a non-linear optimization formulation that aims to extract a set of independent variables that exhibit slow variations over time. The underlying hypothesis is that slow variations are more relevant and informative compared to fast variations. SFA has gained significant popularity in the recent literature on process monitoring [265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282]. For more detailed information on the technique, please refer to Song and Zhao [283].

4. Machine Learning

Artificial intelligence (AI) is a rapidly evolving interdisciplinary field of science and engineering that has gained significant prominence in recent years. It is considered one of the most promising domains, alongside biotechnology, in terms of potential advancements, innovation, and societal impact [284,285,286]. The term “artificial intelligence” was first introduced in 1956 by J. McCarthy, M. L. Minsky, H. Simon, and A. Newell as:

“(…) the ability of machines to understand, think, and learn in a similar way to human beings”.

[286]

Kaplan and Haenlein [287] proposed a more specific definition:

“We define AI as a system’s ability to interpret external data correctly, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation”.

[287]

From these definitions, the significance of the learning process within AI becomes apparent. The subfield of AI that focuses on learning is commonly referred to as machine learning. Although the terms are occasionally used interchangeably, it is essential to emphasize that AI extends beyond machine learning. It encompasses various aspects, including data perception, knowledge representation, and the implementation of intelligent devices [284,287]. For an extensive discussion on the impact of AI in chemical engineering, we recommend reading Venkatasubramanian [285]. Regarding the specific impact of machine learning, several works have been published [16,288,289,290,291,292,293,294,295].

To gain a deeper understanding of the nature of machine learning methods, it is beneficial to examine some proposed definitions of the term [296]:

“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E”.

[297]

“Machine learning is essentially a form of applied statistics with increased emphasis on the use of computers to statistically estimate complicated functions and a decreased emphasis on proving confidence intervals around these functions”.

[298]

The first definition, which takes a pragmatic approach, focuses on understanding the concept of “learning” within the term “machine learning”. It highlights the operational aspects of running a computer program in a manner that can be considered learning. The second definition, taking a more theoretical perspective, provides a framework to visualize the relationship between machine learning and statistics. In this interpretation, machine learning is seen as an application of statistics, with the primary difference lying in the approach used. The use of the term “increased” is particularly intriguing as it captures a historical trend in the definition. As computational processing power continues to advance, learning algorithms increasingly rely on this capacity to estimate unknown functions, often sacrificing the interpretability of the underlying mechanisms and results for humans. This trend has witnessed significant growth, particularly with the emergence of deep neural network models [285,298].

Machine learning techniques are primarily classified based on the type of learning they employ. Supervised learning involves training models using a labeled training dataset, where each sample is associated with a desired solution or outcome. Conversely, unsupervised learning involves training models on an unlabeled training dataset, where the desired solution is unknown. Semi-supervised learning combines aspects of both supervised and unsupervised learning, as it utilizes a training dataset containing both labeled and unlabeled data. Another significant type of learning is reinforcement learning, in which an algorithm interacts with an environment by observing its state, selecting and executing actions, and subsequently receiving rewards or penalties. By exploring the environment and adjusting actions according to its consequences, the algorithm aims to maximize the cumulative reward it obtains over time. Supervised and unsupervised learning algorithms are widely used in process monitoring applications, while semi-supervised learning is gaining traction [230,299,300,301]. Within the context of PSE, reinforcement learning is predominantly used in automatic process control applications [302,303,304,305], although a few applications in process monitoring have been proposed [306,307,308].

The application of approaches from the machine learning literature to process monitoring problems can be traced back to the late 1980s when neural networks were first utilized [309,310,311]. However, during that time, such applications were limited to a few studies, with the predominant focus in the field centered around latent variable models from the perspective of multivariate statistical theory. In recent years, there has been a notable shift in the process monitoring community, driven by the development of data science and the successful application of machine learning methods, particularly deep learning, across various domains. This shift has resulted in an increasing emphasis on data science and machine learning approaches, taking precedence over traditional multivariate statistics, starting in the 2010s.

The analysis of key works by Joe Qin, a prominent researcher in the field, provides a clear illustration of this historical evolution. His important reviews on process monitoring techniques [14,18,25] primarily focus on latent variable models from the perspective of multivariate statistics. In the 2003 article [14], there is no mention of the term “machine learning”. In the 2009 article [25], there is only one vague reference to the term, appearing in the second line of the article, with a contextualizing meaning. By the 2012 article [18], the term “machine learning” starts to gain prominence, although it is presented as an alternative approach to the main methodology being discussed:

“A related category of nonlinear data-driven methods suitable for process monitoring is from the machine learning literature including neural networks [310] and support vector machines (SVM) [312]. Work in this area is rich by itself and a significant amount of work related to process monitoring is reviewed in Venkatasubramanian et al. [313]”.

[18]

It is noteworthy that in an article titled “Survey on data-driven industrial process monitoring and diagnosis,” the author allocates thirteen pages to methodologies related to PCA and PLS, while dedicating only one paragraph to machine learning techniques. Furthermore, the author directed readers seeking more detailed information on machine learning to a review paper that was nearly a decade old at the time of publication.

Two years later, in a perspective article [19], more attention was directed towards machine learning methods in the context of PSE. The article highlights the differences between machine learning and multivariate statistics, acknowledging the need to incorporate advances achieved in other areas into PSE:

“Although multivariate statistical approaches have been the favorite choice in the most recent 2 decades, it should be noted that data have been an integral part of process engineering solutions since time models have been used for process optimization and control. (…) However, these aforementioned data analytics and practice in process systems engineering have apparently not connected to the recent development in machine learning, data mining, and big-data analytics. They differ not only in terms of sizes but also in how and what data should be used in solving real operation problems. In some ways, process systems engineering solutions are confined to one set of principles that are believed to be sound, whereas the machine learning and data-mining communities take the other way and achieve unexpected results and solutions that defy conventional wisdom”.

[19]

In the most recent work [16], a noticeable shift in discourse is observed. Machine learning has taken center stage in the discussion, as evident from the title, “Advances and opportunities in machine learning for process data analytics”. Latent variable models, which were once the core focus, are now categorized as part of the suite of machine learning techniques:

“Although we consider multivariate latent variable methods as a subset of the statistical learning methods, the field of machine learning has grown tremendously in the last two decades (…) Depending on the characteristics of data and models, such as linear, collinear, or nonlinear data, there are respective supervised and unsupervised learning methods to choose from. To deal with collinearity in the data, for example, PCA is unsupervised learning and PLS is supervised learning”.

[16]

Considering multivariate statistical models as a subset of machine learning techniques is a conceptually elegant approach and holds theoretical validity. However, in the current study, we have made a deliberate choice to maintain rigor in describing the historical development. Consequently, it was considered more appropriate to present the techniques in separate sections.

In the subsequent portion of this section, an extensive overview of neural network models, considered one of the most successful models in machine learning, is presented from a historical and didactic standpoint, specifically in the context of process monitoring. Following that, a concise summary and review of other methods will be provided.

4.1. Artificial Neural Networks

4.1.1. Fundamentals

Artificial neural networks constitute a category of learning techniques that draw loose inspiration from the functioning of the human brain [314,315]. In the brain, neurons are interconnected through synapses, which act as connections between dendrites (inputs) and axons (outputs). These synapses undergo modulations in response to external stimuli, and it is through these modulations that the learning process occurs in living organisms [315].

Figure 6 illustrates the architecture of an artificial neuron. Within the network, each neuron k acts as a local processing unit that receives n inputs ($x_1$, $x_2$, …, $x_n$) and generates an output ($y_k$). The connections between the inputs and the neuron are assigned weights, represented as $w_{ik}$ for each connection i. The term $b_k$ is referred to as the bias and serves as a parameter that is independent of the inputs. It is analogous to the intercept in a linear regression model.

The output ($y_k$) of the neuron can be expressed as follows:

$$y_k=\varphi \left(\sum _{i=1}^n{w}_{ik}{x}_i+b_k\right),$$

(21)

where $\varphi$ is a non-linear activation function applied to the weighted sum of its inputs plus the bias term. In the neural network, the connections between multiple neurons form the network architecture. Similar to the learning process in living organisms, learning in this case occurs by adjusting the weights associated with the connections. There are several algorithms used for training neural networks, which can be categorized according to four basic learning rules, enumerated below [316]:

• Error correction learning (delta rule): in this type of supervised algorithm, the weights are adjusted to minimize the network prediction error, which consists of the difference between the network output and the expected result.
• Competitive learning: In this unsupervised algorithm, the weights are adjusted through competitive interactions between neurons. The objective of each neuron is to achieve the highest representation power relative to the patterns in the learning data.
• Boltzmann learning: In this unsupervised algorithm, the weights are adjusted based on the principle of energy minimization. The goal is to capture statistical dependencies in the data and create a model capable of generating samples similar to the training set.
• Hebbian learning: In this unsupervised learning method, weights are adjusted based on Hebb’s principle. According to this principle, if neurons downstream and upstream of a connection are activated simultaneously, the connection should be strengthened. Conversely, if the activation is asynchronous, the connection should be weakened.

Among the four types of learning mentioned, error correction learning is the most extensively applied and will be the primary focus of this section. In this context, the perceptrons are the simplest neurons capable of making decisions. A perceptron is a type of neuron that employs the binary step function as its activation function, defined as:

$$f\left(u\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}u\le 0\hfill \\ 1,\hfill & \mathrm{if}u>0.\hfill \end{array}\right.$$

(22)

By utilizing this activation function, it is possible to train the weights (w) of the neuron to create a binary classifier. To accomplish this, a straightforward algorithm known as the perceptron learning rule is employed. The perceptron learning rule guarantees convergence for cases where linear separability exists, and further details can be found in the work by Russell and Norvig [284].

To introduce greater complexity and flexibility in classification tasks, perceptrons can be connected in a network configuration, as depicted in Figure 7, which illustrates a network with two hidden layers.

It can be theoretically demonstrated that any function can be computed using perceptron networks, no matter how complex they may be [317,318]. In this sense, perceptron networks are computationally universal, meaning that any computational operation can be represented through a network of perceptrons. Despite this strong theoretical guarantee, neural networks were not widely used in the scientific community for many years due to the lack of an efficient training algorithm. However, the landscape changed with the pioneering work of Rumelhart et al. [319], who introduced the sigmoid neuron and the backpropagation algorithm. It is worth noting that the backpropagation algorithm has been independently invented multiple times over the years in various domains, with its origins traceable back to the work of Werbos [320] in behavioral sciences [321].

The process of training a neural network using the delta rule involves adjusting its weights to minimize prediction errors, aiming to achieve the desired output within a minimal margin of error. For successful training, it is crucial that the relationship between the weights and the output is smooth, meaning that small changes in the weights should result in small changes in the output [317]. However, when using perceptrons as neurons, the binary step activation function (Equation (22)) introduces discontinuity, leading to an abrupt and discontinuous relationship between weights and outputs.

To address this issue, Rumelhart et al. [319] proposed the sigmoid neuron, which utilizes the sigmoid function as its activation function. It is worth noting that there is evidence suggesting the use of sigmoid activation functions in biological neurons [321]. The sigmoid function is defined as:

$$f\left(u\right)=\frac{1}{1+exp(-u)}.$$

(23)

Figure 8 demonstrates the comparison between the binary step and sigmoid activation functions. The sigmoid function can be interpreted as a smoothing of the binary step function, thereby facilitating the training process.

Another major contribution from Rumelhart et al. [319] was the development of the error backpropagation algorithm. This algorithm consists of two stages in each iteration: a forward pass and a backward pass. During the forward pass, the network calculates its output, while during the backward pass, the contribution of each layer to the error of the preceding layer is determined (this process occurs in reverse, from output to input). The outcome of this backward pass is an efficient measurement of the error gradient throughout the network, enabling the application of the gradient descent algorithm to minimize errors.

Although the proposals made by Rumelhart et al. [319] generated significant interest in the scientific community regarding the utilization of artificial neural networks, challenges associated with training networks with multiple layers persisted for an extended period. It is only in recent times that these difficulties have been overcome through advancements in the field of deep learning. Deep learning is a methodology that uses multiple layers of information processing to represent complex concepts by associating simpler ones [298]. This approach is inspired by the way biological neural networks process information [321]. In the context of artificial neural networks, deep networks are characterized by having more than one hidden layer. Notable contributions in this field include those by Hinton et al. [322], who introduced a per-layer training strategy utilizing graphics processing units (GPUs), and Glorot and Bengio [323], who proposed alternative activation functions (in addition to the popular sigmoid) and new weight initialization strategies to address the problem of vanishing or exploding gradient across layers. Cutting-edge advancements in machine learning, such as the victory of the AlphaGo system in the complex Chinese board game Go [324,325] and the development of large language models like the generative pre-trained transformer (GPT) [326], predominantly rely on the application of deep neural networks.

One common criticism of methods in this class is the lack of interpretability of the results by humans, resulting from a limited understanding of the internal modeling mechanisms [285]. Recent studies [327,328] aim to address this concern through systematic investigations using carefully selected models. The issue of interpretability and its importance for future research is further discussed in Section 8.

For an extensive review of the field of deep learning, the reader is referred to Sengupta et al. [329]. Yu and Zhang [1] offer a review specifically focused on the application of deep learning in process monitoring. Zhou and Zhu [330] provide a critical review of the application of deep neural networks in modeling dynamical systems.

It is important to note that besides the realm of deep learning, other types of advancement have been proposed in traditional training algorithms. One notable example is the concept of extreme learning machines (ELM) introduced by Huang et al. [331]. ELM facilitates the rapid learning of neural networks by employing a random selection process for the nodes where the weights are trained.

4.1.2. Neural Network Architectures

A variety of architectures can be proposed for neural networks [315,332]. The most common distinction lies between feed-forward (or acyclic) networks and recurrent (or cyclic) networks. In feed-forward networks, information flows unidirectionally, from input to output, as illustrated in Figure 7. Conversely, recurrent networks process inputs sequentially, transforming them into hidden states, which are then retrofed back into the network. This enables the interaction of an input provided at time t with inputs provided at previous times ($t-1,t-2,\dots$). Consequently, recurrent networks prove particularly valuable for modeling time series, textual sentences, or biological sequences [315].

Attention mechanisms, proposed by Bahdanau et al. [333], aim to enhance the performance of deep neural networks by assigning weights to and combining input data based on their relevance to the given task. Particularly, self-attention, introduced by Cheng et al. [334], enables each position in the input data to independently relate to all other positions. This concept has revolutionized the field of deep learning, particularly in the context of sequential data, enabling the development of feed-forward architectures capable of modeling long-range dependencies using parallelizable training algorithms [335]. The Transformer architecture [336], widely employed in large language models such as GPT [326], is built upon self-attention mechanisms.

Convolutional networks are a type of neural network characterized by their sparse architecture, where filters are selectively applied to activate only the neurons essential for optimal learning [285]. This unique design has proven to be highly successful, particularly in computer vision applications, where convolutional networks have recently surpassed human capabilities [337]. By being inherently sparse, convolutional networks excel at learning and recognizing local patterns, which can be identified at various positions within different samples. In the context of image processing, these patterns can encompass edges, curves, and other geometric shapes, allowing convolutional networks to effectively capture the visual characteristics of images. This capacity to detect and extract meaningful features from images has made convolutional networks a cornerstone in various computer vision tasks, such as object recognition, image classification, and segmentation.

Autoencoders are neural networks specifically designed to generate, with certain constraints, output data that closely resembles input data [338]. The utility of this methodology results from the nature of these constraints. For instance, if the hidden layer contains fewer neurons than the input layer, the network is compelled to represent the data in a lower-dimensional latent space, referred to as the “code.” This behavior is reminiscent of the latent variable modeling and dimensionality reduction process observed in the PCA model [339]. For a comprehensive examination of the application of autoencoders in process monitoring, please refer to Qian et al. [340].

Generative adversarial networks (GAN’s), originally proposed by Goodfellow et al. [341], are composed of two neural networks: the generator and the discriminator. These networks are jointly trained to produce synthetic data that progressively resembles real data. The generator network aims to generate data that closely resembles real data and receives feedback from the discriminator network. The discriminator network, on the other hand, is trained to maximize its ability to distinguish between real and synthetic data. GANs have shown promise in various applications, including time series analysis and anomaly detection. Comprehensive reviews on the application of adversarial generative networks to these specific problem domains have been published [342,343].

Restricted Boltzmann machines utilize the Boltzmann learning rule along with topology constraints to enhance the efficiency of the training process [344]. A prominent class of deep learning networks that employ unsupervised learning is known as deep belief networks, which are constructed by composing restricted Boltzmann machines [345]. Moreover, restricted Boltzmann machines are commonly employed in the pre-training phase of conventional (supervised) deep networks [315].

Self-organizing maps combine the principles of competitive learning and Hebbian learning to create typically two-dimensional representations of complex patterns while preserving the topological structure of input data [346,347]. These representations facilitate pattern visualization, particularly in tasks such as clustering or latent variable modeling.

Fuzzy architectures, also known as neuro-fuzzy systems, employ neural networks as a learning mechanism to adapt parameters within a fuzzy logic system [15]. Fuzzy logic represents a generalization of Boolean logic, where the truth values of variables can take continuous values within the range of 0 (false) to 1 (true). It is important to note that fuzzy logic, independent of neural networks, can be utilized to design learning systems, as demonstrated by applications described in Section 4.2.

4.1.3. Applications in Process Monitoring

Applications of neural networks for monitoring purposes can be historically categorized into two approaches. The first approach, classified as supervised learning, involves assigning a scenario (normal or faulty) to each output node of the network. The network is then trained to classify a given dataset according to these scenarios [310,348,349]. In the second approach, classified as unsupervised learning, neural networks are integrated into traditional methods such as principal component analysis (PCA) to introduce nonlinearity [82,83]. Advancements in this second approach include the utilization of neural networks to extract nonlinear features independently from the formalism of traditional techniques, as proposed by Zhao and Lai [350].

There are works involving applications of process monitoring using several network architectures, such as recurrent networks [351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367], convolutional networks [299,349,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382], autoencoders [282,365,379,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405], generative adversarial networks [406,407,408,409,410,411,412,413,414,415,416], Transformers [417,418,419,420,421,422,423,424,425], restricted Boltzmann machines/deep-belief networks [345,412,426,427,428,429,430,431,432], self-organizing maps [433,434,435,436,437,438,439,440,441], neuro-fuzzy networks [442,443,444,445,446,447,448,449,450,451] and extreme learning machines [452,453], including recently in unsupervised form [454,455,456].

Besides specific works on process monitoring, ample perspectives on general applications of neural networks in chemical engineering have been published [332,457,458,459].

4.2. Other Techniques

The field of machine learning encompasses a rich body of literature, presenting numerous other techniques that deserve mention, including:

• $\mathbf{k}$-nearest neighbors ($\mathbf{k}$-NN): This method involves classifying a point in variable space based on the classes of its closest k neighbors. It is a simple and effective machine learning technique commonly used for datasets that are not excessively large and lack known probability distributions [15]. Several applications have utilized k-NN classification in the field of process monitoring, such as application to a semiconductor manufacturing process [460]; identification of faults that affect the entire plant through the application of the k-NN method for estimating delays [461]; diagnostic methods inspired by the ideas of reconstruction and contribution [462,463]; use of Dynamic time warping (DTW) algorithm [464,465,466] to define the Mahalanobis distance between the time series to be used in the k-NN technique [467]; and dealing with variable sample rates through sample grouping [468].
• Support vector machines (SVMs): Support vector machines, originally proposed by Boser et al. [469], aim to identify hyperplanes that can effectively separate the variable space into distinct regions of interest [15,321]. These hyperplanes are determined based on a small subset of observations known as support vectors. To accomplish this, a concept called margin is defined, representing the distance between the hyperplane and the support vectors. The training process involves solving an optimization problem: in classification tasks, the goal is to maximize the margins, while in regression tasks, the objective is to minimize them. To introduce nonlinearity, SVMs utilize mappings through kernel functions [99] to transform the data into alternative mathematical spaces where hyperplanes can be employed to address the specific task at hand. Two methods have been proposed to expand the application of SVM to unsupervised classification scenarios: support vector domain description (SVDD) and one-class SVM (1C-SVM). SVDD, initially proposed by Tax and Duin [470], utilizes a hyperplane as the geometric form for separation, while 1C-SVM, introduced by Schölkopf et al. [471], employs a sphere. Recent applications of SVM in the field of process monitoring include: methodologies to select Gaussian kernel parameters for applications in fault detection [472]; integration of the SVM with Fisher discriminant analysis and wavelets for application in multiscale problems [236]; integration of the SVM with the recursive feature elimination technique for variable selection [473,474]; an adaptive methodology for application in non-stationary processes [475]; outlier treatment through a pruned SVDD algorithm [476]; a two-step SVDD method with genetic algorithm optimization to deal with dynamic, non-linear and non-Gaussian features [477]; development of an SVDD methodology to deal with incomplete data coming from industrial units with multiple local units [478]. It is worth noting that support vector machines (SVMs) have found broader applications in the field of mechanical engineering compared to chemical engineering [479,480]. This discrepancy may be attributed to the inherent characteristics of data generated by mechanical machines, which typically exhibit less redundancy and collinearity compared to data from chemical processes.
• Tree-based methods: The decision tree model constitutes a tree-like representation that encompasses various possible decision paths and their corresponding outputs [481]. The direct utilization of decision trees as machine learning models offers the significant advantage of complete transparency and interpretability regarding the decision-making process. Due to its simplicity, the application of decision trees in the field of process monitoring is relatively limited, with more prevalence in the realm of quality control [482,483]. A commonly employed strategy involves utilizing an ensemble of trees, such as in the random forest algorithm where multiple trees are trained using random subsets of the training data. The final result is determined by aggregating the values from each tree, often through averaging. This approach finds extensive use in process monitoring applications [484,485,486,487,488,489,490,491,492,493,494,495].
• Manifold learning: Within this category of techniques, the aim is to reduce the dimensionality of the data while preserving the underlying nonlinear relationships between its elements [496,497]. These methods operate by projecting the data onto manifolds, which are local topological subspaces that can be treated as approximately Euclidean. This mechanism proves valuable in capturing the local geometric structure of the data, surpassing the limitations of traditional latent variable models such as PCA, which only consider global Euclidean structures for dimensionality reduction. Specific models include t-SNE [498], which models similarities between data points using probability distributions, and UMAP [499], which uses a combination of graph-based and probabilistic approaches to achieve better global structure representation and computational efficiency. Examples of applications in the process monitoring field include: a method based on kernel functions specially designed for batch processes [500]; use of manifold learning in conjunction with clustering methods for temporal alignment and phase identification of batch processes [501]; a new monitoring index for control charts based on manifold learning [502]. A recent trend is the proposition of methodologies that simultaneously represent local and global structures in the data [503,504,505,506,507,508,509].
• Sparse dictionaries: Initially introduced by Olshausen and Field [510] for image recognition tasks, the concept of learning through sparse dictionaries involves the use of bases with dimensions greater than that of the original space, enabling the creation of a sparse representation of the data [511]. The basis itself is referred to as a dictionary, with each constituent element termed an atom. According to Mairal et al. [511], the utilization of sparse representation facilitates the capture of high-order correlations more effectively. Although the application of this methodology to monitoring problems is relatively recent, it holds promising potential. Examples of work in the field of process monitoring related to sparse dictionaries include: a sparse representation classification method for fault detection and diagnosis [512]; application of dictionaries to a semiconductor manufacturing process [513]; generation of sparse contribution graphs [514]; fault detection in multimodal processes [515]; use of kernel functions to capture nonlinearity [516]; a robust methodology for monitoring processes with multimodality, noise and outliers [517]; a methodology for decomposing large systems into modules, in order to facilitate the detection of small faults [518]; a distributed computing methodology to deal with big data [519]; an adaptive technique to deal with multimodal processes [520]; and a method for linear and nonlinear variable partition applied to dynamic process monitoring [521].
• Bayesian networks: Bayesian networks are graphical models represented by directed acyclic graphs (DAGs) that capture probabilistic dependencies among a set of variables [15,522]. This modeling framework excels in handling uncertainty and offers causal inference capabilities, enabling the description of relationships between causes and symptoms [523,524]. Examples of works in the field of process monitoring related to Bayesian networks include: use of specialist knowledge to build a Bayesian network for monitoring a tubular reactor [525]; construction and causal decomposition of the $T^2$ control chart [526]; monitoring of dynamic processes for propagation identification and root cause diagnosis [527]; use of Bayesian networks to filter the smearing-out effect that occurs in contribution methods for fault diagnosis [528]; development of a robust Bayesian network to deal with low-quality data [529]; development of a Bayesian classification network based on statistical alarms to deal with the effect of noise [530]; detection of kicks of hydrocarbons in drilling operations [531]; combination of mechanistic and state transition analysis to recognize propagation paths of process faults [532]; a graphical way to represent the causal interactions of a process using an interpretable Bayesian network [533]. A recent trend involves integrating Bayesian networks with traditional methods such as PCA and ICA [534,535,536]. For a tutorial review on applying Bayesian models to the process monitoring problem, refer to Raveendran et al. [537].
• Methods based on fuzzy logic: Fuzzy logic serves as an extension of Boolean logic, allowing for variable truth values to range continuously between 0 (representing false) and 1 (representing true). Monitoring systems that leverage fuzzy logic have been developed in various studies, including the proposal of an adaptive classification system capable of incorporating information about different modes of operation [538]; a fuzzy logic-based clustering algorithm applied to the Tennessee Eastman benchmark [539]; a combination of genetic algorithms and fuzzy logic-based clustering techniques to identify patterns in multivariate time series data [540]; the proposal of a novel clustering algorithm by integrating k-NN techniques with fuzzy k-means [541]. It is worth noting that fuzzy logic systems can be integrated with the learning capabilities of neural networks, as discussed in Section 4.1.2.

In addition to traditional methodologies, a recent trend has emerged in which techniques that are less limited by predefined model structures are employed. These approaches prioritize the exploration of intrinsic patterns inherent in the data itself. Several notable methods within this trend include:

• Clustering: In this approach, which is one of the most popular unsupervised learning methodologies, data is divided into clusters based on some similarity measure [542]. In the realm of process monitoring, various applications have emerged, including a fuzzy methodology to perform clustering based on wavelets and the PCA model [543]; a hybrid method that combines Euclidean and PCA-based similarity measures [544]; a new clustering algorithm combining the techniques $\mathbf{k}$NN and fuzzy k-means [541].
• Logical analysis of data (LAD): The LAD technique, introduced by Hammer [545], involves the binarization of data followed by the discovery of interpretable patterns using boolean modeling approaches [546]. The key advantage of this technique lies in its ability to produce patterns that can be readily interpreted, facilitating the transformation of information into human understanding and knowledge. Applications of LAD in the field of monitoring have been recently proposed [362,547,548,549,550,551].
• Empirical machine learning: This approach revolves around the density of data in the variable space and yields algorithms that are entirely data-driven, devoid of probabilistic models and user-defined hyperparameters [552,553,554,555,556]. The theory introduces novel concepts such as cumulative proximity (a centrality measure), eccentricity (an anomaly measure), and typicality (a density measure). Applications in process monitoring have been developed [557,558,559,560,561].
• Recurrence matrices and distance matrices: Recurrence matrices and distance matrices are two-dimensional representations of data distance patterns. Applications of this methodology in process monitoring have been proposed in combination with various techniques, such as convolutional neural networks [371], recurrence quantification [562,563,564,565] and texture analysis [566,567,568]. Melo et al. [569] have recently proposed a novel methodology for visual process data analytics and process monitoring based on a direct analysis of distance matrix patterns.

5. Exploration, Characterization and Treatment of Process Data

In classical statistics, the theory is primarily developed based on the assumption that the data being analyzed originate from experiments specifically designed to test particular statistical hypotheses. On the other hand, historical process data arises from unplanned observations of real-world events, and the classical statistical literature strongly discourages the analysis of such data [570]. Consequently, applying classical techniques directly to historical data may not yield satisfactory results.

The field of statistics and data science that aims to explore and characterize data independent of any specific model is known as exploratory data analysis (EDA) [5]. In EDA, according to Morgenthaler, “one is free to choose any procedure to analyze the data, and the primary aims are to look at the data and to think about the data from many points of view”. The application of EDA to process data has been discussed in the literature over the years. Pearson [570] described simple procedures, including quantile-quantile plots, data comparison plots, box plots, and moment characterizations, emphasizing the importance of such exploration. It was argued that unexpected data characteristics, such as outliers from different sources, can significantly invalidate results obtained through process monitoring techniques, particularly those based on quadratic error criteria. The author’s main recommendations encompassed the following: (i) conducting adequacy tests for specific distributions, with particular emphasis on the Gaussian distribution, and comparing distributions across different datasets; (ii) employing robust statistical characterization methods, such as using the median instead of the mean, MAD (median absolute deviation from the median) instead of the standard deviation, and Spearman’s correlation coefficient instead of Pearson’s correlation coefficient; (iii) utilizing visualization methods such as box plots to compare variables. Abonyi [571] utilized box and quantile-quantile plots to analyze real historical data from a polyethylene production plant. The objective was to examine various manufacturing processes for a specific product and investigate the connections between different operational factors and the final product quality. Zhang et al. [260] proposed a data characteristics test to automate the selection of process monitoring techniques. Deviations from Gaussianity and linearity were used as indicators to aid the decision-making process regarding combinations and variations in the PCA and ICA models.

In addition to exploring process data, preliminary treatment procedures are crucial to ensure the effectiveness of modeling. In this context, Xu et al. [572] conducted an extensive review of the treatment of industrial data, referred to as data cleaning in their study. Four key steps in the data cleaning process were identified: filling in missing values, detecting outliers, removing noise, and performing time alignment/delay estimation. A wide range of methods available in the literature for each of these challenges was presented. The authors emphasized that the choice of data processing methods should be guided by the specific properties of the dataset itself, such as the proportion of missing data, as well as the existing knowledge about the process, such as the availability of a known model. Moreover, the methods employed should preserve and reveal the characteristics of regular data while effectively identifying the mechanisms that contribute to contamination, such as the nature of outliers.

In the context of robust methodologies, particularly concerning outliers and missing data, Zhu et al. [154] conducted a review from the perspective of big data. They examined commonly employed techniques for data preprocessing, as well as robust variants of PCA and other monitoring techniques.

An important step prior to applying a monitoring technique is the selection or generation of input variables. The area of machine learning that deals with this question is known as feature engineering. A notable approach gaining attention in recent years is statistical pattern analysis (SPA), proposed by Wand and He [573,574]. In SPA, statistical measures of process variables (mean, variance, autocorrelation, cross-correlation, among others) are selected as input variables for the modeling techniques (typically, PCA), which in many cases improves detection and diagnosis performance. Recent research on variable selection includes: the application of a criteria based on mutual information for variable selection [575]; an efficient technique based on sequential analysis of fault contributions [576]; a technique to extract non-stationary variables and its application to fouling detection [577]; the integration of variable selection methods with moving window techniques for multimodal monitoring, achieving good results on a particularly challenging fault in the TEP benchmark [578]; a comparison of shallow and deep learning methods for selecting variables in an industrial test platform focused on big data and Internet of Things concepts [579]; and the application of causality-based methods for feature selection in the TEP benchmark and in an oil platform fiscal metering plant [580]. Peres and Fogliatto [4] published a review on this subject, highlighting that the most used methods are LASSO regression, the genetic algorithm and regression by direct selection (forward selection).

Siang et al. [581] described good practices for acquiring and preparing process data. The authors emphasized the importance of (i) understanding the nature of the data through exploratory analysis; (ii) appropriately selecting the data subset to be used in the modeling; (iii) paying attention to open or closed loop conditions; (iv) align sampled signals at different rates; and (v) adjust the choice of metrics to the practical needs of the business problems.

Thibault et al. [582] conducted a review of data processing methodologies employed in process industries, focusing specifically on the pulp and paper sector. The authors divided these methodologies into three main categories: data cleaning, steady-state detection, and operating regime detection. Interviews with process specialists and software developers were conducted, leading to interesting insights regarding the gap between research advancements and practical implementations, among other points.

On the broad theme of exploration, characterization and treatment of process data, it is also worth highlighting the recent works by: Li et al. [583], which proposed dataset division based on the nature of the correlation (linear or non-linear) detected between variables and the application of a hierarchical strategy in these datasets to monitor linear and non-linear characteristics at different levels; Thomas et al. [584], who presented an integrated approach to data clustering and feature extraction, as an aid to increase knowledge about process data; Parente et al. [585], who proposed the application of the Monte Carlo technique to increase the amount of data used in training a monitoring model for a paper production process; Offermans et al. [586], which compared different methods for dynamically synchronizing process variables; and Rhyu et al. [587], which proposed an integrated methodology for automated outlier detection and missing data estimation.

6. Computational Tools and Benchmarks

6.1. Softwares

A wide range of computational tools exists for applying data-driven process monitoring techniques, with most of them being commercially developed. However, there is a lack of open-source options available. These options could significantly contribute to scientific progress, as exemplified by the remarkable advancements in the broader open ecosystem of data science and machine learning. This section specifically focuses on open-source tools for process monitoring within the field of PSE. Additionally, Appendix B provides a non-exhaustive list of some key commercial tools.

Open tools for PSE-based process monitoring are commonly developed as extensions within the Matlab environment. Camacho et al. [588] introduced MEDA, a specialized toolbox for multivariate analysis in the realm of chemometrics, featuring PCA and PLS methods, along with various monitoring applications. Jiang et al. [589,590] proposed the DB-KIT, designed for handling key performance indicators (KPIs) using regression algorithms and latent variable modeling. For batch processes, Gonzalez-Matínez et al. [591] introduced MVBatch, which offers notable functionalities such as data preprocessing, model cross-validation, and development on the GitHub platform. Yi et al. [592] presented Pre-Screen, a dedicated toolbox for pre-processing process data. Additionally, Villalba et al. [593] developed a graphical interface tutorial to facilitate comprehension of PCA-based monitoring. Schaeffer and Braatz [594] created a didactic Matlab application to aid in understanding latent variable models and regression. Melo et al. [5] proposed KydLIB, an open-source Python exploratory analysis package specially designed for historical process data. Rhyu et al. [587] made available in a GitHub repository a set of routines for automated outlier detection and missing data estimation. Furthermore, Sun et al. [595] and Alizadeh et al. [596] reported the existence of Matlab toolboxes for latent variable methods and causality analysis, respectively. However, the specific names of these tools and their download links were not provided in the referenced sources, and we were unable to locate them through internet searches.

As is apparent, there is a lack of open computational tools in the field of PSE industrial process monitoring, especially when compared to the wide range of options available in the data science environment. Moreover, the existing tools are mostly extensions of the Matlab environment, which hinders collaborative development due to its closed nature and restricts their usage in industries without the required software licenses. This issue is further discussed in Section 8.

6.2. Benchmarks

Benchmarks are standardized tests used to evaluate and compare the performance of computational systems, encompassing both hardware and software components [597]. In the field of PSE, a variety of open benchmarks exist to assess novel fault detection and diagnosis techniques. These benchmarks provide researchers with standardized datasets or simulation models that enable them to evaluate the efficacy and efficiency of their proposed methods. The remainder of the section provides a compilation of these benchmarks along with concise descriptions.

The Tennesse Eastman Process (TEP) is a chemical process simulation developed by Eastman Chemical Company to serve as a benchmark for evaluating process control and monitoring applications [49]. Originally, the TEP model was available as a Fortran-based process simulator, but it has now been implemented on different platforms. It continues to be the most widely used benchmark for process monitoring in the field of PSE [5].

PenSim is a simulation framework designed for studying the production process of penicillin. It incorporates an unstructured model of penicillin fermentation, initially introduced by Bajpai and Reub [598] and subsequently expanded by Birol et al. [599]. This particular process is the leading benchmark for evaluating methods developed specifically for batch processes [5].

DAMADICS (Development and Application of Methods for Actuator Diagnosis in Industrial Control Systems) is a benchmark comprising both a process simulator and real datasets representing electro-pneumatic actuators used in a sugar production process [600]. The original simulator was developed using the MATLAB environment, while the real data was collected from the Lublin Sugar Factory in Poland. Due to its focus on a single equipment component, the DAMADICS benchmark is particularly suitable for evaluating model-based process monitoring techniques. However, there have also been studies that explore data-driven approaches within this benchmark [5].

The Benchmark Simulation Model No. 1 (BSM1) is a comprehensive specification of a simulation environment specifically created for evaluating control strategies in wastewater treatment plants [601]. Unlike the previous examples, BSM1 is a model specification, requiring users to implement it in their preferred programming language or simulation platform.

IndPenSim is a phenomenological model specifically designed to depict the fermentation process of Penicillium chrysogenum [602,603]. Unlike the conventional PenSim benchmark, IndPenSim stands out due to its industrial-scale representation, utilizing volumes of 100,000 L instead of the typical 100 L vessels employed in PenSim. Additionally, it benefits from the availability of real historical data and employs a structured model [604] that encompasses crucial aspects such as growth, morphology, metabolic production, and biomass degeneration.

The Cranfield Multiphase Flow Facility serves as a real-data benchmark for developing and validating statistical process monitoring procedures [129,605]. It represents a pilot-scale three-phase flow separation process, encompassing pipelines, a gas-liquid two-phase separator, and a water-oil-air three-phase separator.

The PRONTO (Process Network Optimization) heterogeneous benchmark dataset [606,607] builds upon the Cranfield benchmark, expanding its scope to address challenges associated with diverse data acquisition and storage formats. This dataset was specifically designed to incorporate a range of data types, including process measurements (such as high-frequency ultrasonic flow and pressure sensors), alarm records, events and change logs, an operation log, and video recordings.

The 3W dataset [608] offers a comprehensive collection of data encompassing real, simulated, and hand-drawn samples, which represent both normal and faulty operation periods of offshore oil wells across multiple well locations and operational timeframes. This benchmark dataset presents unique challenges inherent to real-world scenarios, including a substantial number of missing observations and frozen or incomplete variables.

BDsim is a biodiesel production process simulator that focuses on the continuous transesterification of oil with methanol [609]. It offers a comprehensive range of potential faults at different levels, including process, sensor, and actuator. Notably, BDsim surpasses existing simulators by providing previously unexplored scenarios like intermittent faults and equipment degradation.

The FCC Fractionator is a model designed to simulate a refining process that incorporates a fluidized bed catalytic cracker and a fractionator [610]. This recent proposal holds significant potential for fostering innovation in the field of process monitoring.

The Arc Loss Challenge is a benchmark that focuses on data from an industrial-scale mining and pyrometallurgy process [611]. This unique challenge introduces a comprehensive benchmarking workflow, featuring submission and evaluation pipelines in a competitive format.

For an extensive review and in-depth exploration of process monitoring benchmarks, the reader is referred to Melo et al. [5].

7. Industrial Applications

This section focuses on the application of data-based process monitoring in real industrial settings. It provides an overview of the historical development of industrial implementations in this field and describes the main works that have contributed to its advancement.

The application of the PCA technique in industrial settings dates back to the late 1980s. Wise et al. [11] introduced the use of PCA for process monitoring and demonstrated its efficacy through an application to data obtained from a ceramic smelter used for reprocessing waste in the nuclear industry. This study revealed that employing PCA on historical data could enhance the understanding of an industrial process, primarily by examining the contribution of each variable to the generation of principal components. Real-time industrial application results pertaining to this equipment were subsequently reported [612].

In the next years, numerous studies described the application of PCA and PLS in various industrial systems. Notably, Slama [613] investigated the operation of a catalytic cracking fluidized bed and the fractionation section of a refinery, shedding light on practical aspects associated with the analysis of real data. The study examined crucial factors such as the validity of data assumptions (e.g., absence of autocorrelation), sampling time, and its relationship with process time constants, among others. The findings of this work significantly contributed to advancing the understanding of the process. For instance, it revealed variations between different operating periods and identified a drift in the overall dynamics of the plant, despite the products being within the desired specifications. However, the presence of nonlinearities posed challenges to the performance of real-time monitoring systems.

During that period, a prominent area of application involved the utilization of multiblock and multi-way methods in batch and semi-batch polymerization systems, with many studies using real data [12,170,614]. However, these works primarily focused on the methodology and mathematical aspects of the techniques, rather than emphasizing their practical implementation. In contrast, notable contributions from the company DuPont stood out during the same period for their strong emphasis on case studies [63,615,616]. For instance, Piovoso and Kosanovich [616] addressed challenges related to the lack of representative data and the resistance of operating staff to conduct planned experiments that encompassed the entire operational range. This resistance primarily resulted from concerns about producing an excessive amount of products that fell outside the desired specifications.

In a survey conducted by Kourti and MacGregor [617], the applications of PCA and PLS methods up to that point were examined. The authors also introduced two novel applications specifically in continuous and semi-batch polymerization processes. The survey highlighted the rapid acceptance and adoption of multivariate statistical monitoring techniques within the industry. One of the case studies presented in the survey demonstrated how the application of these methodologies promptly identified variables that led to low purity and/or recovery. This task had previously consumed a significant amount of time and effort for the plant engineers. In another study, a fault resulting from a typing error made by an operator was successfully detected. However, it is important to note that real-time implementations of these methodologies were not presented in the case studies discussed.

In a highly influential article, Wise and Gallagher [13] provided a comprehensive and didactic presentation of the monitoring methodology using the chemometric approach to the process control community. This work served as a consolidation of the research conducted by the group in previous years. The article included several application examples to illustrate the methodology. These examples encompassed various domains, including (i) the ceramic melting equipment, which had been previously studied by the same authors [11,612]; (ii) monitoring the mitigation of nuclear waste in storage tanks, where subtle trends were identified through principal component analysis; (iii) monitoring a batch process utilizing near-infrared spectrometry (NIR), employing evolving factor analysis and multivariate curve resolution techniques. These application examples illustrated the effectiveness and versatility of the chemometric approach in process monitoring and control.

Following the influential work of Wise and Gallagher [13], there was a notable increase in the variety of research papers featuring examples of industrial applications using latent variables models. An example of this trend is demonstrated by Ignova et al. [618], who applied multi-way PCA and PLS to data from a fermentation bioprocess. Their study revealed that substantial cost reductions were possible by utilizing the developed models to determine feed quality and predict the concentration of the final product. Subsequent works reported the real-time application of the methodology, which was integrated into an expert system called the G2 real-time expert system [619,620].

Gurden et al. [621] reported the introduction of chemometric techniques in an industrial pilot plant. Several noteworthy results were achieved, including fault detection and diagnosis, the utilization of principal components to generate process trajectory graphs for monitoring plant operation, and the application of latent variables in cross-correlation analysis to enhance the interpretive capacity of analyzing time-delay effects. The authors highlighted the significance of their findings, as the gained experience and improved process understanding empowered plant personnel to propose the installation of new sensors and analyzers. This proactive approach was driven by the anticipated competitive advantage that these advancements would provide.

Taylor [622] documented the application of PCA for detecting instabilities in a blast furnace of British Steel. Notably, the implementation of PCA-based monitoring led to a significant increase in anticipation time compared to traditional detection methods. This work stands out for two interesting characteristics: firstly, it provides a detailed account of the real-time implementation within the plant, offering more comprehensive insights than previous studies; secondly, it is a notable industrial work conducted without the participation of university specialists as authors, marking a departure from the trend observed since the DuPont studies in the early 1990s.

Neogi and Schlags [623], from the Air Products and Chemicals company, applied MPCA and MPLS to analyze an emulsion polymerization batch process. A novel approach was adopted, where the trajectories of each batch were compared based on reaction extent rather than batch time, marking a significant departure from previous practices. Although real-time implementation was not carried out, the application of monitoring methods to historical data yielded successful fault detection and diagnosis. Additionally, a PLS model was developed, enabling the prediction of product viscosities. This enhanced process understanding and contributed to increased operational efficiency.

Martin et al. [624] presented an application of the PCA technique to data from a fluidized bed reactor and addressed crucial aspects that they believed were necessary for the widespread industrial adoption of these techniques. These aspects included: (i) developing monitoring schemes that require minimal amounts of process data; (ii) exploring the use of multi-block techniques to monitor the entire plant; (iii) developing generic models capable of monitoring multiple products or grades; (iv) investigating the impact of dynamics and control loops on the performance of monitoring systems and the potential for masking faults; (v) assessing the sensitivity of PCA and PLS and evaluating the suitability of their confidence limits; and (vi) extending the use of density-based confidence limits for monitoring dynamic processes.

Bissessur et al. [625] explored two approaches for fault detection in a papermaking plant. The first approach involved monitoring specific equipment using neural networks applied to vibration data. The second approach utilized PCA to monitor the overall process. Interestingly, the authors made an important observation that faults were better detected by analyzing the lower-order principal components. This finding challenged the conventional belief that higher-order principal components contain the most important information. In this case, the first components primarily captured variations in mass and energy balances as well as movements of setpoints in the plant. This insight highlighted the significance of considering the lower-order principal components for effective fault detection, contrary to the usual assumption about the relative importance of principal components in PCA analysis.

Dudzic et al. [626] described with unprecedented detail the implementation of PCA and PLS techniques in industrial processes at Dofasco, a steel production company. The implementation of these techniques has been ongoing since 1993, as described by the authors who were all employees of the company. The report focused on two specific applications: (i) continuous casting machine monitoring, and (ii) a control system for determining the optimal amount of reagents required for sulfur removal from pig iron. The authors emphasized the progression from offline analyses to online implementations throughout the 1990s, which was supported by close collaboration with the Advanced Control Consortium at McMaster University. The employed techniques offered a balance between ease of development and insights into the process, surpassing the limitations of purely phenomenological or empirical models. Additionally, these techniques demonstrated robustness against noise and missing data. The report highlighted the significance of data pre-treatment, including the selection and application of filters, grouping of measurements to eliminate trends, and handling of missing data. In the monitoring of the casting process, PCA was applied to reduce the dimensionality of the training matrix from 240 to 10 variables, considering the application of delays for dynamic treatment. The authors provided the operators with two control charts based on the $T^2$ index and one based on the Q index, implementing a progressive alarm scheme with yellow and red colors, respectively. The authors reported impressive results, including a 50% reduction in downtime after two years of operating the monitoring system and achieving the highest productivity in the machine’s 12-year history.

Albert and Kinley [627] investigated the application of PCA and PLS techniques to monitor the biosynthesis of the antibiotic tylosin during batch fermentation. The authors encountered interesting challenges associated with poor data quality and variations in data quality across different locations within the plant. These issues posed significant obstacles for data pre-treatment and highlighted the difficulties in integrating monitoring technology into the plant since the variation in data quality added an extra layer of complexity to the monitoring process.

Karim et al. [628] documented their experience with the implementation of data-based methods in the bioprocess industry. The study shed light on specific challenges faced in this field, such as extracting biologically meaningful information from a limited number of real-time process measurements. One of the complications arises from the fact that the measured data primarily reflects the conditions inside the bioreactor, while the desired information pertains to the physiological state of the cells, which traditionally necessitates expensive offline analyses. The authors demonstrated how the utilization of MPCA helped alleviate these limitations by enhancing data interpretation capabilities.

Kumar et al. [629] reported the implementation of PCA in a high-pressure polymerization process. The study demonstrated that the technique effectively identified various multivariate faults, characterized by correlations, which were not detectable using univariate charts, where individual variables remained within their normal operating ranges. Notably, transitions in product grades and process changes were more clearly observed in the latent variable space. While the study mentioned the installation of a monitoring system at the plant, the specifics of the online operation were not described.

In 2003, a symposium titled “Abnormal situation detection and projection methods—industrial applications” took place in Ontario, Canada. Kourti [630] provided a comprehensive summary of the event, including descriptions of the presentations delivered. The symposium was attended by professionals from companies such as Dofasco, DuPont, Perceptive Engineering, and the COREM consortium (Mineral Research Consortium), as well as researchers from various universities. The following are brief summaries of the four lectures presented:

• Paul Nomikos from DuPont presented their successful experience with the utilization of PCA and PLS for offline analysis. They highlighted the effectiveness of these methods in tasks such as identifying and resolving sources of variability, enhancing phenomenological processes, and developing new control strategies. However, Nomikos also emphasized the challenges associated with implementing these techniques in an online setting. Specifically, the long-term maintenance of the models was identified as a significant hurdle that needed to be addressed to enable successful online implementation.
• Theodora Kourti provided practical recommendations for the application of data-driven monitoring systems. She stressed the significance of operator acceptance, noting that even within the same company, similar units may require different screens based on operator preferences. Kourti suggested starting implementations with small-scale problems rather than attempting to monitor the entire plant right from the beginning. Furthermore, she emphasized the importance of process knowledge, as it helps determine appropriate transformations and weights for variables, as well as assess the quality of obtained data. According to her findings, a linear model is typically sufficient for monitoring deviations. Kourti concluded her recommendations by identifying three motivating factors for industries to adopt multivariable monitoring systems: financial return, safety concerns, and compliance with environmental regulations.
• Barry Lenox emphasized that numerous monitoring projects represent pioneering endeavors within their respective industries. He noted that once the initial application is successfully implemented, the transfer of the monitoring system to similar processes becomes relatively straightforward. Lenox further shared his experience, indicating that the confidence of operators, engineers, and managers in these systems can grow to a point where automatic shutdowns or interventions can be programmed based solely on the information provided by the monitoring systems.
• Vit Vaculik from Dofasco highlighted the potential risks associated with inadequately developed monitoring applications. He discussed the various barriers that can hinder the development process, including challenges related to data availability and quality, the technical expertise of staff members, and the acceptance of new technologies within the organization. Vaculik also pointed out the sometimes challenging decision companies face when deciding between purchasing pre-existing monitoring systems or developing their own solutions in-house.

Miletic et al. [631] from Dofasco and Tembec presented an industrial perspective on the online implementation of PCA and PLS techniques. The authors proposed a comprehensive method for developing monitoring systems, consisting of offline and online steps. The offline step involves data selection and model preparation, development, and evaluation. Despite being potentially overlooked by academic researchers, the authors emphasized that data selection is an expensive and vital process for the success of the application. The online stage focuses on system development, integration, and maintenance. The authors highlighted the importance of close collaboration with operators during this phase to understand their requirements and ensure a smooth interaction with the monitoring system. To illustrate the proposed methodology, the authors provided three application examples. First, they demonstrated the use of PCA for fault detection in a continuous casting machine. Second, they applied PLS to develop predictive models for a desulphurization process. Finally, they employed PCA for fault detection in a continuous digester. These practical examples emphasized the importance of a systematic approach to developing and implementing monitoring systems in real-world applications.

Kourti [632] presented a review of industrial monitoring applications utilizing multivariate statistical models. Interesting topics covered include dealing with future observations of missing data, adaptive models, monitoring batch processes, monitoring transitions, control using multivariate statistical models, image analysis and archiving, and data compression and reconstruction.

In the work conducted by Zhang and Dudzic [633], another implementation project at Dofasco was described. This particular implementation introduced a novel trajectory synchronization scheme for monitoring transition operations. Similar to previous publications from the company, the study provided insightful details regarding the implementation process, including the operation of alarms and control system screens.

Qin et al. [634] presented a hierarchical and comprehensive framework for the control and monitoring of new-generation semiconductor factories at that time. One of the notable findings highlighted in the study was the relatively greater significance of drifts compared to sensor faults in semiconductor processes.

Chiang and Colegrove [635] presented a comprehensive application of robust PCA within the industrial setting of Dow Chemical. The focus of their investigation was to effectively monitor the production process of solid epoxy resins, a specific product manufactured by the company. The authors provided valuable insights into practical considerations, including model performance, maintenance and transfer. Notably, they detailed the implementation of an online monitoring system, which soon after its deployment proved instrumental in detecting quality issues with the raw materials.

Kano and Nakagawa [636] presented significant advancements and practical implementations of data-based monitoring in the steel industry, drawing on examples from the operations of Sumitomo Metals Kokura. The researchers proposed a novel approach that effectively handled qualitative data, while also establishing connections between operational conditions and production yield. Notably, their methodology enabled the simultaneous analysis of multiple processing units, thereby offering a comprehensive perspective on the manufacturing process.

Miletic et al. [637] provided further insights and experiences about online applications at Dofasco. The authors specifically highlighted the development of a flexible computational framework that facilitated data manipulation and integration of new models. This framework enabled seamless incorporation of various techniques beyond PCA and PLS, such as Bayesian networks and image analysis techniques.

AlGhazzawi and Lennox [638] presented a study focusing on the development of a monitoring system for an oil refining process at Saudi Aramco. The primary objective of the study was to effectively monitor the non-stationary dynamics of the process. To achieve this, the researchers employed a recursive PCA method and also a multiblock approach for improved fault isolation. Notably, the authors emphasized the significance of incorporating the domain knowledge of plant personnel to ensure proper development and implementation of the monitoring system. It is worth mentioning that a real-time implementation of the system was not reported in this study.

Vanhatalo [639] provided a description of the application of the PCA technique for monitoring a pilot-scale blast furnace. The engineers overseeing the project acknowledged several advantages of this approach. Firstly, the application of PCA provided a comprehensive overview of the equipment’s thermal state, thereby enabling a standardized evaluation of its condition. Additionally, it facilitated the establishment of formal criteria for determining the appropriate timing of control actions. However, the engineers also highlighted a few drawbacks. They mentioned the high costs associated with manpower required for model development, maintenance, and interpretation. Moreover, they noted that the multivariate representations used in PCA introduced unnecessary levels of abstraction, potentially making it more challenging to interpret the results accurately.

Darkow et al. [640] detailed the utilization of the PCA method for early detection of blockages or obstructions in an industrial stripper. The authors identified several key factors contributing to the success of the application. Firstly, they emphasized the importance of implementing preprocessing steps prior to model generation, such as variable selection based on expert knowledge and removal of univariate outliers through visual inspection. Secondly, they highlighted the significance of involving both plant personnel and information technology experts from the project’s inception, ensuring a collaborative approach that incorporated domain expertise and technical knowledge. Lastly, the researchers underscored the development of a user-friendly graphical interface that catered to the needs of different users.

Dumarey et al. [641] presented an application of the PCA method for real-time monitoring of the continuous synthesis of an active pharmaceutical ingredient. The authors addressed the challenges surrounding the adoption of real-time monitoring methods within the pharmaceutical industry, highlighting the hurdles associated with integrating such methods into existing rigid quality control systems. Despite these challenges, the proposed application of PCA yielded significant benefits. It led to an enhanced understanding of the synthesis process, including a deeper comprehension of natural variability. Additionally, the monitoring system facilitated the identification of different fault types and their underlying causes.

Patwardhan et al. [642] discussed various applications of data science within the context of Saudi Aramco. Notably, some applications were related to process monitoring, including the assessment of controller performance and predictive maintenance of equipment. The authors emphasized the significance of utilizing data analysis techniques in effectively managing alarms, thereby enhancing the overall efficacy of monitoring systems when working in conjunction with operators.

Klanderman et al. [643] presented a case involving the application of the PCA technique to develop an adaptive monitoring methodology specifically designed for effluent treatment plants. The authors discussed several practical aspects related to this application. Firstly, they highlighted the enhanced understanding of process changes achieved by operators. However, it was noted that there could be instances where the results obtained from the monitoring system contradicted the operators’ assessments, even when the methods were functioning as expected. As a recommendation, the authors suggested that operators and specialists should review the training data beforehand. Additionally, it was advised to reset the monitoring system after detecting a fault in order to prevent contamination with non-normal data.

Kumar et al. [644] described the development and application of a replicable system that utilizes multivariate statistical models for monitoring reformers in hydrogen production plants. The authors emphasized the significance of regularly retraining the models to ensure their effectiveness. To achieve this, they proposed a methodology that incorporates user feedback regarding false positives.

Based on the descriptions provided in the preceding paragraphs, it is clear that the main publications documenting practical applications of monitoring techniques were published during the 2000s. Subsequently, there has been a significant increase in studies on process monitoring from the 2010s onwards, with a growing emphasis on the utilization of real-world data. However, most of these studies primarily focus on offline analysis of new techniques or variations in traditional techniques applied to such data, while paying limited attention to practical implementation aspects. Furthermore, despite the increasing use of real data, simulated data, particularly from the TEP benchmark, remains prevalent in most research works. It is also worth noting that the main articles describing practical implementations predominantly rely on multivariate statistical models, particularly PCA and PLS, indicating a limited adoption of alternative techniques in this field.

Regarding recent studies involving offline analysis of real historical data, there are some noteworthy works that deserve highlighting, such as: the development of a methodology for detecting the decomposition of reagents in a copolymerization autoclave using PCA [645]; the application of machine learning methods for decentralized monitoring of an effluent treatment plant [646]; the application of PCA to monitor the physicochemical properties of crude oil blends [647]; the application of CCA to historical process data for detection and diagnosis of a sensor fault in a fiscal metering station of an oil processing plant, highlighting the potential for economic gains which may result in such a real-time application [648]; the application of different latent variable models to develop a strategy aimed at an increasing understanding of biorefineries processes [649]; the extension of recursive PCA with big data methodologies for monitoring a fluorochemical plant [650]; the development of a virtual sensor based on machine learning to predict faults in a metallurgical process [651]; the development of a system that combines various techniques such as recurrent neural networks and PCA to predict abnormal conditions in catalytic cracking processes [652]; the application of neural network-based nonlinear PCA for fault detection in gas turbines [653]; and the application of autoencoders for unsupervised monitoring of blast furnaces [405].

8. Perspectives and Challenges

The main perspectives and challenges in process monitoring research for the coming years are described in this section. These challenges can be broadly categorized into three key areas: system performance, significance/usefulness, and development environment. We believe that these categories encompass the primary obstacles encountered by researchers and practitioners in the field of process monitoring. Figure 9 shows a schematic illustration of the perspectives described along the section.

8.1. Data Characteristics

The process monitoring research community primarily focuses on addressing challenges posed by specific data characteristics. Many studies have presented inventive approaches or modifications to existing methods, with a focus on specific data traits like dynamic behavior, nonlinearity, and non-Gaussianity. Several of these studies have already been discussed in the preceding sections of the present article. However, while it is important to introduce new techniques, we feel that the research community often overemphasizes this aspect, potentially overshadowing other critical challenges that also demand attention. The following subsections highlight these additional challenges.

8.2. Hybrid Modeling

The fusion of diverse data-driven methods, or the integration with other process monitoring approaches such as knowledge-based or phenomenological methods, has been widely discussed as a promising future direction in various review and perspective articles [15,165,654,655,656,657,658,659,660,661]. Md Nor et al. [15] conducted a comprehensive review of hybrid methodologies, emphasizing the necessity for such approaches due to the inability of individual methods to handle all potential faults that a process may encounter. Bi et al. [654] emphasized the challenges associated with integrating different methods, particularly in terms of leveraging the strengths of each methodology and establishing a robust decision-making process. Severson et al. [165] assert that hybrid approaches hold the most promise for addressing complex scenarios within the context of process monitoring. Sansana et al. [660] drew interesting conclusions from their literature review, for example, identifying that data-driven models have been used to enhance phenomenological models, but the reverse path of using phenomenological models to improve data-driven models is not yet fully explored. Bradley et al. [659] conducted an extensive review focusing on methodologies that integrate data-driven and phenomenological approaches, categorizing them into three key areas: hybrid modeling, physics-informed machine learning, and model calibration. Hybrid modeling involves constructing models that combine different phenomenological and data-driven submodels. Physics-informed machine learning entails incorporating physical knowledge into the structure of data-driven models. Model calibration refers to using data-driven models to adjust computer simulations and align them with experimental data. The authors provide a comprehensive comparison of these methodologies, emphasizing their respective advantages and limitations.

In addition to enhancing performance, the utilization of mechanistic models holds significant importance in improving model interpretability, a topic that is further discussed in the next subsection.

8.3. Model Interpretability

The importance of model interpretability as a research subject for the future has been recognized in various surveys [16,654,662,663,664,665]. Yan et al. [662] specifically highlighted the challenge of developing a generalized methodology to incorporate interpretability into deep learning methods. They emphasized the significance of achieving this objective to create optimal diagnostic models while acknowledging the trade-off between model interpretability and performance. Bi et al. [654] acknowledged the need for incorporating both human and machine intelligence to develop smart systems, emphasizing the role of interpretability to make humans trust in machine intelligence. The authors identified three approaches to achieve interpretability: intrinsic interpretable models, such as Bayesian neural networks [533]; enhancing black-box models, like deep neural networks, with interpretability capabilities [327,328,666,667,668]; and employing model-agnostic methods [390,669,670] such as LIME (local interpretable model-agnostic explanations) [671] and SHAP (Shapley additive explanations) [672]. Several recent publications have surveyed interpretability techniques for machine learning models [673,674,675,676,677,678,679]. In particular, Carter et al. [679] specifically examined applications in the process industries, focusing on risk management and standardization issues.

Among recent research on model interpretability, it is worth highlighting the works of: Sivaram et al. [327] and Das et al. [328], who investigated hidden representations in neural networks applied to classification and regression tasks, respectively; Harinarayan and Shalinie [680], who utilized the SHAP method to explain the results of a XGBoost [681] model applied to the TEP benchmark; Yang et al. [533], who developed an unsupervised Bayesian network model for monitoring processes at both global and local levels; Bhakte et al. [682], who proposed a methodology using alarm limits for explaining process monitoring results from deep learning models; Ye et al. [683], who combined the use of frequency spectra inputs with a layer-wise relevance propagation strategy to explain predictions of convolutional neural networks; and Melo et al. [569], who proposed a visual and interpretable methodology based on distance matrix patterns.

8.4. Data Patterns over Model Structures

New models that prioritize intrinsic patterns in data rather than specific mathematical structures and hypotheses are increasingly being utilized in process monitoring research, as discussed in Section 3.4. This approach has been extensively discussed by Angelov and Gu [556], who highlighted the shift from an assumptions-centered approach to a data-centered approach. According to the authors, this transition offers several advantages, including enhanced autonomy in the modeling process, adaptability to dynamic changes, and improved interpretability. We believe that the process monitoring community would greatly benefit from more research conducted with this mindset, as it can help address the challenges that arise when model assumptions do not align with the specific characteristics of process data.

8.5. Real-World Implementations

The process monitoring literature is dominated by articles focusing on proposing new mathematical models, while the scarcity of studies reporting real-world implementations is evident. Additionally, as discussed in Section 7, most of these implementations revolve around multivariate statistical models, particularly the PCA model, which suggests the existence of unexplored opportunities concerning the practical application of more recently developed techniques such as deep neural networks.

Unfortunately, many researchers lack access to real industrial facilities that enable the study of practical implementations, thereby aggravating the challenge of bridging the gap between industries and academia. In this context, Bi et al. [654] highlights the criticality of high-quality data availability over model complexity, despite the research community’s predominant emphasis on the latter. To mitigate this issue, one potential solution could be to use the real-world dataset benchmarks proposed in recent years more frequently [5]. This matter is further discussed in the next subsection.

8.6. Benchmark Diversity and Use

In recent years, several new benchmarks for process monitoring have been proposed, as discussed in Section 6.2. However, despite these alternatives, the research community continues to predominantly rely on the well-established TEP benchmark [5]. This raises an important question: why does the TEP benchmark remain the preferred choice, even though the underlying problem it represents has already been addressed by solutions proposed over more than two decades of research?

One potential explanation for the continued popularity of TEP lies in its familiarity and ease of use. Researchers and practitioners have become accustomed to working with this benchmark, making it a comfortable and convenient choice for assessing new models and techniques. Furthermore, the extensive documentation and well-established methodologies contribute to its widespread adoption. However, it is important to critically evaluate whether this reliance on TEP hinders progress in the field. For instance, a significant portion of the literature primarily focuses on achieving marginal performance improvements in fault detection and diagnosis metrics. The proliferation of new models that offer only incremental enhancements may not truly benefit the research community and industry. Moreover, it is important to acknowledge that the TEP benchmark is based on simulated data, which presents significantly fewer challenges compared to real-world data.

Therefore, it may be more beneficial to explore alternatives that introduce challenges beyond those extensively investigated with the TEP benchmark. By shifting attention to new benchmarks, the research community can study different aspects of process monitoring, thereby expanding the field’s knowledge in a more comprehensive manner. The alternative benchmarks encompass various data characteristics, such as nonlinearity, dynamics, and non-Gaussianity, which tend to be more pronounced compared to TEP. Additionally, real-world data substantially differ from simulated data, particularly in terms of data quality and variability. These are important aspects that should be addressed in future research through the utilization and development of more diverse benchmarks. For a more thorough discussion, the reader is referred to Melo et al. [5].

8.7. Open Development and Collaboration

Open development greatly accelerates scientific progress by promoting collaboration among researchers, facilitating synergistic contributions, and avoiding redundant work, among other advantages. In the field of PSE, a particular benefit is the potential to bridge the gap between academia and industry. For example, when implementing process monitoring procedures, companies heavily rely on commercial software solutions that often have limited connections to the cutting-edge methodologies developed in academia [582].

By embracing open development practices, universities and research centers can create open-source tools accessible to both academics and industry professionals. This can lead to the utilization of more advanced techniques in industrial applications, driving innovation, and enhancing the practicality of academic research. This phenomenon is already observed in domains such as data science and machine learning, where major corporations like Google and Facebook actively promote the development of cutting-edge and widely used tools such as Keras [684] and PyTorch [685].

An example of institutional support for open development in the process industries is Petrobras’ Connections for Innovation program. This program aims to integrate the company with the open innovation ecosystem. One of this program’s components, the Open Lab Module, is particularly relevant in the present context as it promotes collaboration to address challenges in the oil, gas, and energy sector through open-source projects. The 3W Project, mentioned in Section 6.2, is an example of an initiative within the Open Lab Module.

Specifically in the domain of PSE, Matlab has historically been the dominant programming language for developing innovative methodologies [686]. However, Matlab’s proprietary nature, closed-source code, and licensing requirements make it less suitable for the open innovation paradigm. Therefore, we believe that transitioning to open-source tools like Python would be advantageous for the PSE community. This shift would not only facilitate the adoption of research findings by the industrial sector but also promote cross-domain integration, a topic discussed in the following subsection.

8.8. Cross-Domain Integration

In the field of PSE, the term process monitoring has been often used interchangeably with the term fault detection and diagnosis [20]. However, in other domains, such as signal processing and medical data analysis, the same concept is known by different names, such as anomaly detection [687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709], novelty detection [710,711,712,713,714,715,716,717,718,719,720,721,722,723,724] and change-point detection [725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741]. Although these three terms refer to variations in the same fundamental problem (at least from a computational and mathematical perspective), there is a noticeable lack of information exchange between the communities focused on these studies and the process system engineering community. This subsection highlights the potential opportunities for fostering interaction between these groups.

The differences in nomenclature contribute, at least partially, to the limited communication aforementioned. Moreover, the relative isolation can be attributed to the nature of the analyzed data and the methodologies utilized to address these problems. For instance, in the field of disease diagnosis from medical data, methods are typically developed for batch analyses [729], since the detection of a disease (anomaly) most often does not require the real-time and online analysis of a sequence while it is being generated. Moreover, medical data is often univariate, involving variables such as heartbeats [736]. These two conditions are not typical of PSE problems. Furthermore, process data exhibit specific characteristics, in particular high redundancy and collinearity between variables [24].

One could argue that this premise holds partially true, as the nature of problems in different domains has not undergone significant changes. However, with the disruptive phase that all domains are undergoing, particularly the industrial sector with the advent of Industry 4.0, the almost complete segregation of their literature on the same topic is becoming increasingly questionable. It is highly improbable that no potential for a productive exchange of ideas exists, even if the specifics of solution methods differ. The argument is also becoming increasingly unrealistic: works such as the one conducted by Alippi et al. [726], which examines the theoretical limitations of change detection methods applied to continuously generated multidimensional data, clearly share scope with PSE problems.

It is worth noting that other fields in industry and engineering employ similar terminology to address the monitoring problem, such as the term “fault detection and diagnosis”. While communication between the PSE domain and these fields is generally better, we believe that it remains unsatisfactory. Notable examples of such fields include mechanical engineering [742,743,744], nuclear engineering [745,746], electrical engineering [747,748], and mechatronics [749,750].

In the realm of data science, the analysis of continuously generated data is commonly referred to as streaming data [751,752,753]. In this context, “concept drift” is a term used by data scientists to describe the evolution over time of patterns and relationships captured by machine learning techniques [754,755,756,757,758,759]. This idea could be of significant value for PSE, as process drift constitutes one of the most important process states and can usually be associated with process malfunctions [760]. This nomenclature has recently been adopted by research in the PSE field [761,762]. A current trend is the utilization of ensembles of methods to enhance efficiency in adapting to data evolution [763,764,765,766]. The streaming data literature offers potentially valuable works on clustering [767], pre-processing [768], outlier treatment [769,770], and event prediction [771], although we were unable to identify mutual references in the analyzed literature.

Causality analysis is an example of an application based on a time series that has gained prominence in the PSE field, as evidenced by numerous studies [580,772,773,774,775,776,777,778,779,780,781,782]. This technique uses statistical tests, such as the Granger causality test, to determine whether a given time series is useful in predicting another. Despite the significant advances made in the PSE literature, there seems to be a lack of attention from other fields towards causality-related works in PSE. This is particularly evident in recent reviews on causality conducted by Shojaie and Fox [783] and Guo et al. [784], as they do not reference any recent PSE studies.

Nonlinear dynamic analysis, with its rich history in both PSE [785,786,787,788,789,790] and time series analysis [791,792,793,794], holds immense potential for further applications in the evolving era of Industry 4.0. For instance, Aldrich [81] describes methods based on phase spaces, such as correlation dimension, Lyapunov exponents, and information entropy, emphasizing that adoption of these methods in industry is still limited due to the impact of noise on the quality of the descriptions. However, the fact is that in the domain of process monitoring, in particular, the potential use of nonlinear tools for the representation and evaluation of process states remains largely unexplored. As a matter of fact, the nonlinear characteristics of a time series can provide information about the process health, indicate possible process changes and suggest the use of different numerical tools for qualitative and quantitative description of the process behavior [81].

9. Conclusions

This paper presented a comprehensive overview of research on data-driven process monitoring in the PSE literature. Process monitoring has become increasingly important in the context of digital transformation, attracting attention from both academia and industry. It was observed that a large part of academic research is dedicated to developing data-driven models that can handle specific characteristics of process data, such as dynamic and nonlinear behavior. Furthermore, there is a recent trend toward exploring innovative machine learning methodologies. However, industrial applications still rely mainly on conventional statistical techniques, especially the PCA model. This reveals a gap between academic and industrial practices, but also an opportunity for transferring novel concepts and techniques from academia to industry. Furthermore, additional challenges and directions for future research were identified, such as assessing the relevance and applicability of academic proposals in real-world settings and fostering a collaborative and supportive environment among researchers and practitioners that enables the adoption and implementation of research results in practice.