# Capturing Causality for Fault Diagnosis Based on Multi-Valued Alarm Series Using Transfer Entropy

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Alarm Series and Its Extended Form

#### 2.1. Binary Alarm Series

_{i}configured by time series A and B are shown in Figure 2. It is easy to find that the time series that are completely different from each other become similar to the original time series. The max CCF value that is calculated from B to A by using the binary alarm series is 0.24580 and the corresponding lag is −71. It is obvious that the conclusion based on the binary alarm series x

_{i}is incorrect.

#### 2.2. Multi-Valued Alarm Series

**S**(i = 1, 2, 3, …, n), which are the different states of a multi-valued alarm series. Hence, the causality between variables or equipment or systems via multi-valued alarm series can be detected.

_{i}## 3. Transfer Entropy and Mutual Information

#### 3.1. Transfer Entropy

**X**= [x

_{1}, x

_{2}, x

_{3}, …, x

_{t}, …, x

_{n}]’ to

**Y**= [y

_{1}, y

_{2}, y

_{3}, …, y

_{t}, …, y

_{n}]’ is defined as

_{t}and y

_{t}denote the values of the variables

**X**and

**Y**, respectively, at time t; k and l denote the orders of the cause variable and effect variable, respectively; x

_{t}and its l-length past are defined as ${\mathit{x}}_{t}^{\left(l\right)}=\left[{x}_{t},{x}_{t-\tau},\dots ,{x}_{t-\left(l-1\right)\tau}\right]$; y

_{t}and its past are defined as ${\mathit{y}}_{t}^{\left(k\right)}=\left[{y}_{t},{y}_{t-\tau},\dots ,{y}_{t-\left(k-1\right)\tau}\right]$; τ is the sampling period; h denotes the prediction horizon; ω is the random vector $\left[{y}_{t+h},{\mathit{y}}_{t}^{\left(k\right)},{\mathit{x}}_{t}^{\left(l\right)}\right]$; and f denotes the complete or conditional PDF. In this TE method, the PDF can be estimated by kernel methods or a histogram [23], which are nonparametric approaches, to fit any shape of the distributions.

**X**to

**Y**, then it is helpful to predict

**Y**via the historical data of

**X**. In other words, the information of

**Y**can be obtained from the historical values of

**Y**and

**X**. Then, $\frac{f\left({y}_{t+h}|{\mathit{y}}_{t}^{\left(k\right)},{\mathit{x}}_{t}^{\left(l\right)}\right)}{f\left({y}_{t+h}|{\mathit{y}}_{t}^{\left(k\right)}\right)}>1$ and the TE should be positive. Otherwise, it should be close to zero.

**Y**and

**X**are two continuous time series. So, Equation (3) is not suitable for discrete time series. Then, a discrete transfer entropy (TE

_{disc}) from

**X**to

**Y**, for discrete time series, is estimated as follows:

#### 3.2. Conditional Mutual Information

#### 3.2.1. Mutual Information

**X**= [x

_{1}, x

_{2}, x

_{3}, …, x

_{n}]’ and

**Y**= [y

_{1}, y

_{2}, y

_{3}, …, y

_{n}]’ is defined as [25]

_{t}and y

_{t}denote the value of variables

**X**and

**Y**, respectively, at time t; $p\left(x,y\right)$ denotes the joint PDFs of

**X**and

**Y**; and $p\left(x\right)$ and $p\left(y\right)$ are the marginal PDFs of

**X**and

**Y**, respectively. The summation is on the feasible space of x and y and will be omitted hereinafter for brevity.

**Y**and the historical data of

**X**, the modified MI method is defined as

#### 3.2.2. Conditional Mutual Information

**X**= [x

_{1}, x

_{2}, x

_{3}, …, x

_{n}]’ and

**Y**= [y

_{1}, y

_{2}, y

_{3}, …, y

_{n}]’ conditioned on

**Z**= [z

_{1}, z

_{2}, z

_{3}, …, z

_{n}]’ is defined as [25]

_{t,}y

_{t}

_{,}and z

_{t}denote the value of the variables

**X**,

**Y**, and

**Z**respectively at time t; $p\left(x,y,z\right)$ denotes the joint PDF of

**X**,

**Y**, and

**Z**; and $p\left(x,y|z\right)$ denotes the joint PDF of

**X**and

**Y**given

**Z**.$p\left(x|z\right)$ and $p\left(y|z\right)$ are the PDF of

**X**given

**Z**and the PDF of

**Y**given

**Z**, respectively.

**Y**= y

_{t}

_{+h}, $\mathit{X}={\mathit{x}}_{t}^{\left(l\right)}$, and $\mathit{Z}={\mathit{y}}_{t}^{\left(k\right)}$, the CMI formula in Equation (7) is expressed as Equation (8). By doing so, the CMI is equivalent to the TE formula in Equation (3). Therefore, the TE formula is a special case of Equation (7) [26].

**X**and

**Y**given variable

**Z**is also a special case of CMI, and is formulated as [16]

**Y**to

**X**; and h2 denotes the prediction horizon from

**Z**to

**X**.

## 4. Detection of Direct Causality via Multi-Valued Alarm Series

#### 4.1. Detection of Causality via TE

#### 4.2. Detection of Direct Causality via CMI

## 5. Significance Test

**X**and

**Y**are independent of each other, then $p\left({\tilde{y}}_{t+h}|{\tilde{\mathit{y}}}_{t}^{\left(k\right)},{\tilde{\mathit{x}}}_{t}^{\left(l\right)}\right)=p({\tilde{y}}_{t+h}\left|{\tilde{\mathit{y}}}_{t}^{\left(k\right)}\right)$. So, the value of TE should be zero in an ideal state, implying that there is no causality between process variables

**X**and

**Y**. However, in an actual industrial process, variables without causality are not completely independent due to noise or other disturbance. So, the value of TE between them maybe is not zero. Similarly, if the connectivity between variables

**X**and

**Y**is indirect given variable

**Z**, then

**X**and

**Y**are independent given

**Z**and $p\left(x,y|z\right)$=$p\left(x|z\right)*p\left(y|z\right)$. So, the value of CMI should be zero in an ideal state. However, in practice, like TE, the value of CMI maybe is not zero. For this reason, a significant threshold needs to be identified to determine whether the values of TE or CMI are significant. In order to obtain a threshold, the Monte Carlo method with surrogate data is adopted [28].

_{T}denotes the threshold for TE), it indicates a significant causality from variable

**X**to

**Y**.

_{T}and the number of Monte Carlo simulations (N), another test with N = 1, 2, ···, 100 is applied, as shown in Figure 8. It is easy to see that when N > 40, the calculated θ

_{T}is close to the actual threshold.

_{I}denotes the threshold for modified CMI), a direct causality from

**X**to

**Y**based on

**Z**can be detected. Therefore, the process topology is obtained. To test the effectiveness of the proposed method, a numerical case and a simulated industrial case are given below.

## 6. Case Studies

#### 6.1. Numerical Example

#### 6.2. Industrial Example

## 7. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Yang, F.; Duan, P.; Shah, S.L.; Chen, T.W. Capturing Causality from Process Data. In Capturing Connectivity and Causality in Complex Industrial Processes; Springer: New York, NY, USA, 2014; pp. 57–62. [Google Scholar]
- Khandekar, S.; Muralidhar, K. Springerbriefs in applied sciences and technology. In Dropwise Condensation on Inclined Textured Surfaces; Springer: New York, NY, USA, 2014; pp. 17–72. [Google Scholar]
- Pant, G.B.; Rupa Kumar, K. Climates of south asia. Geogr. J.
**1998**, 164, 97–98. [Google Scholar] - Donges, J.F.; Zou, Y.; Marwan, N.; Kurths, J. Complex networks in climate dynamics. Eur. Phys. J. Spec. Top.
**2009**, 174, 157–179. [Google Scholar] [CrossRef] - Hiemstra, C.; Jones, J.D. Testing for linear and nonlinear granger causality in the stock price-volume relation. J. Financ.
**1994**, 49, 1639–1664. [Google Scholar] - Wang, W.X.; Yang, R.; Lai, Y.C.; Kovanis, V.; Grebogi, C. Predicting catastrophes in nonlinear dynamical systems by compressive sensing. Phys. Rev. Lett.
**2011**, 106, 154101. [Google Scholar] [CrossRef] [PubMed] - Hlavackova-Schindler, K.; Palus, M.; Vejmelka, M.; Bhattacharya, J. Causality detection based on information. Phys. Rep.
**2007**, 441. [Google Scholar] [CrossRef] - Duggento, A.; Bianciardi, M.; Passamonti, L.; Wald, L.L.; Guerrisi, M.; Barbieri, R.; Toschi, N. Globally conditioned granger causality in brain-brain and brain-heart interactions: A combined heart rate variability/ultra-high-field (7 t) functional magnetic resonance imaging study. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci.
**2016**, 374. [Google Scholar] [CrossRef] [PubMed] - Liu, Y.-K.; Wu, G.-H.; Xie, C.-L.; Duan, Z.-Y.; Peng, M.-J.; Li, M.-K. A fault diagnosis method based on signed directed graph and matrix for nuclear power plants. Nucl. Eng. Des.
**2016**, 297, 166–174. [Google Scholar] [CrossRef] - Maurya, M.R.; Rengaswamy, R.; Venkatasubramanian, V. A systematic framework for the development and analysis of signed digraphs for chemical processes. 2. Control loops and flowsheet analysis. Ind. Eng. Chem. Res.
**2003**, 42, 4811–4827. [Google Scholar] [CrossRef] - Bauer, M.; Thornhill, N.F. A practical method for identifying the propagation path of plant-wide disturbances. J. Process Control
**2008**, 18, 707–719. [Google Scholar] [CrossRef] - Chen, W.; Larrabee, B.R.; Ovsyannikova, I.G.; Kennedy, R.B.; Haralambieva, I.H.; Poland, G.A.; Schaid, D.J. Fine mapping causal variants with an approximate bayesian method using marginal test statistics. Genetics
**2015**, 200, 719–736. [Google Scholar] [CrossRef] [PubMed] - Ghysels, E.; Hill, J.B.; Motegi, K. Testing for granger causality with mixed frequency data. J. Econom.
**2016**, 192, 207–230. [Google Scholar] [CrossRef] - Yang, F.; Fan, N.J.; Ye, H. Application of PDC method in causality analysis of chemical process variables. J. Tsinghua Univ. (Sci. Technol.)
**2013**, 210–214. [Google Scholar] - Bauer, M.; Cox, J.W.; Caveness, M.H.; Downs, J.J. Finding the direction of disturbance propagation in a chemical process using transfer entropy. IEEE Trans. Control Syst. Technol.
**2007**, 15, 12–21. [Google Scholar] [CrossRef] [Green Version] - Duan, P.; Yang, F.; Chen, T.; Shah, S.L. Direct causality detection via the transfer entropy approach. IEEE Trans. Control Syst. Technol.
**2013**, 21, 2052–2066. [Google Scholar] [CrossRef] - Duan, P.; Yang, F.; Shah, S.L.; Chen, T. Transfer zero-entropy and its application for capturing cause and effect relationship between variables. IEEE Trans. Control Syst. Technol.
**2015**, 23, 855–867. [Google Scholar] [CrossRef] - Staniek, M.; Lehnertz, K. Symbolic transfer entropy. Phys. Rev. Lett.
**2008**, 100, 158101. [Google Scholar] [CrossRef] [PubMed] - Yu, W.; Yang, F. Detection of causality between process variables based on industrial alarm data using transfer entropy. Entropy
**2015**, 17, 5868–5887. [Google Scholar] [CrossRef] - Yang, Z.; Wang, J.; Chen, T. Detection of correlated alarms based on similarity coefficients of binary data. IEEE Trans. Autom. Sci. Eng.
**2013**, 10, 1014–1025. [Google Scholar] [CrossRef] - Yang, F.; Shah, S.L.; Xiao, D.; Chen, T. Improved correlation analysis and visualization of industrial alarm data. ISA Trans.
**2012**, 51, 499–506. [Google Scholar] [CrossRef] [PubMed] - Schreiber, T. Measuring information transfer. Phys. Rev. Lett.
**2000**, 85, 461–464. [Google Scholar] [CrossRef] [PubMed] - Dehnad, K. Density Estimation for Statistics and Data Analysis by Bernard Silverman; Chapman and Hall: London, UK, 1986; pp. 296–297. [Google Scholar]
- Cover, T. Information Theory and Statistics. In Proceedings of the IEEE-IMS Workshop on Information Theory and Statistics, Alexandria, VA, USA, 27–29 October 1994; p. 2. [Google Scholar]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; Tsinghua University Press: Beijing, China, 2003; pp. 1600–, 1601. [Google Scholar]
- Palus, M.; Komárek, V.; Hrncír, Z.; Sterbová, K. Synchronization as adjustment of information rates: Detection from bivariate time series. Phys. Rev. E Stat. Nonlinear Soft Matter Phys.
**2001**, 63, 046211. [Google Scholar] [CrossRef] [PubMed] - Runge, J.; Heitzig, J.; Petoukhov, V.; Kurths, J. Escaping the curse of dimensionality in estimating multivariate transfer entropy. Phys. Rev. Lett.
**2012**, 108, 258701. [Google Scholar] [CrossRef] [PubMed] - Kantz, H.; Schreiber, T. Nonlinear Time Series Analysis; Cambridge University Press: Cambridge, UK, 1997; p. 491. [Google Scholar]
- Downs, J.J.; Vogel, E.F. A plant-wide industrial process control problem. Comput. Chem. Eng.
**1993**, 17, 245–255. [Google Scholar] [CrossRef] - Ricker, N.L. Decentralized control of the tennessee eastman challenge process. J. Process Control
**1996**, 6, 205–221. [Google Scholar] [CrossRef]

**Figure 1.**The time series A (the upper) and B. The red lines are the upper limits (H) of the normal states.

**Figure 2.**The binary alarm series A (the red line) and B (the blue line) according to H. (“false” for a fault state while “true” for a normal state).

**Figure 3.**Four types of alarms of MEAS_MON.

**U_AH**denotes the high alarm limit (the threshold is chosen as 100);

**U_AL**denotes the low alarm limit (the threshold is chosen as 0);

**U_WH**denotes the high warning limit (the threshold is chosen as 95); and

**U_WL**denotes the high alarm limit (the threshold is chosen as 5).

**Figure 4.**Multi-valued alarm series. (

**a**) shows the four alarm series (HI, LO, HH, LL) of A; (

**b**) shows the four alarm series (HI, LO, HH, LL) of B; (

**c**) shows the two multi-valued alarm series that is a combination of the four alarm series shown in Figure 4a,b, respectively. The multi-valued alarm series of A has three states and B has five states. HH: high alarm; HI: high warning; LO: low warning; LL: low alarm.

**Figure 5.**Transfer entropy (TE) (red line) and modified mutual information (MI) (blue line) with different h.

**Figure 6.**TEs between variables A, B, C, D in Example 1 (to be analyzed later). The gray color indicates a true direct causality. The red color indicates an indirect causality.

**Figure 7.**N-trial Monte Carlo simulations. (

**a**) TEs between variables stream 4 and stream 10; (

**b**) TEs between all variables calculated from the surrogate data follow standard deviation.

**Figure 8.**The relationship between θ and N. (

**a**) an example of 100 pairs of uncorrelated random series of a couple of variables; (

**b**) an example of 100 pairs of uncorrelated random series of every couple of variables.

**Figure 10.**Process data with thresholds. The most of states of each sequence is the normal state. It coincides with the real alarm situation very well. HHH: high error; LLL: low error.

**Figure 11.**Multi-valued alarm series: A1, B1, C1, and D1. The 0 denotes a normal state; 3 denotes high error (HHH); −3 denotes low error (LLL); −2 denotes HH; −2 denotes LL; 1 denotes HI; and −1 denotes LO.

**Figure 12.**(Color online) Causality map of all sub-processes. The red link denotes an indirect relationship.

**Figure 14.**Tennessee–Eastman Process and its control scheme [30]. The chosen variables are represented by red circles.

CCF | Original Time Series | Binary Alarm Series | Multi-Valued Alarm Series |
---|---|---|---|

value | 0.15080 | 0.24580 | 0.14663 |

Lag | 245 | −71 | 245 |

**Table 2.**The TEs based on Example 1 by using multi-valued alarm series and their corresponding significance thresholds (in brackets).

To A | To B | To C | To D | |
---|---|---|---|---|

From A | - | 0.041(0.036) | 0.016(0.025) | 0.159(0.046) |

From B | 0.031(0.035) | - | 0.021(0.026) | 0.175(0.049) |

From C | 0.021(0.029) | 0.023(0.030) | - | 0.055(0.041) |

From D | 0.028(0.037) | 0.035(0.045) | 0.027(0.030) |

**Table 3.**TEs based on the Tennessee–Eastman Process example by using multi-valued alarm series and their corresponding significance thresholds (in brackets).

To Stream 4 | To Stream 10 | To Level | To Stream 11 | |
---|---|---|---|---|

From stream 4 | 0.456(0.027) | 0.290(0.025) | 0.182(0.019) | |

From stream 10 | 0.379(0.028) | 0.423(0.021) | 0.257(0.016) | |

From level | 0.028(0.027) | 0.289(0.022) | 0.139(0.015) | |

From stream 11 | 0.152(0.023) | 0.030(0.019) | 0.035(0.018) |

**Table 4.**Conditional mutual information (CMIs) and their corresponding significance thresholds (in brackets).

Condition | To Stream 4 | |
---|---|---|

From stream 11 | Stream 4 and level | 0.026(0.036) |

From stream 11 | Stream 10 and level | 0.016(0.038) |

From stream 10 | Stream 4 | 0.119(0.020) |

From stream 10 | Stream 4 and stream 11 | 0.027(0.037) |

From stream 4 | level | 0.018(0.021) |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Su, J.; Wang, D.; Zhang, Y.; Yang, F.; Zhao, Y.; Pang, X.
Capturing Causality for Fault Diagnosis Based on Multi-Valued Alarm Series Using Transfer Entropy. *Entropy* **2017**, *19*, 663.
https://doi.org/10.3390/e19120663

**AMA Style**

Su J, Wang D, Zhang Y, Yang F, Zhao Y, Pang X.
Capturing Causality for Fault Diagnosis Based on Multi-Valued Alarm Series Using Transfer Entropy. *Entropy*. 2017; 19(12):663.
https://doi.org/10.3390/e19120663

**Chicago/Turabian Style**

Su, Jianjun, Dezheng Wang, Yinong Zhang, Fan Yang, Yan Zhao, and Xiangkun Pang.
2017. "Capturing Causality for Fault Diagnosis Based on Multi-Valued Alarm Series Using Transfer Entropy" *Entropy* 19, no. 12: 663.
https://doi.org/10.3390/e19120663