# On the Potential of Time Delay Neural Networks to Detect Indirect Coupling between Time Series

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction to Indirect Coupling between Time Series

## 2. Time Delay Neural Networks and Coupled Lorenz Systems

## 3. Results of Coupling Detection

## 4. Time Localization of the Mutual Influence

_{all}indicates the total root square error in the prediction when all the signals are used as inputs, and Err

_{wt}the total error when one of the time series has been deselected from the input list (there are therefore three different Err

_{wt}). A time series is considered as influencing another when:

_{threshold}is the Z-score, set at a value of two for the cases reported in the following. The plots in Figure 11 show the results obtained for one of the most important cases analyzed (case 4).

_{32}have been shifted abruptly from 0 to 1 at every time slice, in which Z

_{2}is higher than 10 (see Figure 11 top plot on the right). The absolute error follows the same trend of the coupling coefficient, and the mutual influence is detected in the right intervals (again see Figure 11 top plots). The mutual coupling between Y and X has been kept constant at a value of 1. This situation allows detecting when Y has a sufficient amplitude to really influence X. Indeed, it can be seen from the absolute errors that when Y has a minuscule amplitude, it cannot exert any influence on X, even if the coupling coefficient is 1. Therefore, the oscillations in the bottom right plot of Figure 11 accurately reflect the actual evolution of the real mutual influence between the systems.

## 5. First Analysis of Noise Effects

_{noise}has been scanned over a wide range. Table 8 shows how the TDNNs properly determined the causal relationships between the three systems X, Y, and Z, except for the cases in which the noise becomes excessive. Basically, provided the signal of noise ratio is higher than 2, the TDNNs always correctly identify the causal relationships between the time series. A more systematic analysis of the noise influence will have to be carried out, but the first indications are very encouraging, and there is no reason to expect that they will not be confirmed in the future.

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Pearl, J.; Mackenzie, D. The Book of Why: The New Science of Cause and Effect; Pengiun Books: London, UK, 2018. [Google Scholar]
- Granger, C.W.J. Investigating causal relations by econometric models and cross-spectral methods. Econometrica
**1969**, 37, 424. [Google Scholar] [CrossRef] - Schreiber, T. Measuring information transfer. Phys. Rev. Lett.
**2000**, 85, 461–464. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Marwan, N.; Romano, M.C.; Thiel, M.; Kurths, J. Recurrence plots for the analysis of complex systems. Phys. Rep.
**2007**, 438, 237–329. [Google Scholar] [CrossRef] - Sugihara, G.; May, R.; Ye, H.; Hsieh, C.-H.; Deyle, E.; Fogarty, M.; Munch, S. Detecting causality in complex ecosystems. Science
**2012**, 338, 496–500. [Google Scholar] [CrossRef] [PubMed] - Krakovska, A.; Jakubík, J.; Chvosteková, M.; Coufal, D.; Jajcay, N.; Paluš, M. Comparison of six methods for the detection of causality in bivariate time series. Phys. Rev. E
**2018**, 97, 042207. [Google Scholar] [CrossRef] [PubMed] - Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Wiener, N. The Theory of Prediction Modern Mathematics for Engineers; Beckenbach, E., Ed.; McGraw-Hill: New York, NY, USA, 1965. [Google Scholar]
- Zou, Y.; Romano, M.C.; Thiel, M.; Marwan, N.; Kurths, J. Inferring indirect coupling by means of recurrences. Int. J. Bifurc. Chaos
**2011**, 21, 4. [Google Scholar] [CrossRef] [Green Version] - Waibel, A.; Hanazawa, H.; Hinton, G.; Shikano, K.; Lang, K.J. Phoneme recognition using time-delay neural networks. IEEE Trans. Acoust. Speech Signal Process.
**1989**, 37, 328–339. [Google Scholar] [CrossRef] - Carlson, R.R.; Lomax, R.; Freed, M.N.; Ryan, J.M.; Hess, R.K. Statistical concepts: A second course for education and the behavioral sciences. Am. Stat.
**1993**, 47, 308. [Google Scholar] [CrossRef] - Chandra, R.; Ong, Y.S.; Goh, C.K. Co-evolutionary multi-task learning for dynamic time series prediction. Appl. Soft Comput.
**2018**, 70, 576–589. [Google Scholar] [CrossRef] [Green Version] - Chandra, R.; Jain, K.; Deo, R.V.; Cripps, S. Langevin-gradient parallel tempering for Bayesian neural learning. Neurocomputing
**2019**, 359, 315–326. [Google Scholar] [CrossRef] [Green Version] - Romanelli, F.; Kamendje, R. Overview of JET results. Nucl. Fusion
**2009**, 49, 104006. [Google Scholar] [CrossRef] - Ongena, J.; Monier-Garbet, P.; Suttrop, W.; Andrew, P.; Becoulet, M.; Budny, R.; Corre, Y.; Cordey, G.; Dumortier, P.; Eich, T.; et al. Towards the realization on JET of an integrated H-mode scenario for ITER. Nucl. Fusion
**2003**, 44, 124–133. [Google Scholar] [CrossRef] - Murari, A.; Lupelli, I.; Gelfusa, M.; Gaudio, P. Non-power law scaling for access to the H-mode in tokamaks via symbolic regression. Nucl. Fusion
**2013**, 53, 43001. [Google Scholar] [CrossRef] - Murari, A.; Peluso, E.; Gelfusa, M.; Lupelli, I.; Lungaroni, M.; Gaudio, P. Symbolic regression via genetic programming for data driven derivation of confinement scaling laws without any assumption on their mathematical form. Plasma Phys. Control. Fusion
**2014**, 57, 014008. [Google Scholar] [CrossRef] - Murari, A.; Peluso, E.; Lungaroni, M.; Gelfusa, M.; Gaudio, P. Application of symbolic regression to the derivation of scaling laws for tokamak energy confinement time in terms of dimensionless quantities. Nucl. Fusion
**2015**, 56, 26005. [Google Scholar] [CrossRef] - Murari, A.; Lupelli, I.; Gaudio, P.; Gelfusa, M.; Vega, J. A statistical methodology to derive the scaling law for the H-mode power threshold using a large multi-machine database. Nucl. Fusion
**2012**, 52, 63016. [Google Scholar] [CrossRef] [Green Version] - Murari, A.; Pisano, F.; Vega, J.; Cannas, B.; Fanni, A.; González, S.; Gelfusa, M.; Grosso, M.; Contributors, J.E. Extensive statistical analysis of ELMs on JET with a carbon wall. Plasma Phys. Control. Fusion
**2014**, 56, 114007. [Google Scholar] [CrossRef] - Craciunescu, T.; Murari, A. Geodesic distance on Gaussian manifolds for the robust identification of chaotic systems. Nonlinear Dyn.
**2016**, 86, 677–693. [Google Scholar] [CrossRef] - Amari, S.-I.; Nagaoka, H. Methods of Information Geometry; American Mathematical Society (AMS): Providence, RI, USA, 2007; Volume 191. [Google Scholar]
- Murari, A.; Boutot, P.; Vega, J.; Gelfusa, M.; Moreno, R.; Verdoolaege, G.; De Vries, P.C.; Contributors, J.-E. Clustering based on the geodesic distance on Gaussian manifolds for the automatic classification of disruptions. Nucl. Fusion
**2013**, 53, 33006. [Google Scholar] [CrossRef] - Giovanni, D.D.; Marchi, F.; Fiorito, R.; Luttazzi, E.; Latini, G. Two Realistic Scenarios of Intentional Release of Radionuclides (Cs-137, Sr-90)—The Use of the HotSpot Code to Forecast Contamination Extent. WSEAS Transactions on Environment and Development; WSEAS: Cambridge, MA, USA, 2014; Volume 10, pp. 106–122. [Google Scholar]
- Ciparisse, J.-F.; Malizia, A.; Poggi, L.A.; Cenciarelli, O.; Gelfusa, M.; Carestia, M.C.; Di Giovanni, D.; Mancinelli, S.; Palombi, L.; Bellecci, C.; et al. Numerical simulations as tool to predict chemical and radiological hazardous diffusion in case of nonconventional events. Model. Simul. Eng.
**2016**, 2016, 1–11. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Simple networks showing the seven trivariate cases of mutual influence between three systems.

**Figure 3.**Architecture of the time delay neural networks used to investigate the indirect coupling between three systems.

**Figure 11.**Top left: absolute error for the ZY interaction. Top right: modulation of the μ

_{32}coupling coefficient above the detection of the coupling intervals by the TDNNs. Bottom left: absolute error for the YX interaction. Bottom right: constant μ

_{21}coupling coefficient above the detection of the coupling intervals by the TDNNs (due to the amplitude variations of Y).

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 99.82% | 97.46% |

Y | 81.01% | 0.00% | 40.84% | |

Z | 74.24% | 44.57% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 3.97E-07 | 88.17% |

Y | 27.57% | 0.00% | 7.89% | |

Z | 80.36% | 40.01% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 4.55E-09 | 4.23E-55 |

Y | 68.33% | 0.00% | 67.01% | |

Z | 10.34% | 54.85% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 0.00% | 73.18% |

Y | 84.94% | 0.00% | 0.00% | |

Z | 40.19% | 74.63% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 1.16% | 0.00% |

Y | 96.52% | 0.00% | 0.00% | |

Z | 61.21% | 59.18% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 0.00% | 82.33% |

Y | 48.62% | 0.00% | 0.00% | |

Z | 0.00% | 58.31% | 0.00% |

Removed Variable | ||||
---|---|---|---|---|

X | Y | Z | ||

Predicted | X | 0.00% | 24.81% | 30.65% |

Y | 0.00% | 0.00% | 16.03% | |

Z | 0.00% | 60.72% | 0.00% |

**Table 8.**Causal Relationships for Case 4 of Figure 1.

I_{noise} | Mean SNR | Z to Z | Z to Y | Z to X | Y to Z | Y to Y | Y to X | X to Z | X to Y | X to X |
---|---|---|---|---|---|---|---|---|---|---|

0.01 | 30 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

0.02 | 16 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

0.05 | 7 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

0.1 | 3 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

0.2 | 1.2 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

0.5 | 0.4 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

1 | 0.2 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |

Expected | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rossi, R.; Murari, A.; Gaudio, P.
On the Potential of Time Delay Neural Networks to Detect Indirect Coupling between Time Series. *Entropy* **2020**, *22*, 584.
https://doi.org/10.3390/e22050584

**AMA Style**

Rossi R, Murari A, Gaudio P.
On the Potential of Time Delay Neural Networks to Detect Indirect Coupling between Time Series. *Entropy*. 2020; 22(5):584.
https://doi.org/10.3390/e22050584

**Chicago/Turabian Style**

Rossi, Riccardo, Andrea Murari, and Pasquale Gaudio.
2020. "On the Potential of Time Delay Neural Networks to Detect Indirect Coupling between Time Series" *Entropy* 22, no. 5: 584.
https://doi.org/10.3390/e22050584