# Kernel Methods for Nonlinear Connectivity Detection

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Problem Formulation

#### 2.1. Estimation and Asymptotic Considerations

#### Data Workflow

- If ${\mathrm{KCF}}_{ij}^{\left(r\right)}\left(\tau \right)$ analysis does not suggest feature space model residual whiteness, p is increased by 1, and the procedure from step 1 is repeated until feature space model residual whiteness is obtained and ${}_{g}\mathrm{AIC}\left(k\right)$ attains its first local minimum meaning that the ideal model order has been reached;
- Once the best model is attained, one employs the (39) to infer connectivity.

## 3. Numerical Illustrations

#### 3.1. Example 1

#### 3.2. Example 2

#### 3.3. Example 3

#### 3.4. Example 4

#### 3.5. Example 5

## 4. Conclusions and Future Work

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Schumacker, R.E.; Lomax, R.G. A Beginner’S Guide to Structural Equation Modeling, 4th ed.; Taylor & Francis: New York, NY, USA, 2016. [Google Scholar]
- Applebaum, D. Probability and Information: An Integrated Approach, 2nd ed.; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2008; p. 273. [Google Scholar] [CrossRef]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley Series in Telecommunications; Wiley: New York, NY, USA, 2006; p. 774. [Google Scholar]
- Hlaváčková-Schindler, K.; Paluš, M.; Vejmelka, M.; Bhattacharya, J. Causality detection based on information- theoretic approaches in time series analysis. Phys. Rep. Rev. Sect. Phys. Lett.
**2007**, 441, 1–46. [Google Scholar] [CrossRef] - Schreiber, T. Measuring Information transfer. Phys. Rev. Lett.
**2000**, 85, 461–464. [Google Scholar] [CrossRef] [PubMed] - Bendat, J.S.; Piersol, A.G. Engineering Applications of Correlation and Spectral Analysis; Wiley: New York, NY, USA; Chichester, UK, 1980. [Google Scholar]
- Baccalá, L.A.; Sameshima, K. Overcoming the limitations of correlation analysis for many simultaneously processed neural structures. Prog. Brain Res.
**2001**, 130, 33–47. [Google Scholar] [CrossRef] [PubMed] - Granger, C.W.J. Investigating causal relations by econometric models and cross-spectral methods. Econometrica
**1969**, 37, 424–438. [Google Scholar] [CrossRef] - Baccalá, L.A.; Sameshima, K. Multivariate time series brain connectivity: A sum up. Methods in Brain Connectivity Inference Through Multivariate Time Series Analysis; CRC Press: Boca Raton, FL, USA, 2014; pp. 245–251. [Google Scholar]
- Lütkepohl, H. New Introduction to Multiple Time Series Analysis; Springer: Berlin, Germany, 2005. [Google Scholar]
- Baccalá, L.A.; Sameshima, K. Partial directed coherence: A new concept in neural structure determination. Biol. Cybern.
**2001**, 84, 463–474. [Google Scholar] [CrossRef] [PubMed] - Vapnik, V.N. Statistical Learning Theory, 1st ed.; John Wiley and Sons: Hoboken, NJ, USA, 1998; p. 736. [Google Scholar]
- Priestley, M.B. Spectral Analysis and Time Series; Probability and Mathematical Statistics; Academic Press London: New York, NY, USA, 1981; p. 890. [Google Scholar]
- Nikias, C.; Petropulu, A.P. Higher Order Spectra Analysis: A Non-linear Signal Processing Framework; Prentice Hall Signal Processing Series; Prentice Hall: Upper Saddle River, NJ, USA, 1993; p. 528. [Google Scholar]
- Subba-Rao, T.; Gabr, M.M. An Introduction to Bispectral Analysis and Bilinear Time Series Models; Lecture Notes in Statistics; Springer: New York, NY, USA, 1984. [Google Scholar]
- Schelter, B.; Winterhalder, M.; Eichler, M.; Peifer, M.; Hellwig, B.; Guschlbauer, B.; Lücking, C.H.; Dahlhaus, R.; Timmer, J. Testing for directed influences among neural signals using partial directed coherence. J. Neurosci. Methods
**2006**, 152, 210–219. [Google Scholar] [CrossRef] [PubMed] - Massaroppe, L.; Baccalá, L.A.; Sameshima, K. Semiparametric detection of nonlinear causal coupling using partial directed coherence. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 5927–5930. [Google Scholar] [CrossRef]
- Schölkopf, B.; Smola, A.J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond; MIT Press: Cambridge, MA, USA, 2002. [Google Scholar]
- Mercer, J. Functions of positive and negative type, and their connection with the theory of integral equations. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci.
**1909**, A, 415–446. [Google Scholar] [CrossRef] - Golub, G.H.; van Loan, C.F. Matrix Computations, 4th ed.; Number 3 in Johns Hopkins Studies in the Mathematical Sciences; Johns Hopkins University Press: Baltimore, MD, USA, 2013; p. 784. [Google Scholar]
- Hable, R. Asymptotic normality of support vector machine variants and other regularized kernel methods. J. Multivar. Anal.
**2012**, 106, 92–117. [Google Scholar] [CrossRef] [Green Version] - Filliben, J.J. The probability plot correlation coefficient test for normality. Technometrics
**1975**, 17, 111–117. [Google Scholar] [CrossRef] - Vogel, R.M. The probability plot correlation coefficient test for the normal, lognormal, and Gumbel distributional hypotheses. Water Resour. Res.
**1986**, 22, 587–590. [Google Scholar] [CrossRef] - Vogel, R.M. Correction to “The probability plot correlation coefficient test for the normal, lognormal, and Gumbel distributional hypotheses”. Water Resour. Res.
**1987**, 23, 2013. [Google Scholar] [CrossRef] - Massaroppe, L.; Baccalá, L.A. Método semi-paramétrico para inferência de conectividade não-linear entre séries temporais. In Proceedings of the Anais do I Congresso de Matemática e Computacional da Região Sudeste, I CMAC Sudeste, Uberlândia, Brazil, 20–23 September 2011; pp. 293–296. [Google Scholar]
- Massaroppe, L.; Baccalá, L.A. Detecting nonlinear Granger causality via the kernelization of partial directed coherence. In Proceedings of the 60th World Statistics Congress of the International Statistical Institute, ISI2015, Rio de Janeiro, RJ, USA, 26–31 July 2015; pp. 2036–2041. [Google Scholar]
- Gourévitch, B.; Bouquin-Jeannès, R.L.; Faucon, G. Linear and nonlinear causality between signals: Methods, examples and neurophysiological applications. Biol. Cybern.
**2006**, 95, 349–369. [Google Scholar] [CrossRef] [PubMed] - Chen, Y.; Rangarajan, G.; Feng, J.; Ding, M. Analyzing multiple nonlinear time series with extended Granger causality. Phys. Lett. A
**2004**, 324, 26–35. [Google Scholar] [CrossRef] [Green Version] - Amblard, P.O.; Vincent, R.; Michel, O.J.J.; Richard, C. Kernelizing Geweke’s measures of Granger causality. In Proceedings of the 2012 IEEE International Workshop on Machine Learning for Signal Processing, Santander, Spain, 23–26 September 2012; pp. 1–6. [Google Scholar] [CrossRef]
- Kallas, M.; Honeine, P.; Francis, C.; Amoud, H. Kernel autoregressive models using Yule-Walker equations. Signal Process.
**2013**, 93, 3053–3061. [Google Scholar] [CrossRef] - Honeine, P.; Richard, C. Preimage problem in kernel-based machine learning. IEEE Signal Process. Mag.
**2011**, 28, 77–88. [Google Scholar] [CrossRef] - Kumar, R.; Jawahar, C.V. Kernel approach to autoregressive modeling. In Proceedings of the Thirteenth National Conference on Communications (NCC 2007), Kanpur, India, 26–28 January 2007; pp. 99–102. [Google Scholar]
- Marinazzo, D.; Pellicoro, M.; Stramaglia, S. Kernel method for nonlinear Granger causality. Phys. Rev. Lett.
**2008**, 100, 144103. [Google Scholar] [CrossRef] [PubMed] - Park, I.; Príncipe, J.C. Correntropy based Granger causality. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA, 31 March–4 April 2008; pp. 3605–3608. [Google Scholar] [CrossRef]
- Príncipe, J.C. Information Theoretic Learning: Rényi’s Entropy and Kernel Perspectives, 1st ed.; Number XIV in Information Science and Statistics; Springer Publishing Company, Incorporated: New York, NY, USA, 2010; p. 448. [Google Scholar]
- Massaroppe, L.; Baccalá, L.A. Kernel-nonlinear-PDC extends Partial Directed Coherence to detecting nonlinear causal coupling. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25–29 August 2015; pp. 2864–2867. [Google Scholar] [CrossRef]
- Massaroppe, L.; Baccalá, L.A. Causal connectivity via kernel methods: Advances and challenges. In Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016. [Google Scholar]

**Figure 1.**The sequence kernel correlation functions (KCF$\left(\tau \right)$) respectively for the quadratic and quartic kernels are contained in Figure 1a,b for Example 1. Horizontal dashed lines represent $95\%$ significance threshold interval out of which the null hypothesis ${\mathcal{H}}_{0}$ of no correlation is rejected. Asterisks (*) further stress signficant values.

**Figure 2.**The residue kernel correlation functions (KCF${}^{\left(r\right)}\left(\tau \right)$) respectively for the quadratic and quartic kernels are shown in Figure 2a,b for Example 1. Comparing them to Figure 1, it is clear that the kernel correlations are reduced after modelling as it is now impossible to reject KCF${}^{\left(r\right)}\left(\tau \right)$ nullity at $95\%$ as no more than $5\%$ of the values lie outside the dashed interval around zero. Asterisks (*) further stress significant values.

**Figure 3.**Ensemble normal probability plots for ${\widehat{a}}_{21}$, respectively for 3a quadratic and 3b quartic kernels, illustrate and confirm asymptotic normality.

**Figure 4.**Filliben squared-correlation coefficient convergence to Gaussianity as a function of ${n}_{s}$ for both kernels used in Example 1.

**Figure 5.**True positive and false positive rates from the kernelized Granger causality test for various samples sizes (${n}_{s}$) for $\alpha =1\%$. Note that the false-positive-rates for both kernels overlap.

**Figure 6.**True positive (${x}_{1}\to {x}_{2}$) and false positive rates (${x}_{2}\to {x}_{1}$) from the kernelized Granger causality test under a quadratic kernel as a function ${n}_{s}$ in Example 2.

**Figure 7.**True positive and false positive rates (Example 3) from the kernelized Granger causality test using a quadratic kernel as a function of ${n}_{s}$. Note that the false-positive-rate for the connections $1\leftarrow 2$, $2\leftarrow 3$ and $1\leftarrow 3$ overlap over the investigated ${n}_{s}$ range.

**Figure 8.**True positive and false positive rates from the kernelized Granger causality test under a quadratic kernel as function of record length ${n}_{s}$ in Example 4.

**Figure 9.**Generalized Hannan–Quinn criterion (${}_{\mathrm{g}}\mathrm{AIC}\left(k\right)$) with ${c}_{{n}_{s}}=\mathrm{ln}\left(\mathrm{ln}\left({n}_{s}\right)\right)$ as a function of model order for various observed record lengths ${n}_{s}$ using a typical realization from (48).

**Figure 10.**Generalized Hannan–Quinn criterion (${}_{\mathrm{g}}\mathrm{AIC}\left(k\right)$) as a function of model order for the various data lengths ${n}_{s}$ from a typical realization from (49).

**Figure 11.**Observed true positive and false positive rates from the kernelized Granger causality test under a quadratic kernel for various record lengths ${n}_{s}$ in Example 5. Note that the false-positive-rate for the connections $2\leftarrow 3$, $3\leftarrow 1$ and $1\leftarrow 3$ overlap over the ${n}_{s}$ range, except for $1\leftarrow 2$, which, however, attains the same level as the others after ${n}_{s}=128$.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Massaroppe, L.; Baccalá, L.A.
Kernel Methods for Nonlinear Connectivity Detection. *Entropy* **2019**, *21*, 610.
https://doi.org/10.3390/e21060610

**AMA Style**

Massaroppe L, Baccalá LA.
Kernel Methods for Nonlinear Connectivity Detection. *Entropy*. 2019; 21(6):610.
https://doi.org/10.3390/e21060610

**Chicago/Turabian Style**

Massaroppe, Lucas, and Luiz A. Baccalá.
2019. "Kernel Methods for Nonlinear Connectivity Detection" *Entropy* 21, no. 6: 610.
https://doi.org/10.3390/e21060610