# Dynamical and Coupling Structure of Pulse-Coupled Networks in Maximum Entropy Analysis

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Results

**Fact**

**1.**

## 3. Conclusions and Discussion

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Mirollo, R.E.; Strogatz, S.H. Synchronization of pulse-coupled biological oscillators. SIAM J. Appl. Math.
**1990**, 50, 1645–1662. [Google Scholar] [CrossRef] - Stricker, J.; Cookson, S.; Bennett, M.R.; Mather, W.H.; Tsimring, L.S.; Hasty, J. A fast, robust and tunable synthetic gene oscillator. Nature
**2008**, 456, 516–519. [Google Scholar] [CrossRef] [PubMed] - Wang, Z.; Ma, Y.; Cheng, F.; Yang, L. Review of pulse-coupled neural networks. Image Vis. Comput.
**2010**, 28, 5–13. [Google Scholar] [CrossRef] - Dan, Y.; Alonso, J.M.; Usrey, W.M.; Reid, R.C. Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nat. Neurosci.
**1998**, 1, 501–507. [Google Scholar] [CrossRef] [PubMed] - Vinje, W.E.; Gallant, J.L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science
**2000**, 287, 1273–1276. [Google Scholar] [CrossRef] - Ohiorhenuan, I.E.; Mechler, F.; Purpura, K.P.; Schmid, A.M.; Hu, Q.; Victor, J.D. Sparse coding and high-order correlations in fine-scale cortical networks. Nature
**2010**, 466, 617–621. [Google Scholar] [CrossRef] [Green Version] - Shemesh, Y.; Sztainberg, Y.; Forkosh, O.; Shlapobersky, T.; Chen, A.; Schneidman, E. High-order social interactions in groups of mice. eLife
**2013**, 2, e00759. [Google Scholar] [CrossRef] [PubMed] - Knill, D.C.; Pouget, A. The Bayesian brain: The role of uncertainty in neural coding and computation. TRENDS Neurosci.
**2004**, 27, 712–719. [Google Scholar] [CrossRef] - Karlsson, M.P.; Frank, L.M. Awake replay of remote experiences in the hippocampus. Nat. Neurosci.
**2009**, 12, 913. [Google Scholar] [CrossRef] - Ganmor, E.; Segev, R.; Schneidman, E. Sparse low-order interaction network underlies a highly correlated and learnable neural population code. Proc. Natl. Acad. Sci. USA
**2011**, 108, 9679–9684. [Google Scholar] [CrossRef] [Green Version] - Schneidman, E.; Berry, M.J.; Segev, R.; Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature
**2006**, 440, 1007–1012. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Shlens, J.; Field, G.D.; Gauthier, J.L.; Grivich, M.I.; Petrusca, D.; Sher, A.; Litke, A.M.; Chichilnisky, E. The structure of multi-neuron firing patterns in primate retina. J. Neurosci.
**2006**, 26, 8254–8266. [Google Scholar] [CrossRef] [PubMed] - Tang, A.; Jackson, D.; Hobbs, J.; Chen, W.; Smith, J.L.; Patel, H.; Prieto, A.; Petrusca, D.; Grivich, M.I.; Sher, A.; et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J. Neurosci.
**2008**, 28, 505–518. [Google Scholar] [CrossRef] - Marre, O.; El Boustani, S.; Frégnac, Y.; Destexhe, A. Prediction of spatiotemporal patterns of neural activity from pairwise correlations. Phys. Rev. Lett.
**2009**, 102, 138101. [Google Scholar] [CrossRef] [PubMed] - Bury, T. Statistical pairwise interaction model of stock market. Eur. Phys. J. B
**2013**, 86, 89. [Google Scholar] [CrossRef] - Watanabe, T.; Hirose, S.; Wada, H.; Imai, Y.; Machida, T.; Shirouzu, I.; Konishi, S.; Miyashita, Y.; Masuda, N. A pairwise maximum entropy model accurately describes resting-state human brain networks. Nat. Commun.
**2013**, 4, 1370. [Google Scholar] [CrossRef] [Green Version] - Barreiro, A.K.; Gjorgjieva, J.; Rieke, F.; Shea-Brown, E. When do microcircuits produce beyond-pairwise correlations? Front. Comput. Neurosci.
**2014**, 8, 10. [Google Scholar] [CrossRef] [PubMed] - Martin, E.A.; Hlinka, J.; Davidsen, J. Pairwise network information and nonlinear correlations. Phys. Rev. E
**2016**, 94, 040301. [Google Scholar] [CrossRef] - Ganmor, E.; Segev, R.; Schneidman, E. The architecture of functional interaction networks in the retina. J. Neurosci.
**2011**, 31, 3044–3054. [Google Scholar] [CrossRef] - Yu, S.; Yang, H.; Nakahara, H.; Santos, G.S.; Nikolić, D.; Plenz, D. Higher-order interactions characterized in cortical activity. J. Neurosci.
**2011**, 31, 17514–17526. [Google Scholar] [CrossRef] - Zhou, D.; Xiao, Y.; Zhang, Y.; Xu, Z.; Cai, D. Causal and structural connectivity of pulse-coupled nonlinear networks. Phys. Rev. Lett.
**2013**, 111, 054102. [Google Scholar] [CrossRef] - Newman, M.E. The structure and function of complex networks. SIAM Rev.
**2003**, 45, 167–256. [Google Scholar] [CrossRef] - Bullmore, E.; Sporns, O. Complex brain networks: Graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci.
**2009**, 10, 186–198. [Google Scholar] [CrossRef] [PubMed] - Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 620. [Google Scholar] [CrossRef] - Amari, S.I. Information geometry on hierarchy of probability distributions. IEEE Trans. Inf. Theory
**2001**, 47, 1701–1711. [Google Scholar] [CrossRef] [Green Version] - Xu, Z.Q.J.; Crodelle, J.; Zhou, D.; Cai, D. Maximum Entropy Principle Analysis in Network Systems with Short-time Recordings. arXiv, 2018; arXiv:1808.10506. [Google Scholar]
- Xu, Z.Q.J.; Bi, G.; Zhou, D.; Cai, D. A dynamical state underlying the second order maximum entropy principle in neuronal networks. Commun. Math. Sci.
**2017**, 15, 665–692. [Google Scholar] [CrossRef] - Gerstner, W.; Kistler, W.M. Spiking Neuron Models: Single Neurons, Populations, Plasticity; Cambridge University Press: Cambridge, UK, 2002. [Google Scholar]
- Cai, D.; Rangan, A.V.; McLaughlin, D.W. Architectural and synaptic mechanisms underlying coherent spontaneous activity in V1. Proc. Natl. Acad. Sci. USA
**2005**, 102, 5868–5873. [Google Scholar] [CrossRef] [Green Version] - Shlens, J.; Field, G.D.; Gauthier, J.L.; Greschner, M.; Sher, A.; Litke, A.M.; Chichilnisky, E. The structure of large-scale synchronized firing in primate retina. J. Neurosci.
**2009**, 29, 5022–5031. [Google Scholar] [CrossRef] - Macke, J.H.; Opper, M.; Bethge, M. Common input explains higher-order correlations and entropy in a simple model of neural population activity. Phys. Rev. Lett.
**2011**, 106, 208102. [Google Scholar] [CrossRef] - Nasser, H.; Marre, O.; Cessac, B. Spatio-temporal spike train analysis for large scale networks using the maximum entropy principle and Monte Carlo method. J. Stat. Mech. Theory Exp.
**2013**, 2013, P03006. [Google Scholar] [CrossRef]

**Figure 2.**Anatomical structure vs. effective interactions of integrate-and-fire networks. Each row shows a numerical case. In the first column, black arrows and red arrows represent excitatory and inhibitory connections, respectively. In the second column, red and green dots are the strengths of ${\Delta}_{ij}\left(H\right)$ of dependent and independent pairs, respectively. Blue dots and cyan dots are the strengths of ${\Delta}_{ij}\left(H\right)$ of dependent and independent pairs from ten shuffled spike trains, respectively. Each dot is for one ${\Delta}_{ij}\left(H\right)$. The third and fourth columns display absolute effective interaction strengths (blue bars). The corresponding node indexes for each effective interaction are shown in the abscissa. The mean and standard deviation of absolute strengths of each effective interaction of ten shuffled spike trains are also displayed by garnet bars. The simulation time for each network is $1.2\times {10}^{8}\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$. The time bin size for analysis is $10\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$ [12,13]. Independent Poisson inputs for each network are $\mu =0.1\phantom{\rule{0.166667em}{0ex}}{\mathrm{ms}}^{-1}$ and $f=0.1\phantom{\rule{0.166667em}{0ex}}{\mathrm{ms}}^{-1}$. The firing rate of each node is about $50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$. Parameters are chosen [28] as ${x}_{\mathrm{ex}}=14/3$, ${x}_{\mathrm{in}}=-2/3$, ${\sigma}^{\mathrm{ex}}=2\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$, ${\sigma}^{\mathrm{in}}=5\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$, $\tau =20\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$, ${x}_{\mathrm{th}}=1$, ${x}_{r}=0$, and ${\tau}_{\mathrm{ref}}=2\phantom{\rule{0.166667em}{0ex}}\mathrm{ms}$, ${S}_{ij}^{\mathrm{ex}}={S}_{ij}^{\mathrm{in}}=0.02$.

**Figure 3.**Non-zero effective interactions in the Erdos-Renyi random networks. We generate 1000 Erdos-Renyi random networks of 100 nodes (the same connection probability but different random samples). The connection probability between two nodes is $0.05$. The number of non-zero effective interaction is plotted against effective interaction order. The mean and standard deviation are respectively shown by the black line and shaded area.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xu, Z.-Q.J.; Zhou, D.; Cai, D.
Dynamical and Coupling Structure of Pulse-Coupled Networks in Maximum Entropy Analysis. *Entropy* **2019**, *21*, 76.
https://doi.org/10.3390/e21010076

**AMA Style**

Xu Z-QJ, Zhou D, Cai D.
Dynamical and Coupling Structure of Pulse-Coupled Networks in Maximum Entropy Analysis. *Entropy*. 2019; 21(1):76.
https://doi.org/10.3390/e21010076

**Chicago/Turabian Style**

Xu, Zhi-Qin John, Douglas Zhou, and David Cai.
2019. "Dynamical and Coupling Structure of Pulse-Coupled Networks in Maximum Entropy Analysis" *Entropy* 21, no. 1: 76.
https://doi.org/10.3390/e21010076