Identification of Couplings in Adaptive Dynamical Networks of Time-Delayed Feedback Oscillators

Abstract

An approach to solve the inverse problem of the reconstruction of the network of time-delay oscillators from their time series is proposed and studied in the case of the nonstationary connectivity matrix. Adaptive couplings have not been considered yet for this particular reconstruction problem. The problem of coupling identification is reduced to linear optimization of a specially constructed target function. This function is introduced taking into account the continuity of the nonlinear functions of oscillators and does not exploit the mean squared difference between the model and observed time series. The proposed approach allows us to minimize the number of estimated parameters and gives asymptotically unbiased estimates for a large class of nonlinear functions. The approach efficiency is demonstrated for the network composed of time-delayed feedback oscillators with a random architecture of constant and adaptive couplings in the absence of a priori knowledge about the connectivity structure and its evolution. The proposed technique extends the application area of the considered class of methods.

1. Introduction

Networks consisting of interacting oscillatory elements are extremely widespread in nature and technology. When studying spatially developed systems, it is necessary to take into account that signals propagate in them with a finite speed and, therefore, need time to overcome distances [1,2,3]. Moreover, many self-sustained oscillation systems of various natures (physical, chemical, climatic, and biological) are characterized by the presence of time-delayed feedback [4,5,6]. Thus, the use of time-delay oscillators as network nodes brings the studied model systems closer to real objects of nature and human-created network structures.

In general, oscillatory networks may have a complex architecture of connections between nodes and, along with network nodes, couplings between the nodes can also have their own dynamics and exhibit evolution over time. Such adaptive dynamical networks, in which the topology of connections can be rearranged and the intensities of connections have their own dynamics, are widespread in the real world, for example, in neurodynamics and power supply networks [7,8,9,10,11]. The topology of couplings plays an important role in the occurrence of a particular type of collective dynamics, while the dynamics of nodes and couplings affects the restructuring of the topology, i.e., there is a mutual interaction between the network dynamics and the evolution of network topology. The processes of information spreading in networks, synchronization of network elements, and the reliability and stability of the network with respect to external disturbances depend on the architecture and intensity of couplings [12,13,14].

Thus, it is important to know the architecture of couplings in the studied network of oscillators. A correlation and spectral analysis, information-theoretic and nonlinear dynamic characteristics are widely used to identify dependencies between the interacting systems. However, to evaluate directional couplings, that is, to answer the question of whether one node affects another one and with what strength, the most suitable approach is often to reconstruct the operator of evolution from the observed time series of oscillatory systems [15,16,17,18,19,20]. More general approaches such as different versions of Granger causality [21,22,23,24,25], phase dynamics modelling [26,27,28], partial directed coherence [29], transfer entropy [30], and other methods [31,32,33,34] are also used to reconstruct couplings in multi-element systems.

It should be noted that most methods for identifying directional couplings in networks of oscillators are developed for the case of a constant topology of connections within the network. However, as noted above, adaptive networks with time-varying connections between the nodes are widespread in the real world. Such networks are the most difficult to reconstruct. To solve the ambitious task of reconstructing directed couplings in adaptive dynamical networks consisting of systems with an internal time delay, we propose an original approach based on a piece-wise linear interpolation of coupling dynamics in time, optimization of a specially constructed target function, and the separation of the recovered coupling coefficients into significant and insignificant coefficients.

2. Method

Let us consider a network of linearly coupled first-order time-delay oscillators:

$${\epsilon }_i{\dot{x}}_i\left(t\right)=-x_i\left(t\right)+f_i\left(x_i(t-{\tau }_i)\right)+\sum _{j=1,j\ne i}^D{k}_{i,j}\left(t\right)\left(x_j\left(t\right)-x_i\left(t\right)\right),$$

(1)

where $x_i\left(t\right)$ is the observed variable of the i-th oscillator, D is the number of oscillators, ${\tau }_i$ is the delay time, the parameter ${\epsilon }_i$ characterizes the inertial properties of the oscillator, $f_i$ is a continuous nonlinear function, $\dot{x_i}\left(t\right)$ is a time derivative, and $k_{i,j}\left(t\right)$ are the varying in time coupling coefficients characterizing the strength of influence $j\to i$, i.e., from the j-th oscillator to the i-th one. It is assumed that the variation of $k_{i,j}\left(t\right)$ occurs slowly compared to the main time scale of oscillations, which is mainly determined by the delay times ${\tau }_i$.

Following [18], we divided the reconstruction procedure into two main steps. First, the model for each oscillator was reconstructed individually with all possible couplings $k_{i,j}$. Second, insignificant (spurious) coupling terms were detected by the special algorithm and removed from the model, and the reconstruction for each oscillator was performed once again.

Since we considered $f_i$ to be an unknown nonlinear continuous function, there were two main possible approaches to its reconstruction. First, one can use series expansion with polynomials [35] or with trigonometric series [36]. This approach can be named as “explicit” due to the explicit approximation of an unknown nonlinear function. The target function for optimization in such a case is usually constructed using the difference between the state vector components and their approximated values. Another approach was first proposed in [37] and with some changes used in [17,18]. It exploits the target function that is constructed taking into account the continuity of nonlinear function. In this case, no explicit approximation is necessary: the nonlinear function is obtained as a table, being calculated in a finite set of argument values. This approach does not require an explicit approximation of $f_i$ and can be named as “implicit”. Its shortcoming is that the obtained estimates of parameter are only asymptotically unbiased with $N\to \infty$, where N is the length in points of the observed time series. We followed the second approach here because of its efficiency for other applications [17,18] and since it significantly reduces the number of parameters to be fitted to data.

We can rewrite (1) as follows:

$$f_i\left(x_i(t-{\tau }_i)\right)=x_i\left(t\right)+{\epsilon }_i{\dot{x}}_i\left(t\right)-\sum _{j=1,j\ne i}^D{k}_{i,j}\left(t\right)\left(x_j\left(t\right)-x_i\left(t\right)\right).$$

(2)

Let us assume that we had the time series ${\left\{x_i\left(t_n\right)\right\}}_{n=0}^N$ measured at $t_n=t_0+n\Delta t$, where $\Delta t$ is the sampling time. We denoted $x_i\left(n\right)=x_i\left(t_n\right)$. Then, we applied the smoothing polynomial of order B fitted to the observable using the least-squares in each data point n, where $(m-1)/2⩽n⩽N-(m-1)/2$ and m is the number of points with the n-th point as a central one. The values of the derivative $\dot{x_i}\left(n\right)$ can be estimated by differentiation of this polynomial. This approach is known as the Savitzky–Golay filter [38].

If each $k_{i,j}\left(t\right)$ is a continuous, relatively slow function of time, the simplest possible approximation is a piece-wise linear (3), where the time series of all oscillators are divided into L fragments having the length $N_l$,

$$k_{i,j,l}\left(t\right)=k_{i,j,l}+k_{i,j,l}^{\prime }(t-t_l),$$

(3)

where $k_{i,j,l}$ is the mean value at the l-th fragment, $k_{i,j,l}^{\prime }$ is the slope, and $t_l$ is the central point of the l-th fragment. One can use another approximation, but (3) gives a clear understanding of both $k_{i,j,l}$ and $k_{i,j,l}^{\prime }$. The approximation (3) also provides good computational properties, since it minimizes the range of normalized time at the l-th fragment. At each fragment, one has to estimate $2(D-1)$ coupling coefficients.

We considered the reconstruction of network for each time fragment separately. Let us denote ${\alpha }_{i,0,l}={\epsilon }_i$, ${\alpha }_{i,j,l}=-k_{i,j,l}$, and ${\alpha }_{i,j+D-1,l}=-k_{i,j,l}^{\prime }$ for the l-th fragment and rewrite Equation (2) as follows:

$$\begin{array}{ccc}\hfill f_i\left(x_i\left(n\right)\right)& =& x_i(n+{\theta }_i)+{\alpha }_{i,0,l}{\dot{x}}_i(n+{\theta }_i)\hfill \\ & +& \sum _{j=1,j\ne i}^D{\alpha }_{i,j^{\prime },l}\left(x_j(n+{\theta }_i)-x_i(n+{\theta }_i)\right)\hfill \\ & +& \sum _{j=D,j\ne i}^{2D-1}{\alpha }_{i,j^{\prime },l}\left(x_j(n+{\theta }_i)-x_i(n+{\theta }_i)(n+{\theta }_i)\Delta t\right),\hfill \end{array}$$

(4)

where ${\theta }_i=\lfloor {\tau }_i/\Delta t\rfloor$, $n=-(N_l-1)/2,\cdots ,(N_l-1)/2$, $j^{\prime }=j$ if $j<i$ and $j^{\prime }=j-1$ if $j>i$. For simplicity, we considered the situation where ${\tau }_i/\Delta t$ is an integer and all $N_l$ were odd numbers.

Then, we considered transform $Q_{i,l}$, which maps the number n of the value $x_i\left(n\right)$ in the l-th fragment of the original time series to its number $Q_{i,l}\left(n\right)$ in the time series of the l-th fragment sorted by the increase in $x_i$. The inverse map we denoted as $Q_{i,l}^{-1}$. Let us denote $p_n=Q_{i,l}^{-1}(Q_{i,l}\left(n\right)+1)$, which is the original number of value next to $x_i\left(n\right)$ in the sorted time series. Since we considered functions $f_i$ to be continuous, small absolute differences between $x_i\left(p_n\right)$ and $x_i\left(n\right)$ resulted in small absolute differences between $f_i\left(x_i\left(p_n\right)\right)$ and $f_i\left(x_i\left(n\right)\right)$. Actually, ${\delta }_i\left(n\right)=|f_i\left(x_i\left(p_n\right)\right)-f_i\left(x_i\left(n\right)\right)|\to 0$$\forall n\in ({\nu }_{l-1};{\nu }_l]$ for $N_l\to \infty$, where ${\nu }_l={\sum }_{l^{\prime }=1}^l{N}_{l^{\prime }}$ is the number of the final time moment at the l-th fragment. Since the coupling was assumed to be nonstationary, we could not use a very long time of observation in order to increase $N_l$. Instead, we could increase $N_l$ by decreasing the sampling time, i.e., by $\Delta t\to 0$.

Using all the values of ${\delta }_{i,l}\left(n\right)$ for the chosen i and l, we constructed the following scalar function that depends on the vector ${\overrightarrow{\alpha }}_{i,l}=({\alpha }_{i,0,l},\cdots ,{\alpha }_{i,2D-1,l})$:

$$S({\overrightarrow{\alpha }}_{i,l})=\sum _{n={\nu }_{l-1}+1,Q\left(n\right)\ne {\nu }_l}^{{\nu }_l}{\delta }_{i,l}^2\left(n\right).$$

(5)

Note that for every ${ϵ}_i^2$ and for all natural l, there is $N_l$ such that $S({\overrightarrow{\alpha }}_{i,l})$ in (5) is smaller than ${ϵ}_i^2$ if the following conditions are true:

• $\left(x_i\left(n\right)\right)$ is bounded in n;
• $f_i\left(x_i\left(n\right)\right)$ is continuous on $[inf\left\{x_i\left(n\right)\right\};sup\left\{x_i\left(n\right)\right\}]$;
• $x_i\left(t\right)$ is continuous on $[t_0,t_N]$;
• $f_i\left(x_i\right)=O(x_i^{P_i})$ as $N_l\to \infty$ with $P_i\ge 1$, where $P_i$ is the order of polynomial used for the expansion of $f_i$.

This means that $\underset{N_l\to \infty }{lim}S({\overrightarrow{\alpha }}_{i,l})=0$ if $|f_i\left(x_i\right)|$ increases faster than $\sqrt{x_i}$ and the range of $x_i$ is bounded which is true for real physical processes.

Indeed, if $f_i\left(x_i\right)=O(x_i^{P_i})$, then ${\delta }_i^2\left(n\right)={\left(f_i\left(x_i\left(p_n\right)\right)-f_i\left(x_i\left(n\right)\right)\right)}^2=O(\Delta x_i^{2P_i})$ as $N_l\to \infty$, where $\Delta x_i\left(n\right)=x_i\left(p_n\right)-x_i\left(n\right)=O(1/N_l)$. Therefore, ${\delta }_i^2\left(n\right)=O(N_l^{-2P_i})$ and $S({\overrightarrow{\alpha }}_{i,l})=O\left(N_l{\delta }_i^2\left(n\right)\right)=O(N_l^{1-2P_i})$, which implies the asymptotically uniform estimate for a given $N_l$ and $N_l^{\prime }>N_l$:

$${\left(N_l^{\prime }/N_l\right)}^{1-2P_i}<C{ϵ}_i^2$$

(6)

with a certain $C>0$. Therefore:

$$N_l^{\prime }>N_l{\left(\frac{1}{C{ϵ}_i^2}\right)}^{\frac{1}{2P_i-1}}.$$

(7)

The inequality (7) gave us an estimate of $N_l^{\prime }$ for $N_l$ matching the desired accuracy for estimation of parameters ${\alpha }_{i,j,l}$ using the minimization of the target function $S({\overrightarrow{\alpha }}_{i,l})$. As far as $S({\overrightarrow{\alpha }}_{i,l})$ is a positively defined quadratic form of ${\alpha }_{i,j,l}$, it has a unique extremum, which is the minimum, and this minimum can be found using the linear least-squares.

After the network reconstruction, the statistical significance of all coupling coefficients was evaluated for each time fragment l as it was proposed in [18]. In particular, logarithms of absolute values of all $k_{i,j,l}$ for the fixed l were considered as a set, sorted, and divided into two clusters using the standard K-means algorithm. The coefficients in the first cluster with the higher values of $log\left(\right|k_{i,j,l}\left|\right)$ were believed to be significant, while the coefficients from the second cluster were declared to be insignificant. If $k_{i,j,l}$ belonged to a significant cluster for at least one l, it was considered as significant for the entire time series. We assumed that a coupling coefficient could increase from zero or decrease to zero during the observation time. We did not evaluate the statistical significance of coefficients $k_{i,j,l}^{\prime }$. At the next step, the significant coupling coefficients $k_{i,j}\left(t\right)$ were excluded from the model (1) and the reconstruction was performed once again for the reduced model.

The proposed method assumed the delay times ${\tau }_i$ to be known. However, in general case, ${\tau }_i$ can be unknown. To solve this problem, special techniques can be used [39]. The proposed approach can also solve this problem by constructing the dependency of target function on the trial delay time ${\theta }_i^{\prime }$. Since each oscillator was reconstructed independently and the delay time could be adjusted with an accuracy up to $\Delta t$, a simple linear search for ${\theta }_i^{\prime }$ in the range from 1 to some ${\theta }_{i,max}^{\prime }$ was possible. In such a case, the delay time would be $O({\theta }_{i,max}^{\prime })$. Since the core part of the algorithm was the well-known linear least-squares problem for which the efficient solvers were available [40], this extension was acceptable.

3. Results

3.1. Simulation

To test the proposed technique, we considered a network of eight nonidentical oscillators (1) with a quadratic nonlinear function:

$$f_i\left(x_i\right)={\lambda }_i-x_i^2,$$

(8)

where ${\lambda }_i$ is the parameter of nonlinearity. The values of the network parameters were taken the same as in the paper [18]: ${\lambda }_i=1.78+0.02i$, ${\tau }_i=(225+25i)\Delta t$, ${\epsilon }_i=0.0011+0.0004i$, where $i=1,\cdots ,8$, and $\Delta t={10}^{-4}$. The equations (1) were solved using the Euler method with the time step equal to the sampling time $\Delta t$. After the transient process, the time series of all oscillators were recorded. The length of time series was $N=$ 100,000.

Each oscillator in the network was affected by two other oscillators; Figure 1. In particular, the first oscillator was affected by the fourth and eighth oscillators, and its model equation contained two nonzero coupling coefficients $k_{1,4}\left(t\right)$ and $k_{1,8}\left(t\right)$. The presence of unidirectional couplings from the fourth oscillator to the first oscillator and from the eighth oscillator to the first one is illustrated in Figure 1 by the two black squares in the first row that correspond to the coupling coefficients $k_{1,4}\left(t\right)$ and $k_{1,8}\left(t\right)$.

The total number of couplings in the network was equal to 16. The values of 14 coupling coefficients were randomly chosen from the interval $(0.08;0.18)$ and they were constant during the simulation. Two couplings varied in time. The coupling coefficient $k_{3,5}\left(t\right)$ was a piece-wise linear. It was equal to zero in the first third of the observation time, then it increased linearly from zero to 0.13 during the second third of the observation time, and was equal to 0.13 in the last third of the observation time. The coupling coefficient $k_{5,7}\left(t\right)$ varied by a sine law with the period equal to the time of observation T: $k_{5,7}\left(t\right)=0.1(sin(2\pi t/T)+1)$.

Figure 2 shows the part of the simulated chaotic time series for all eight oscillators.

3.2. Identification

The time series of oscillators were divided into $L=20$ not overlapping fragments each containing $N_l=5000$ points. In Figure 2, the first three fragments of time series are presented. The boundaries between the fragments are shown with bold gray vertical lines. The fragment length was chosen as a compromise between the desired time resolution and the number of points necessary for the estimation of $2D(D-1)+D=120$ model coefficients for the fixed l: $D(D-1)$ coefficients $k_{i,j,l}$, $D(D-1)$ coefficients $k_{i,j,l}^{\prime }$, and D coefficients for ${\epsilon }_i$. We also took into account that each fragment had to contain at least 5–6 oscillations, otherwise the sorting map $Q_{i,l}$ would not be efficient.

In the present paper, we did not focus on the reconstruction of individual node parameters that were recovered with good accuracy. Delay times ${\tau }_i$ were identified with an error not larger than one time step $\Delta t$, while the parameters ${\epsilon }_i$ were estimated with an error of less than 1%. An even better accuracy of reconstruction could be achieved if longer series were used.

The results of coupling identification are presented in Figure 3. This Figure shows only significant coupling coefficients. Comparing Figure 3 to Figure 1, one can see that the architecture of couplings in the network was accurately reconstructed. Actually, all 16 existing couplings were detected and nonexisting couplings were not detected. Each coupling coefficient $k_{i,j}$ in Figure 3a corresponds to the black square in Figure 1. For example, the coupling coefficients $k_{1,4}$ and $k_{1,8}$ in Figure 3a correspond to the black squares in the fourth and eighth line, respectively, in the first row in Figure 1.

Figure 3a shows that 14 coupling coefficients were constant and two coefficients ($k_{3,5}$ and $k_{5,7}$) noticeably changed their values during the observation time. In Figure 3b, the values of $k_{3,5}^{\prime }$ and $k_{5,7}^{\prime }$ exhibited the largest difference from zero over all $k_{i,j}^{\prime }$. Note that Figure 3a,b agree very well between themselves. Figure 3 indicates that variation of couplings can be easily detected by inspecting the mean value and the slope of coupling coefficients (3). However, the slope $k_{i,j}^{\prime }$ was not very stable for the used length of fragments and errors were present at some l.

We obtained qualitatively similar results of coupling identification for networks containing a different number of oscillators and a different number of couplings within the network.

4. Discussion and Conclusions

In the previous studies, where the specially developed implicit approaches were applied to systems of different natures, including time-delay oscillators [17,18], generalized van der Pol equations [41], and first-order neurooscillators [42,43], the coupling architecture and strength were assumed to be constant during the entire time of observation. However, for many real-world networks, this assumption is a significant disadvantage, since couplings can evolve over time. Moreover, the identification of evolving couplings can be an even more important task than the reconstruction of the oscillator parameters. For adaptive networks, specially designed time-variant versions of the Granger causality [44,45] and related techniques such as partial directed coherence [46] and transfer entropy [47] are usually used in neuroscience together with simpler nonlinear measures such as mutual information [48] and nonlinear correlation [49].

Since Granger causality uses general models in the form of multidimensional linear or nonlinear autoregressive processes, this study is the first in which the temporal evolution of couplings was included in a specially developed method of reconstruction for networks of time-delay systems. The inclusion of the temporal evolution of couplings in the model can be performed in two ways. First, one can use an explicit but parameterized law of evolution of the coupling coefficient for the entire time series, for example, polynomial, harmonic, or sigmoidal. Second, one can divide the time series into fragments and propose some simple formulas for the coupling at each fragment. For example, the coupling can be constant at the fragment of time series or can vary by linear, quadratic, or another law. The first approach provides statistically more accurate parameter estimates because it typically uses fewer model coefficients. Moreover, increasing the length of the time series can improve the accuracy of the method. However, this approach requires a priori knowledge of the law of coupling evolution, so it lacks generality. Here we implemented the second approach using a linear function with two coefficients (mean and slope) that needed to be estimated at each fragment. The use of linear approximation for each coupling is also advantageous because it gives a clear interpretation for both coefficients: $k_{i,j,l}$ characterizes the mean coupling strength for the particular time fragment, while $k_{i,j,l}^{\prime }$ showing whether this coupling is constant, increases, or decreases during this time fragment.

It should be noted that the target function construction and the approximation of coupling coefficients in the proposed method did not depend on the presence of time-delayed feedback in the network nodes. Therefore, the proposed technique can be applied for the identification of couplings not only in networks of time-delay systems, but also in networks composed of other types of oscillators.