Modeling Temporal Lobe Epilepsy during Music Large-Scale Form Perception Using the Impulse Pattern Formulation (IPF) Brain Model

Abstract

Musical large-scale form is investigated using an electronic dance music piece fed into a Finite-Difference Time-Domain physical model of the cochlea, which again is input into an Impulse Pattern Formulation (IPF) Brain model. In previous studies, experimental EEG data showed an enhanced correlation between brain synchronization and the musical piece’s amplitude and fractal correlation dimension, representing musical tension and expectancy time points within the large-scale form of musical pieces. This is also in good agreement with a FitzHugh–Nagumo oscillator model.However, this model cannot display temporal developments in large-scale forms. The IPF Brain model shows a high correlation between cochlea input and brain synchronization at the gamma band range around 50 Hz, and also a strong negative correlation with low frequencies, associated with musical rhythm, during time frames with low cochlea input amplitudes. Such a high synchronization corresponds to temporal lobe epilepsy, often associated with creativity or spirituality. Therefore, the IPF Brain model results suggest that these conscious states occur at times of low external input at low frequencies, where isochronous musical rhythms are present.

1. Introduction

Musical large-scale forms; the temporal organization of a musical piece over its whole length, e.g., the song’s verse and refrain structure; electronic dance music tension build-up and relaxation; and sonata song structures are poorly researched in terms of brain dynamics [1,2]. It is widely accepted that music is organized according to Gestalt principles [3,4], often with hierarchical structures, psychologically [5,6], or in terms of music theory [7], and is subject to long-term memory [8].

The processing of sound in the brain has historically been understood as a bottom-up process, starting from the transition of sound in the cochlea into neural spikes [9], further processed in the auditory pathway [10], to coincidence detection [9], tonotopy, interaural level and time difference detection, and pitch perception [11,12], among others. Still, even within the auditory cortex, multiple bottom-up and top-down connections are present [13], and so viewing the auditory pathway as a complex, self-organizing neural network seems more appropriate.

Such a self-organizing view is standard in terms of cortical processing, and existing neural models vary in terms of complexity and scaling [14]. Only a few brain models try to understand the brain with simple principles like the free-energy principle [15], assuming adaptation of the brain to external surprises; the global workspace view [16], assuming synchronization and de-synchronization of brain parts over time; or the synergetic approach of Gestalt perception [17].

Machine learning models also view conscious content as a process of complex, often nonlinear interactions between single neurons, also leading to heuristic and coherent Gestalt-like connectionist [18,19] and music models [20,21] when analyzing large musical databases such as Computational Phonogram Archiving [22] for streaming platforms or archives in general [23].

The self-organizing concept is also reflected in the idea of fifty-millisecond intervals of organized neural spatiotemporal patterns followed by short, chaotic disturbances associated with olfactory [24] or auditory [25,26] conscious content. Enlarging the picture to interactions between subjects results in the idea of a self-organizing society of brains [27] or the inclusion of cultural artifacts and nature in Physical Culture Theory [28].

Incorporating brains with cultural objects requires a general framework which was proposed as the Impulse Pattern Formulation (IPF), first developed for musical instruments [29]. As the only force we physically experience—next to gravity—is electricity, it is straightforward to formulate acoustic as well as electric nervous spike impulses as being of the same nature. The IPF then takes a viewpoint of a neuron or musical instrument part, from which an impulse is sent out to several other neurons, musical instrument parts, or any kind of object. This impulse is processed, damped, and returned back to the viewpoint object [12,29,30]. Such an iterative, nonlinear dynamical process is scale-free, capable of modeling sudden phase changes, and includes convergent, bifurcating, complex, and chaotic states. Although often modeling a system with only very few nodal points, the IPF has already been shown to be of high precision in musical instrument applications [30] as well as in rhythm perception and production [31].

The model was also formulated as a brain model [28] with neural adaptation and plasticity, non-trivially finding a concentration of inhibitory vs. excitatory neurons of 10–20%, corresponding to the real relation in the brain [32], as a maximum of possible system convergence. The model also finds a maximum reflection strength around the usual time for event-related potentials, as well as a decay in memory in the system corresponding to short-term memory. These general findings strongly point to the validity of the model.

Investigating the brain dynamics using large-scale musical forms has already shown that musical tension can be represented in the increasing synchronization of brain parts [1,2,33]. Such findings correspond to synchronization caused by expectancy [34] of a climax. This represents the development of tension over the large-range form of a musical piece, but it also might correspond to other semantic content like anger, anxiety, drama, relaxation, spirituality, or meditation. Such a relation might be found when referring to the lyrics of a musical song, discussed in the conclusion section below with respect to the piece investigated. Nevertheless, in EEG measurements, the main finding is that brain synchronization is strongest around 50 Hz, and thus in the gamma band of brain dynamics, although of course other brain rhythms exist [32].

An interesting aspect of the IPF Brain model is the presence of convergence, i.e., the complete synchronization of neurons in the model. This corresponds to epilepsy [35], which is not the usual state and is a very dangerous brain state. Still, partial synchronization in the brain is well known in chimera states [36]. Also, in terms of the auditory cortex, temporal lobe epilepsy has often been reported [37], and is associated with spiritual or meditative states of mind. Such states are naturally associated with longer time spans than pitch or rhythm and are subject to large-scale forms.

The present paper applies the IPF Brain model as suggested previously [28] to the case of an electronic dance music (EDM) piece One Mic by the rapper NAS, which has been investigated before in EEG experiments [1,2] by using a FitzHugh–Nagumo dynamical brain model [33]. Strong correlations between the large-scale form of the model, associated with the sound amplitude and fractal correlation dimension as a measure of musical density [29], have been found experimentally and in the model, with a maximum synchronization in the gamma band of brain dynamics of around 50 Hz in both cases. Still, due to the FitzHugh–Nagumo model not dissolving the temporal development of brain dynamics, a deeper understanding of the reason for this frequency-dependent synchronization requires a model capable of such a temporal resolution, which is represented by the IPF Brain model in this paper.

The paper first introduces the method used. As in the previous paper, the input to the Brain IPF is the output of a Finite-Difference Time-Domain (FDTM) physical model of the cochlea, for which the musical sound is used as an input. The output of the cochlear model is fed into the input of the Brain IPF. To estimate the influence of the amount of neurons used in the model, N = 50, 100, and 200 neurons are taken, showing the independence of the model with respect to this parameter. In the Section 2, the post-processing of these modeled parameters is discussed, especially the Kuramoto order parameter to measure the synchronization of neurons and the correlation of this parameter with the cochlea input. The Section 3 then mainly concentrates on this correlation and discusses the reason for the model behavior. In the Section 4, an overview of different conscious states with respect to the synchronization frequency is given.

2. Methods

2.1. Cochlea Model

The present model assumes the differential equation of a membrane

$$\frac{K\left(x\right)}{\mu \left(x\right)}\frac{{\partial }^{2}u}{\partial x^2}-d\left(x\right)\frac{\partial u}{\partial t}=\frac{{\partial }^{2}u}{\partial t^2}+f\left(t\right),$$

(1)

with basilar membrane displacement u along a one-dimensional axis x; basilar membrane stiffness $K\left(x\right)=2\times {10}^9{e}^{-3.4x}{\mathrm{dyn}/\mathrm{cm}}^3$, changing along the x axis; linear density $\mu \left(x\right)=m/A\left(x\right)$, with the mass over area again changing along the basilar membrane; and $A\left(x\right)=0.1\mathrm{cm}\times (0.1\mathrm{cm}+0.02\mathrm{cm}\times x/l)$ with basilar membrane length $l=3.5\mathrm{cm}$, taking the slight widening of the basilar membrane over its length into account.

A 1D model is sufficient to model a basilar membrane, as shown in [38], which compared 1D and 2D models based on the anisotropy of the basilar membrane as discussed by [39]. Here, it was found numerically and based on experimental data that the inclusion of a second dimension contributes less than 1% of the results already obtained by a 1D model. This is reasonable, as the basilar membrane has dimensions of about 3.5 cm in length but only about 1 mm in width and is therefore more a rod than a membrane. Also, the Young’s modulus in the y-direction is only about 10% of that in x-direction [38] and does therefore not add considerably to the overall basilar membrane movement.

To confirm this finding, a two-dimensional model was built:

$$\frac{K_x\left(x\right)}{\mu (x,y)}\frac{{\partial }^{2}u}{\partial x^2}+\frac{K_y\left(x\right)}{\mu (x,y)}\frac{{\partial }^{2}u}{\partial y^2}-d(x,y)\frac{\partial u}{\partial t}=\frac{{\partial }^{2}u}{\partial t^2}+f\left(t\right),$$

(2)

with a Young’s modulus in the x-direction of $K_x\left(x\right)=2\times {10}^9{e}^{-3.4x}{\mathrm{dyn}/\mathrm{cm}}^3$ as above and in the y-direction of $K_y\left(x\right)=0.1\times K_x\left(x\right)$ according to the literature [38]. The linear density $\mu (x,y)=mu\left(x\right)$ of the 1D model holds also for damping $d(x,y)=d\left(x\right)$. The model consists again of 48 modal points in the x-direction and 6 nodal points in the y-direction. The boundary conditions were again simply supported. Still, the results did not differ considerably [9]. For the sake of simplicity and also taking the computational cost into consideration, the 1D model was then used further.

The electronic dance music piece One Mic by the rapper NAS was used as a sound input to the cochlea model. The FDTM sample rate was 192 kHz to ensure model stability. The output is a set of spike time points $S_B^i$ at each of the 24 Bark bands B with i = 1, 2, 3 … $N_B$, where $N_B$ is the maximum number of spikes at Bark band B. Each $S_B^i$ has an associated amplitude of $A_B^i$. Although single spikes have a more or less uniform amplitude, the cochlea nerve fiber output of the model sums many spikes at each position $S_B^i$. Therefore, $A_B^i$ represents the amount of spikes or the output strength.

As input to the IPF Brain model, all outputs, as accumulated with respect to 20 ms time intervals, were used, corresponding to $f_{m}ax=$ 500 Hz, which is then the time constant of the IPF Brain model discussed below. Therefore, the frequency range of interest in the brain is well represented up to about 200 Hz. The resulting cochlea output time series is then denoted as

$$C_t=\sum _{B=1}^{24}\sum _{i=1}^{f_{max}\times T_{max}}G\left(A_B^i\right)H\left(S_B^i\right),$$

(3)

where $T_{max}$ = 266 s—the length of the musical piece—and the functions G and H detect if the respective spike is within the respective time window and gives $G\left(A_B^i\right)=A_B^i$ and $H\left(S_B^i\right)=1$ if so. For further correlation with the IPF Brain model output, $C_T$ is calculated as the time series averaged over time windows of 1 s. The unit-less mean amplitude of the $\overline{C_t}$ = 0.000118.

2.2. Brain Impulse Pattern Formulation (IPF)

The brain is modeled using N = 50 neurons. Each neuron is a reflection point, returning impulses from a viewpoint neuron. The system state of the viewpoint neuron is g, which represents a time period and an amplitude strength. Each reflection neuron i has a damping ${\alpha }_i$. The IPF is then

$$g_t=\frac{1}{N}\left|\sum _{i=1}^N{P}_{i}ln{\alpha }_t^i{g}_{t-i}\right|+w_2{g}_t^I.$$

(4)

Here, the viewpoint neuron is i = 1; the reflections come from neurons i = 2, 3, 4,… N. $P_i$ are the polarizations of the neuron, where $P_i$ = 1 is an excitatory neuron and $P_i$ = −1 is an inhibitory neuron. Throughout the paper, a relation of 10% of inhibitory neurons is used. Note that the sum of all reflections is normalized using the amount of neurons N. The model is discrete with time steps t = 0, 1, 2, 3… Therefore, the earlier states of the viewpoint neuron, which this neuron has sent out to the other neurons, return after a delay in a damped and polarized form.

For a deeper discussion of the model, see [28].

2.3. Plasticity Model

The plasticity of each neuron is calculated for each time step t. Plasticity refers to a change in the damping parameter ${\alpha }_i$, where each time step t then might have a different damping ${\alpha }_t^i$. Note that in the IPF reasoning, the damping is originally $1/\alpha$, which, for the sake of convenience, is skipped, using $\alpha$ instead.

For each time step, the new damping is calculated as

$${\alpha }_t^i=\left|{\alpha }_{t-1}^i+w_{1}ln(1+(g_{t-i}-g_t))\right|.$$

(5)

If the reflection point neuron t-i, $g_{t-i}$, has the same value as the viewpoint neuron value, $g_t$, the logarithm becomes zero, and no change in damping ${\alpha }_{t^i}$ happens. If the reflection point neuron $t-$i has a larger value than the viewpoint neuron, the logarithm becomes larger than zero and ${\alpha }_i$ increases. Otherwise, the logarithm assures a negative influence, and ${\alpha }_i$ decreases. The plasticity process is generally modeled using a constant $w_1$. Therefore, plasticity can be switched off in the model by using $w_1=0$. To examine different model behaviors, $w_1$ will systematically be altered, as shown below. Again, the absolute value of $\alpha$ is used, not allowing negative or complex values. This, again, does not change the model behavior due to the logarithms used. Still, positive values are more convenient. Indeed, negative arguments of the logarithm in the simulations shown below appear very rarely and are additionally suppressed by using the absolute value.

2.4. External Musical Instrument Input IPF

Like in a previous study [33], the electronic dance music piece One Mic by the rapper NAS has been used.

2.5. Detection of System Behavior

Each neuron has its own time series

$$g_t^i=P_{i}ln{\alpha }_t^i{g}_{t-i}$$

(6)

when reflecting back to the central neuron, as can be seen in Equation (4), where $g^i$ is the ith neuron at time point t with a certain ${\alpha }_t^i$ at that time point. These time series are Fourier-analyzed with time windows of one second, resulting in $F_T^i\left(f\right)$, where $0\le T\le 266$ s, i.e., the length of the musical piece. All IPF simulations in this paper were performed with a maximum of $f_{max}$ = 500 Hz; therefore, 1 ≤ f ≤ 500 Hz. The time series of the central neuron will be labeled below as simply g.

Synchronization was measured, as in a previous paper, using the Kuramoto order parameter

$$K_T\left(f\right)=\frac{1}{N}\left|\sum _{i=1}^N{e}^{ı{\theta }_T^i\left(f\right)}\right|,$$

(7)

where ${\theta }_T^i\left(f\right)$ is the phase of the ith neuron at time interval T of frequency f taken from $F_T^i\left(f\right)$. N is the amount of neural reflection points; in this study, N = 50, N = 100, and N = 200. The Kuramoto order parameter is used as it is the most widely accepted synchronization measurement parameter. It holds that 0 $\le K_T\left(f\right)\le$ 1 with $K_T\left(f\right)$ = 0 in the case of no synchronization and $K_T\left(f\right)$ = 1 in the case of maximum synchronization.

The synchronization order parameter $K_T\left(f\right)$ is time-dependent. To estimate the overall synchronization strength from $K_T\left(f\right)$, a time-averaged mean $K\left(f\right)$ is calculated.

Also, the correlation of $K_T\left(f\right)$ with the cochlea input time series $C_T$ is performed, leading to $K^C\left(f\right)$, an estimation of the frequency dependency of the correlation strength of the synchronization with the time series.

The Finite-Difference Time-Domain model of the cochlea was implemented using C++, C#, and CUDA code in Visual Studio software 2012 and 2017. CUDA code implements the model on an NVIDIA Graphic Processing Unit in parallel with calculation of the model nodal points. The IPF model was run and the results analysis was performed using Mathematica 12 software.

3. Results

To estimate the influence of the amount of neurons used, an analysis was performed for N = 50, as in a previous study, as well as N = 100 and N = 200.

3.1. System Behavior

At first, the system parameter g is expected to follow the cochlea input systematically. Figure 1 shows the cochlea input $C_T$ at the bottom (blue) and the system parameter g of the central neuron (yellow) at the top. The simulation was performed for N = 50, N = 100, and N = 200 neural reflection points, and the figure shows the N=100 case. Visually, the time series correspond well. The correlations are 0.37 for N = 50, 0.38 for N = 100, and 0.40 for N = 200. This is the first example of a high independence of the results with respect to the amount N of neural reflection points.

3.2. Gamma Band Synchronization Strength

The IPF Brain model is expected to reproduce experimental data. It was found that the strongest correlation between brain synchronization and musical sound input is in the gamma band around 50–80 Hz. Below and above the gamma band, synchronization is still there, but decreases considerably.

Figure 2 shows $K^C\left(f\right)$, the correlation of the Kuramoto order parameter with cochlea input $C_T$ for frequencies up to 200 Hz for the three reflection points N = 50, N = 100, and N = 200. As expected, there is a peak in $K^C\left(f\right)$ around 50 Hz, with decreasing correlation both above 50 Hz, up to about 90–100 Hz, and below 50 Hz. This is consistent over N = 50, 100, and 200, again pointing to the independence of the choice of N. Still, the correlation for N = 50 is slightly lower than for N = 100 and N = 200, a tendency already found above.

As an example of the correspondence between $K^C\left(f\right)$ and $C_T$ in Figure 3, the time series of $C_T$ and $K^C\left(50\right)$, so at 50 Hz for the N = 100 case, is shown. Clearly, synchronization follows the amplitude of the cochlea input, the musical piece, nicely. The synchronization is not as smooth as found experimentally; however, the model is only about listening to this musical piece while the EEG experimental data measure the whole brain performing many other tasks at the same time.

Correlations decrease up to 90–100 Hz and increase again at higher frequencies. Oscillations higher than about 100 Hz are not considered within ‘classical’ bands like delta (0.5–3.5 Hz), theta (4–7 Hz), alpha (8–12 Hz), beta (13–30 Hz), and the gamma band, which is found at frequencies above 30 Hz [40], or split into the lower gamma range (30–60 Hz) or higher gamma range (60–120 Hz) [41]. Frequencies above 120 Hz are referred to as fast oscillations [32] and are only briefly associated with conscious content.

Looking for an explanation of the gamma band strength, we could expect some connection between the frequency dependency of $K^C\left(f\right)$ and the cochlea input $C_t$ or the system parameter g. However, connections in terms of their spectral amplitudes are not straightforward. In Figure 4, both spectra are shown. Both spectra are very similar, pointing to a response of the Brain IPF to its input, as expected. Indeed, at around 50 Hz, a peak is present in both time series, corresponding to the 50 Hz peak of $K^C\left(f\right)$. Still, the 50 Hz peaks in $C_t$ and g are much sharper. Also, $C_t$ and g spectra have a wider peak around 90–100 Hz, while there is a gap in the correlation with Kuramoto synchronization in this region.

Furthermore, both the spectra of $C_t$ and g have an enhanced energy in the low frequency range, pointing to the rhythm content of the musical piece. The piece is 92 BPM, corresponding to 1.53 Hz. During the high-amplitude sections of the piece, semiquaver notes are played, resulting in 6.1 Hz; during the low-amplitude sections, the piece uses mainly quaver notes at 3.1 Hz. Thus, much energy is expected below about 6 Hz, which is the case.

Again, connections to $K^C\left(f\right)$ are not straightforward, as below about 10 Hz, $K^C\left(f\right)$ is mainly negative. This is not found experimentally and is surprising at first. Still, when examining an example of N = 100 at 2 Hz, as shown in Figure 5, the reason becomes clear. During times of low cochlea input, amplitude synchronization is very strong, while it is lower when the cochlea input is stronger. In a previous study on the principle behavior of the IPF Brain model [28], it appeared that at a relation of 10% of inhibitory vs. excitatory neurons in the brain, an optimum in terms of convergence or stability of the system is reached. Therefore, we expect the system to converge when a low external input is presented, which is the case here. This leads to a systematic increase in synchronization at times of low external input and therefore to a negative correlation.

For comparison, an IPF Brain model fed with white noise was studied to see if the low-frequency behavior might be a systematic bias of the model. However, over the whole frequency range, low correlations around zero were found, with slightly positive and negative values and no frequency dependence, as expected.

Such convergence would correspond to epilepsy, as all neurons are synchronized. Indeed, epilepsy is reported in the auditory cortex, associated with creativity or spirituality [37]. Such non-everyday experiences are known to occur as a trance, often associated with either long periods of isochronous rhythm or during silence. Both are present in the musical piece during times of low cochlea input. Of course, as discussed above, the model is only about listening to a musical piece without performing other tasks like a real brain does. Still, temporal lobe epilepsy is only present at the auditory cortex, and therefore, the negative correlations might model such a case.

Indeed, in the EEG data, there is enhanced synchronization at such low frequencies, which is again not found in the FitzHugh–Nagumo model [33]. This synchronization is positive and not negative. This might be due to the EEG data being taken from all regions of the human skull, while the IPF Brain model is not specifying brain regions and therefore is also suitable to represent the auditory cortex only. The enhanced synchronization at lower frequencies might also be caused by motor action taken in the brain at low frequencies in the body movement dancing range.

3.3. Scaling Law

EEG data of the brain show a scaling law in the brain spectrum. Plotting this spectrum in a log–log plot exhibits a constant slope over a wide range of frequencies [24]. This corresponds to a so-called 1/f-noise, as the spectrum follows a $1/f^{\alpha }$ rule, where a fractal dimension of $\alpha$∼2 is found. The scaling law points to a close and systematic connection between the different frequency regions of brain activity and the self-organizing nature of the brain.

The IPF Brain model also shows such a scaling law, as shown in Figure 6. Experimental EEG data also show a plateau or even a positive slope up to about 5 Hz, consistent with the IPF Brain findings. The rippling of the spectrum indicates enhanced activity in certain regions like with the gamma band found already above. The only aspect not corresponding to EEG data is the fractal dimension of the IPF at ∼0.43, rather than the expected ∼2. The reason here is unclear and will be analyzed in future.

4. Conclusions and Discussion

The association of temporal lobe epilepsy with spiritual or meditative experiences corresponds well with the intention of the musical piece One Mic used in this investigation. The lyrics report about the hard struggle in criminal gangs with police interaction and shootings. Several references are made to spiritual, especially Christian, symbols and similar ways of suffering. Such lyrics are rapped during the large amplitude time frames of the piece, i.e., the verses, where also a police siren can be heard. These sections are followed by the low-amplitude parts, where the refrain All I need is one mic is repeated, pointing to music and lyric production as an alternative or a weapon against such a hard struggle. These sections contrast the verses, as they are presented in a contemplative or meditative way.

This compositional tool of presenting a meditative alternative by reducing the volume, reducing the beat from semi-quavers to quavers, and omitting most of the sound effects and keyboard pads of the verse is shown in this paper to produce strongly enhanced synchrony and convergence of the neural network. This synchrony is found in the low frequency range, where the musical rhythm is represented.

Such musical structures are very simple and are present in many musical pieces, where regions of low volume and isochronous rhythm are present. Further investigations are needed to model and measure the neural reaction to such musical content in more detail. Still, due to the simplicity of this compositional tool, one can expect composers and musicians to use it in many musical scenarios.

In a previous similar EEG study of a musical piece, Classical Symphony by Shemian, increased brain synchronization was found towards an expectancy point, after which synchronization decreased again [1]. This is a typical electronic dance music piece in that a tension is built up by compositional tools like increased amplitude and event density, only to climax at a point where the dense structure ends and a four-to-the-floor bass drum starts. Again, this is a typical compositional tool in electronic dance music, a tension build-up and decay repeated dozens of times during a song. This compositional tool is clearly present again in the EEG data here in its large-scale musical form.

In this study, brain synchronization followed the reasoning of a coincidence detection mechanism of cortical oscillators, modeling neural activity in the striatum [34]. After the start of neural oscillation, increased synchronization peaks occur at the point a subject expects an event to happen, like waiting at a traffic light to turn green. The oscillation is then expected to include motor regions, making us nervously shake towards the expected time point.

This might be considered another compositional tool to make people dance. A tension build up, leading to a neural oscillation and including the motor region, enhances the will of subjects to move and, in the case of music, to dance. Although there is strong evidence that this is the case, there is no final proof, as measurements in the motor region are still missing in the musical case.

This line of reasoning seems fundamentally different than that followed in the present paper of neural synchronization leading to a meditative mood. Still, both cases are confirmed experimentally, as is the case of temporal lobe epilepsy [37]. The difference might indeed be found in the different frequency ranges, where increased synchronization at higher frequencies, around 50 Hz, might cause the perception of increased tension and synchronization at low frequencies, contrary to that of a meditative state. Taking into account that synchronization and de-synchronization are fundamental activities in the brain, and brain activity represents all possible states of mind, perception, and consciousness, such a differentiation seems plausible.