1. Introduction
Since the mid 1990s, mathematical modelling of human virus infections has been intensively developed for various infections including those with the human immunodeficiency virus type 1 (HIV) and the hepatitis B virus. Reviews on existing approaches are presented, i.e., in [
1,
2,
3,
4]. Mathematical modelling of virustarget cell interactions based on the understanding of the underlying biological processes can provide deeper insights into the mechanisms of the infection dynamics, see [
3,
5,
6,
7]. The majority of these studies are based on deterministic descriptions with continuous populations of virus particles (virions) and target cells. This approximation is accurate when the number of virions, infected cells and other interacting components are large enough that is typical for the later stages of an infection process. However, this deterministic framework has its drawbacks because an infection process is intrinsically stochastic. Populations of interacting species undergo stochastic fluctuations that are more profound at the early stage of an infection when the number of virions and infected cells are still small. Furthermore, random fluctuations may lead to extinction of an infection process. Such effects are well suited for being analyzed with methods of stochastic processes theory.
The stochasticity of a virus infection may be accounted for in models using stochastic differential equations (SDEs) governed by Brownian motions (BMs), i.e., [
8,
9,
10,
11]. This modelling approach seems more realistic and can explain phenomena like virus extinction (cf. [
12]). However, the applicability of such an approach is restricted by the daunting task of parameter identification for processes like volatility or the probability density function (PDF). For example, the volatility of the Brownian process varies with the population size of interacting components and that is not easy to account for.
Another approach to model stochastic infection dynamics is the use of a discrete or continuous Markov Chain (MC) in the framework of the MonteCarlo method. This approach and algorithms for numerical simulations have been developed first for chemical kinetics (cf. [
13,
14]). Pearson et al. [
15] considered a twocomponent model for early infections with the use of a MC. A similar simulation of the ’consensus’ threecomponent virus dynamics model (cf. [
2]) was proposed and studied in [
16]. In such an approach, there is no necessity to work out parameters of Brownian motion or other processes because the transition rates (propensities) for the Markov chain are determined directly from parameters of the deterministic equations. A comparison between the SDE approach and the discrete and continuous MC approach for a simple population dynamics model can be found in [
17].
Insightful models of virus infections may take the intracellular phase of the virus lifecycle into account and thus, the time delay between infection of a cell and the production of new virus progeny (cf. [
18]). A deterministic model with a fixed time delay has been proposed by Herz et al. [
19] (see also [
20,
21]). A more sophisticated model accounting for two distributed time delays was developed in [
22]. Stochastic modelling of infection dynamics with fixed and distributed time delays based on the SDE approach can be found in [
11,
23,
24].
In our present work we generalized the ideas of [
16] on a model with time delay and used a Markov Process (MP) with a fixed delay instead of a Markov Chain (MC) considered in [
16]. This complicates computation but makes the model more realistic. We also propose a hybrid model for effective computation of statistical parameters of the stochastic virus dynamics.
In
Section 2, we describe the wellknown deterministic model of virus infection dynamics with delay. The related stochastic model is presented in
Section 3. In
Section 4, the hybrid modelling algorithm is developed. We summarize the results in
Section 5.
2. Deterministic Model
The standard and classic ODE system for the three species virus dynamics has been introduced in [
25,
26] (see also [
1,
2,
19]):
where
x,
y and
v are, respectively, the numbers of uninfected cells, infected cells and the number of free virus particles (virions) in a fixed volume compartment. The authors suppose that uninfected cells are produced at a constant rate,
$\lambda $, and die at a rate
$d\phantom{\rule{0.56905pt}{0ex}}x$; virions infect uninfected cells at a rate proportional to the product of their numbers,
$\beta \phantom{\rule{0.56905pt}{0ex}}xv$; infected cells produce free virus at a rate
$k\phantom{\rule{0.56905pt}{0ex}}y$; infected cells die at a rate
$a\phantom{\rule{0.56905pt}{0ex}}y$; free virus particles are removed from the system at a rate
$u\phantom{\rule{0.56905pt}{0ex}}v$.
Herz et al. [
19] have modified ODEs (1) by including the delay accounting for a latent period between the time when target cells are contacted with virions and the time when proviral DNA integrates into the cellular genome and proposed the following DDE system
where
$\tau $ is the time lag between the fusion of the virion with the cell membrane and integration of proviral HIV DNA in the cell genome. The term
$\beta \phantom{\rule{0.56905pt}{0ex}}x(t\tau )\phantom{\rule{0.7113pt}{0ex}}v(t\tau ){e}^{a\tau}$ indicates that the secretion rate for infected cells is proportional to the number of uninfected cells and virions at the time
$t\tau $ decreased by factor
${e}^{a\tau}$ because of natural and immunemediated death infected cells with the rate
a in time
$\tau $. Variables in all other terms are considered at time
t.
Ciupe et al. [
27] have proposed a four species model with account for the second type delay: the time lag between virion’s DNA penetrates the cell and new virions are produced and released. We present here a similar, three species DDE version:
Here,
$\tau $ is the time delay between penetration of a virion into a cell and release of new virions. Term
$k\phantom{\rule{0.56905pt}{0ex}}y(t\tau ){e}^{a\tau}$ indicates that the growth rate for virions is proportional to the number of cells infected by virus at the time
$t\tau $ decreased by a factor
${e}^{a\tau}$ because of natural and immunemediated death of infected cells with the rate
a in time
$\tau $.
A more general model accounting of four species interaction for the both types of delay has been proposed by Pawelek et al. [
22]. Its three species version is studied in [
28].
Applying results obtained in [
28] to DDEs (3), we can show that the basic reproduction number of the model is
The model has two equilibrium states: the infectionfree equilibrium
$\{x={x}_{0},y=0,v=0\}$ where
${x}_{0}=\lambda /d$ and the infection equilibrium
$\{x={x}_{\infty},y={y}_{\infty},v={v}_{\infty}\}$ where
The infection equilibrium exists and is asymptotically stable if
${R}_{0}>1$.
Let
${v}_{0}$ virions arrive at instant
$t=0$. It is natural to take
${x}_{0}=\lambda /d$ as an initial number of uninfected cells in the virus dynamics and to assume that there were no infected cells before first virions arrived. Then we have the following initial conditions
For DDEs it is necessary to set also the history function defined in the interval
$[\tau ,0)$:
Equation (3) with initial conditions (
6) and history (
7) form a complete initial value problem (sometimes called the initial data problem) for the DDEs and uniquely define the virus dynamics for
$t>0$. They can be numerically integrated, for example, with the use of Matlab function
dde23().
Nowak and May [
2] have elaborated typical parameters for a typical HIV virus infection process:
These parameters have units of inverse days:
${\mathrm{d}}^{1}$. These values have been used in a number of works including our previous work [
16].
Examples of computations of the DDEs with parameters (
8) and different types of delay are shown in
Figure 1. The same value
$\tau =1$ d is used in both cases: for the first (2) and the second (3) types of delay. We denote the solution for the first case as
${x}_{1}\left(t\right),{y}_{1}\left(t\right),{v}_{1}\left(t\right)$ and for the second case as
${x}_{2}\left(t\right),{y}_{2}\left(t\right),{v}_{2}\left(t\right)$.
Observe that for the same delay, the dynamics in both cases is identical for the number of uninfected cells ${x}_{1}\left(t\right)\equiv {x}_{2}\left(t\right)$ and the number of virions ${v}_{1}\left(t\right)\equiv {v}_{2}\left(t\right)$ for all t, whereas the dynamics of infected cells differs by time shifting and scaling: ${y}_{1}\left(t\right)={y}_{2}(t+\tau ){e}^{a\tau}$. This means that contribution of both delays is similar into the virus dynamics.
This fact motivates a detailed study of the model with the second type delay (3) to build an equivalent stochastic model. Building of a stochastic model with the first type delay (2) is more involved and will be considered in the subsequent works.
4. The Hybrid Stochastic Model
The direct numerical simulation is rather time consuming because of a large number of species to be accounted: $X=O\left({X}_{0}\right)\sim {10}^{6},{V}_{max}>{X}_{0}$. This causes the time steps between events to be very small. To compute statistical characteristics of the infection process, it is required to run a huge number of realizations especially for evaluation of the infection/extinction probability. Therefore one needs to consider methods to accelerate the computation process.
Plots in
Figure 2a show that the randomness in realizations appears mainly due to the relatively high fluctuations at the beginning of viral dynamics while the numbers of virions and infected cells are not large. The curves become rather smooth at a later time, the numbers of all dynamic participants are large (several orders), and the fluid dynamics limit works rather well. This dynamics is typical for an infection process with small initial number of virions (
${V}_{0}\ll {X}_{0}$): see [
16] for viral dynamics and [
35,
36,
37,
38] for an epidemic outbreak. Therefore, we can employ the hybrid modelling approach developed in [
16,
35,
37] in which the stochastic dynamics is to be split into two main phases: a genuine stochastic and quasideterministic ones.
Phase 1 is the time interval, at which the numbers of virions and infected cells are not large, and hence, the stochasticity of interaction between species has to be taken into account. The number of uninfected cells is large enough and its randomness is not essential in all the phases. Its global variation remains also small for long time after infection starts. The last fact enabled the authors of [
16] (similar to what is done in [
35,
36,
37,
38]) to simplify description of the earlier phase model by excluding the uninfected cells dynamics from the total process approximating the number of uninfected cell by its initial value:
$X\approx {x}_{0}$.
In a developed infection process, populations $Y,V$ reach large numbers at a certain instant ${t}_{\star}$. Thus all the populations become large enough that the total process can be switched to a deterministic behaviour described by DDE (3) (phase 2).
Here we propose a modified method for phase 1 in which the number of uninfected cells $x\left(t\right)$ is not approximated by the constant ${x}_{0}$ but can vary obeying a differential equation. This makes the algorithm more flexible as at the switching point the number of uninfected cells can be differ significantly from it initial value ${x}_{0}$ (remaining $X\gg 1$).
This coupled deterministicstochastic process can be described as follows
Process (
Table 2) is a reduced MP with delay. Differential Equation (
23) has a stochastic parameter
$V\left(t\right)$.
In this approach switching from simplified stochastic process (
23) to deterministic process (3) occures at the time
$t={t}_{\star}$ at which the following weaker condition should be fulfilled:
and, therefore,
$x\left({t}_{\star}\right)$ can be essentially different from
${x}_{0}$. Here
${X}_{\star}$ is a threshold—the minimum number of a component at which its dynamics can be accurately approximated by a deterministic model. For example, in the computations below, we use
${X}_{\star}={10}^{3}$.
The initial conditions for process (
23) are similar to (
10):
The history is also described by (
12).
The convergence of MP (
23)–(
25)–(
12) to the deterministic process described by (3)–(
6)–(
7) is similar to that for MP (
Table 1)–(
10)–(
12).
Stochastic process (
Table 2) has been implemented in the same way as MP (
Table 1) and described in Algorithm 2. The state vector becomes
$\mathit{X}={[Y\left(t\right),V\left(t\right)]}^{T}$ and the matrix of events has the size
$N\times M=2\times 4$:
The augmented vector for this process is
$\stackrel{\u02c7}{\mathit{X}}=[Y\left(t\right),V\left(t\right),Y(t\tau )]$. The propensity vector
$\mathit{\nu}={[{\nu}_{1},\dots ,{\nu}_{M}]}^{T}$ now depends on
x and
$\stackrel{\u02c7}{\mathit{X}}$. The dependence is indicated in
Table 2. All other vectors
$\mathit{J}={[{J}_{1},\dots ,{J}_{M}]}^{T}$,
$\mathit{R}={[{R}_{1},\dots ,{R}_{M}]}^{T}$,
$\mathit{t}={[{t}_{1},\dots ,{t}_{M+1}]}^{T}$ used in Algorithm 1 are the same with the new
$M=4$.
The twostep predictorcorrector method is implemented to integrate ODE (
23)
where
$\Delta {t}_{i}={t}_{i+1}{t}_{i}$ is the time interval between two subsequent events in process (
Table 2);
${x}_{i}=x\left({t}_{i}\right),{V}_{i}=V\left({t}_{i}\right)$.
Algorithm 2 Implementation of the hybrid Markov process with delay. 
Set the current time $t=0$ and the final time of the process ${t}_{f}$; Set $x\left(0\right)={x}_{0}$ and initialise the augmented state vector $\stackrel{\u02c7}{\mathit{X}}={[0,{V}_{0},0]}^{T}$ and vector $\mathit{J}$: ${J}_{m}=0,m=1,\dots ,M$; Compute the propensity vector $\mathit{\nu}(x,\stackrel{\u02c7}{\mathit{X}})$ using values of propensity indicated in Table 2; Generate M random numbers ${r}_{m}$ and compute vector $\mathit{R}$: ${R}_{m}=ln\left({r}_{m}\right),\phantom{\rule{0.277778em}{0ex}}m=1,\dots ,M$; Compute the time steps to the next event in all the transitions: $\Delta {t}_{m}={R}_{m}/{\nu}_{m},\phantom{\rule{0.277778em}{0ex}}m=1,\dots ,M$ and set $\Delta {t}_{M+1}=\infty $; Allocate arrays ${\mathit{t}}^{\mathrm{Y}}$ and ${\mathit{y}}^{\mathrm{Y}}$, set ${t}_{1}^{\mathrm{Y}}=0$, ${y}_{1}^{\mathrm{Y}}=0$; set counters ${i}_{1}=1$, ${i}_{2}=0$ for these arrays; Find transition p with the minimal time step: $\Delta {t}_{p}=min\{\Delta {t}_{1},\dots ,\Delta {t}_{M+1}\}$; Update vector $\mathit{J}$: ${J}_{m}={J}_{m}+{\nu}_{m}\phantom{\rule{0.166667em}{0ex}}\Delta {t}_{p}$, $m=1,\dots ,M$; Update the current time $t=t+\Delta {t}_{p}$; If$p<M+1$then  (a)
set $\Delta x=\lambda d\phantom{\rule{0.56905pt}{0ex}}x\beta \phantom{\rule{0.56905pt}{0ex}}{\stackrel{\u02c7}{X}}_{N}\phantom{\rule{0.56905pt}{0ex}}x$ (recall ${\stackrel{\u02c7}{X}}_{N}=V,N=2$)  (b)
set $\tilde{x}=x+\Delta {t}_{p}\phantom{\rule{0.56905pt}{0ex}}\Delta x$  (c)
update the state vector $\stackrel{\u02c7}{\mathit{X}}$: ${\stackrel{\u02c7}{X}}_{n}={\stackrel{\u02c7}{X}}_{n}+{T}_{np},n=1,2$  (d)
set $x=x+\frac{1}{2}\phantom{\rule{0.56905pt}{0ex}}\Delta {t}_{p}\phantom{\rule{0.56905pt}{0ex}}\left(\Delta x+(\lambda d\phantom{\rule{0.56905pt}{0ex}}\tilde{x}\beta \phantom{\rule{0.56905pt}{0ex}}{\stackrel{\u02c7}{X}}_{N}\phantom{\rule{0.56905pt}{0ex}}\tilde{x})\right)$  (e)
set ${J}_{p}=0$  (f)
if$p=1$ or $p=2$ (transition in which Y has been changed) then set ${i}_{2}={i}_{2}+1$, ${Y}^{\mathrm{Y}}\left({i}_{2}\right)={\stackrel{\u02c7}{X}}_{1}$, ${t}^{\mathrm{Y}}\left({i}_{2}\right)=t+\tau $;
If$p=M+1$then  (a)
update the state vector $\stackrel{\u02c7}{\mathit{X}}$: ${\stackrel{\u02c7}{X}}_{M+1}={Y}_{{i}_{1}}$  (b)
set ${i}_{1}={i}_{1}+1$;
If${i}_{2}<{i}_{1}$ and ${\stackrel{\u02c7}{X}}_{1}={\stackrel{\u02c7}{X}}_{2}=0$ then flag the realisation as extinct and terminate the computation. Update the propensity vector $\mathit{\nu}=\mathit{\nu}(x,\stackrel{\u02c7}{\mathit{X}})$; If$p\le M$then generate random number r and compute ${R}_{p}=ln\left(r\right)$; Update the time step vector $\Delta {t}_{m}=({R}_{m}{J}_{m})/{\nu}_{m}$, $m=1,\dots ,M$; If${i}_{2}\ge {i}_{1}$then set $\Delta {t}_{M+1}={t}^{Y}\left({i}_{1}\right)t$ else set $\Delta {t}_{M+1}=\infty $; Store the current state and time; If$t\ge {t}_{f}$then terminate the computation. If$min\{{\stackrel{\u02c7}{X}}_{1},{\stackrel{\u02c7}{X}}_{2}\}\ge {X}_{\star}$then set ${t}_{\star}=t$ and switch to phase 2 otherwise go to 7.

If the process has not died out, the computation is continued till time
${t}_{\star}$ when condition (
24) is satisfied. For
$t\ge {t}_{\star}$ the process can be switched to a deterministic one described by DDE (3) with the initial conditions
at time
$t={t}_{\star}$ where
$x\left({t}_{\star}\right),Y\left({t}_{\star}\right),V\left({t}_{\star}\right)$ are the final results of computation (
23).
For DDE (3) we have to define also the history $y\left(t\right),t\in [{t}_{\star}\tau ,{t}_{\star})$. It is complicated to employ the computed stochastic piecewise function $Y\left(t\right)$ as it contains too many parameters and its use requires elaborate search within intervals between its discontinuities. Instead we employ an approximate solution of DDEs (3) valid at early stage of viral dynamics with $x\left(t\right)=const={x}_{0}$.
Substituting
$x={x}_{0}$ into DDEs (3) we reduce it down to two equations
In contrast to the case
$\tau =0$ considered in [
16], Equation (29) do not admit a solution in the closed form for all times and should be integrated numerically. Nevertheless, in the interval
$[0,\tau ]$ the solution is explicit: here the number of virions decreases exponentially:
$v(t\le \tau )={v}_{0}exp\{ut\}$,
$y(t\le \tau )\equiv 0$.
DDEs (29) admit an exponential (selfsimilar) solution
where
C is a constant determined by
${v}_{0}$,
$\alpha $ is a positive solution to the equation
Dependance of
$\alpha $ on
$\tau $ for parameters (
8) is shown in
Figure 7.
Solution (
30)–(
31) describes an intermediate asymptotic behaviour in the interval between the initial decreasing of the virions number and approaching the infection peak. This asymptotics is clearly seen in
Figure 1. Almost linear parts of the curves for
$y\left(t\right)$ and
$v\left(t\right)$ plotted in logarithmic scale indicate an exponential growing function. An almost exponential infection growth is also seen in the plot of a nonextinct realization in
Figure 2a.
Therefore we approximate it by an exponential function utilising the selfsimilar solution (
30) at stage 2 by selecting time
${t}_{\star}$ such that the instant of the history beginning,
${t}_{\star}\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}\tau $, belongs to stage 2. To this aim we solve Equation (
31) with respect to parameter
$\alpha $ and approximate function
$y\left(t\right)$, necessary for the history, as
Computations show that the use of the hybrid model accelerates the process by several orders. For example, the full model with ${x}_{0}={10}^{6},{v}_{0}=8,\tau =0.4,{t}_{final}=20$ d needs 32.2 s of the CPU time on Intel© Core™ i77500U CPU 2.70GHz×2 whereas the hybrid version is computed in 3 ms on average.
An example of several nonextinct realizations computed by the hybrid method for
${V}_{0}=100$,
$\tau =1$ d and
${X}_{\star}={10}^{3}$ is shown in
Figure 8. Deterministic part of the curves computed by integration DDEs (3) (phase 2) are drawn by lighter colours. Switching from phase 1 to phase 2 is indicated by circles on the curves for
$Y\left(t\right)$ and
$V\left(t\right)$. In these realizations, the switching occurs when
$Y=min\{Y,V\}={X}_{\star}={10}^{3}$. Value of
V in these realizations is greater:
$V\left({t}_{\ast}\right)\approx 5\times {10}^{3}$. Thus,
${t}_{\star}$ varies for different realizations.
Figure 8 illustrates that the stochastic dynamics in phase 1 causes the scattering of the deterministic curves depicting the infection dynamics in phase 2.
The statistical characteristics, median and interdecimal range, for nonextinct realizations obtained by the hybrid method are shown in
Figure 9 for all populations for
${V}_{0}=20$ and
$\tau =1$ d. The notations are the same as in
Figure 5. Here for comparison, the same statistical characteristic obtained using the direct simulations (see
Figure 5b) are plotted by dashdot (mean) and thin solid (IDR) lines by a darker colour. Observe that both models give the same statistical characteristics with a high accuracy.
The infection probability
${P}_{i}$ computed by the hybrid method is shown in
Figure 3 by open black circles and black dashed lines. Here it is also seen that the hybrid method provides high enough accuracy for computation of this important characteristic of infection process.
Another important characteristics of infection process is the time of infection development that can be defined as the time lag between first virions arrived and peak of the infection. It is a challenging problem to determine peak of a stochastic process as it can have high local fluctuations shifting the global maximum from expected position. The paper [
35] employs the Gaussian processes to fit the model. The hybrid model allows computation the maximum of the number of infected cells,
${Y}_{max}$, or the number of virions,
${V}_{max}$, easily as the peak is located on the smooth deterministic part of the curve (phase 2). The mean value and standard deviation (SD) of
${Y}_{max}$ and
${V}_{max}$ versus time delay for different initial number of virions
${V}_{0}$ are shown in
Figure 10. The averaging is performed over all nonextinct realizations. For comparison, analogous curves obtained by deterministic modelling are indicated as well. The second scenario:
${k}_{\tau},{R}_{0}=const$ is considered in these simulations. In the first scenario:
${k}_{\tau},{R}_{0}\propto {e}^{a\tau}$ (
9) we obtain very large peak time up to
${10}^{3}$ d.
Observe that the mean peak is shifted to earlier time for small number of initial virions (1 and 10) and not noticeable for larger numbers (${10}^{2}$ and ${10}^{3}$). Especially this shift is large for ${V}_{0}=1$, so that the corresponding curve almost coincides with that for ${V}_{0}=10$, therefore the later one is drawn by the dashed line. The coefficient of variation (SD over mean) is about 5%–6% for ${V}_{0}=1,10,{10}^{2}$ and about 1% for $V={10}^{3}$. Therefore the SD patch is very thin and not visible for ${V}_{0}={10}^{3}$ in this scale.
5. Conclusions
In this work, a stochastic viral dynamics model with the time lag between virions production and infection of cells is developed on the base of a Markov process with a time delay. The model has been computationally implemented and studied numerically. The model provides a useful tool to calculate statistical characteristics of virus infection dynamics.
The key statistical characteristics of a virus infection process has been computed: variation of the mean, median and interdecimal range (IDR) for all variables in time, the infection and extinction probabilities and time of the infection development. The dependence of the infection dynamics on the time delay between cell infection and virus progeny production has been studied. Two approaches to incorporate time delays have been tested: (a) the fixed parameters approach in which the reproduction number exponentially decays with the growth of the time delay, and (b) the constant reproduction number approach in which the parameters of the model are adjusted to preserve a constant reproduction number. It is shown that with the second approach, the extinction probability is independent of time delay but the infection peaktime grows nonlinearly with the increase of the time delay and differs from the peaktime predicted by the deterministic model for a small number of initial virions. It is also shown that for a small number of initial virions, the number of virions during the infection process, averaged over the nonextinct realizations, exceeds that number calculated via the deterministic model.
A novel and fast computational algorithm to simulate the viral dynamics based on a Markov process with time delay has been proposed, implemented and compared with the full stochastic MP model. In this hybrid model, the dynamics of the components with large numbers of state variables is computed by integration of the ODE/DDE. This essentially accelerates the simulation and computation of the process statistics parameters. It is shown numerically that this hybrid modelling algorithm provides appropriate accuracy in computation of the statistical parameters.
In subsequent work, the full and the hybrid scheme has to be generalized by accounting for a greater number of interacting components (like in the model described in [
39]), for a distributed delay, and for more than one delay. This will enable to develop more realistic virus infection models that shall help to better understand regulation and sensitivity of the underlying biological processes.