A Semi-Deterministic Random Walk with Resetting

Abstract

We consider a discrete-time random walk $\left(x_t\right)$ which, at random times, is reset to the starting position and performs a deterministic motion between them. We show that the quantity $Pr\left(x_{t+1}=n+1|x_t=n\right),n\to \infty$ determines if the system is averse, neutral or inclined towards resetting. It also classifies the stationary distribution. Double barrier probabilities, first passage times and the distribution of the escape time from intervals are determined.

1. Preliminaries

In a previous paper [1], we introduced the Sisyphus random walk as an infinite Markov chain that moves on the space state $\mathbb{N}=\{0,1,2,\dots ,\infty \}$ and that, at every step, can either jump one unit rightward or return to the initial state from where it is restarted. The system was named after the king of Ephyra, Sisyphus, who was condemned to lift a heavy stone in an endless cycle.

Here, we generalize the above idea and consider a random walk on the integers ${\left(x_t\right)}_{t\in \mathbb{N}}$ whose dynamics alternates deterministic linear motion with resets which drive the system to the starting point at the random times ${\left({\mathbf{t}}_n\right)}_{n\ge 1}$. At every clock tick, the position of the random walker is such that $|x_t|$ either increases one unit, or returns to the ground state, whereupon the evolution continues. Such resetting occurs through an independent mechanism superimposed onto the original semi-deterministic evolution. Once $\left(x_t\right)$ is reset to the origin at ${\mathbf{t}}_1$, it begins the evolution anew from scratch which is deterministic between resets.

Using translational invariance, we can suppose that $x_0=0$ with no loss of generality. Concretely, starting from $x_0=0$, three possibilities exist for the future position $x_1$: the system may remain at $x_1=0$ provided a reset occurs at $t=1$; otherwise, it goes one unit to the right with probability $\rho$ or to the left with probability $\overline{\rho }=1-\rho$. In addition, if the system has wandered into the positives so that at a certain time $t\ge 0$ is $x_t>0$ (respectively $x_t<0$) then, at time $t+1$, it may either be reset to the origin $x_{t+1}=0$ with arbitrary probability or else increase (respectively, decrease) one unit to $x_{t+1}=x_t+1$ (respectively $x_{t+1}=x_t-1$).

Such apparent simplicity is misleading, as this simple evolution law can exhibit a surprisingly complex and rich behavior. Indeed, at each site, we allow arbitrary probabilities for the random walk to reset to the origin and, additionally, the possibility to move both in the positive and negative integers. The only restriction in this general dynamics is the requirement that ${\left(x_t\right)}_{t\in \mathbb{N}}$ be a Markov chain. The resulting system is a natural, non-trivial generalization of that of [1], which is recovered when the reset probability is independent of the location and when $\rho =1$.

In a different setting, such a system may be used as an idealized model of the random dynamics of a “mobile” in a trap, say, who is trying to climb stepwise a ladder or wall given that, at every step, there is a probability of slipping to the bottom, resulting in the need to restart again. Here, the natural question would be the determination of the location probability and expected time to escape the trap.

A related mechanism—Sisyphus cooling— was proposed by Claude Cohen-Tannoudji in certain optical contexts to the effect that an atom may climb up a potential hill, till suddenly it is returned to some ground state where it can restart anew. The hallmark of such systems is the possibility to display “back-to-square-one” behavior, a feature common in real life systems. Indeed, the study of stochastic processes subject to random resets is a problem that has attracted great interest in recent years after the seminal work of Manrubia and Zanette [2] and Evans and Majumadar [3]. Presently, the dynamics of systems with resets is being subjected to intense study, see the recent review [4]. Other mechanisms for random walks that are suddenly refreshed to the starting position are considered in [5,6,7]. Brownian motion with resets is considered in [3,8] while in [9] the propagator of Brownian motion under time-dependent resetting is obtained (see also [10] for further elaboration). In [11], these ideas are applied to the case of a compound Poisson process with positive jumps and constant drift. Further elaboration appears in [12]. Reset mechanisms have also been thoroughly applied to search strategies in mathematical and physical contexts as well as to behavioral ecology, see [10,13,14,15,16,17,18]. Surprisingly, strategies that incorporate reset to pure search are advantageous in certain contexts in ecology and biophysics and molecular dynamics, [19,20,21,22]. A generalization of the Kardar–Parisi–Zhang (KPZ) equation that describes fluctuating interfaces and polymers under resetting is covered in [23]. Dynamical systems with resets have also been used as proxies of the classical integrate-and-fire models of neuron dynamics, see [24,25]. In the context of Lévy flights with resetting, see the interesting papers [26,27]. For other applications, see also the recent papers [28,29,30,31,32,33,34,35].

As commented, the main aim of this paper is to study the main features of the semi-deterministic random walk with resets $\left(x_t\right),t=0,1,\dots \infty$. The evolution rules for such a random walk are described in Section 2. We then study the propensity towards resetting of the system. According to this important property, we denote systems as reset averse, neutral or reset-inclined, and characterize them in terms of the transition probabilities and behavior of $Pr\left(x_{t+1}=n+1|x_t=n\right),n\to \infty$. In Section 3, we study the stationary distribution that the system approaches for large time. Section 4 considers first-passage problems and, in particular, two-sided exit probabilities; concretely, given levels $a,b\in \mathbb{N},$ we study the probability that x reaches $a>0$ before having reached $-b$ and distributions of the escape time. First passage times (FPT) also play a key role in statistical decision models, or to devise optimal strategies for seeking information; the rate at which a Brownian particle, under the influence of a metastable potential, escapes from a potential well is also a critical subject in the study of polymers, the so-called Kramers problem [36].

Under the simplest election $\rho =1$ and $q_n:=Pr\left(x_{t+1}=n+1|x_t=n\right)=q_1$ constant, we have that the distribution of the FPT to level $k\ge 1$ is that of the number of trials required in an unfair coin-toss to obtain k consecutive successes, a classical problem in probability. Even with $k=2$, the distribution of such a problem is not trivial.

2. The Model

The evolution rules for the random walk $\left(x_t\right),t=0,1,\dots \infty$ are as follows. Let $x_0:=0$ be the initial position. We suppose that, if for any $t\ge 0$ is $x_t=0$, then the random walk satisfies

$$Pr\left(x_{t+1}=n|x_t=0\right)={\overline{q}}_1{\delta }_{n0}+q_1\rho {\delta }_{n1}+q_1\overline{\rho }{\delta }_{n,-1},n\in \mathbb{Z}$$

(1)

We denote $q_1=Pr(x_{t+1}\ne 0|x_t=0)\in (0,1)$ the probability that, starting form zero, the system moves away from the origin at the next instant and $\rho :=Pr(x_{t+1}=1|x_t=0,x_{t+1}\ne 0)\in [0,1]$ the probability that, if the system abandons the originat time t, it goes to position $x_{t+1}=1$. To ease notation, for any value p, we set $\overline{p}:=1-p$, ${\overline{q}}_1:=1-q_1$. Besides ${\delta }_{nk}$ is the Kronecker delta.

Further, we suppose that the random walk $\left(x_t\right),t=0,1,\dots \infty$ is a Markov chain where, if $x_t\equiv n\ne 0$, the only allowed transitions are either to site $n+\mathrm{sign}\left(n\right)$ if no reset occurs, which happens with probability $q_{n+1}$; or else to $\left\{0\right\}$, when a reset occurs, with probability $1-q_{n+1}$. Here, the sequence $\left(q_n\right)$ satisfies that $0\le q_n\le 1$ for all n. It follows that the chain has the transition rules

$$Pr\left(x_{t+1}=m|x_t=n\right)=\left\{\begin{array}{c}{q}_{n+1},m=n+1\hfill \\ 1-q_{n+1},m=0\hfill \end{array}\right.,t\ge n>0$$

(2)

$$Pr\left(x_{t+1}=m|x_t=n\right)=\left\{\begin{array}{c}{q}_{\left|n\right|+1},m=n-1\hfill \\ 1-q_{\left|n\right|+1},m=0\hfill \end{array}\right.,t\ge -n>0$$

(3)

$$Pr\left(x_{t+1}=m|x_t=0\right)=\left\{\begin{array}{c}\rho q_1,m=1\hfill \\ \overline{\rho }{q}_1,m=-1\hfill \\ 1-q_1,m=0\hfill \end{array}\right.$$

(4)

and 0 otherwise. We also suppose that the infinite product with general term $q_n$ satisfies

$$\underset{n\to \infty }{lim}\prod _{j=1}^n{q}_j=0;\mathrm{alternatively}\sum _{j=1}^{\infty }(1-q_j)=\infty$$

(5)

This mild requirement does not imply that ${lim}_{n\to \infty }{q}_n=0$ (see Equation (12) below).

The model considered in [1] is recovered assuming $\rho =1$ and that the jump-probability is constant: $q_1=q_2=\dots q_n=\dots$. Figure 1 displays a typical sample path.

2.1. Reset Times

We denote as ${\mathbf{t}}_1$ the random time at which the first reset happens. Here, we consider its distribution probability $p_n:=Pr({\mathbf{t}}_1=n),n=1,\dots ,\infty$. Similarly, we denote as ${\mathbf{t}}_k$ the random time at which the $k{-}^{\mathrm{th}}$ reset happens. To this end, note that for $n=1,2,\dots$ the reset takes place at time n if in all previous times no reset has occurred—and so $|x_1|=1,\dots ,|x_{n-1}|=n-1$ and $x_n=0$. Thus, we have transitions $\left\{0\right\}\mapsto \left\{1\right\}\dots \mapsto \{n-1\}\mapsto \left\{0\right\}$ and the corresponding probability

$$p_n:=Pr({\mathbf{t}}_1=n)=q_1\dots q_{n-1}{\overline{q}}_n$$

(6)

where ${\mathbf{t}}_1$ is a proper random variable, in view of Equation (5).

The following representation clarifies the meaning and different roles of $\left(p_n\right)$ and $\left(q_n\right)$

$$p_n:=Pr({\mathbf{t}}_1=n)=Pr({\mathbf{t}}_t=n):=Pr\left(x_{t+n}=0,x_{t+j}\ne 0,0<j<n|x_t=0\right)$$

(7)

and

$${\overline{q}}_n=Pr\left(x_{t+n}=0|x_t=0,x_{t+j}\ne 0,0<j<n\right)$$

(8)

We relate both probabilities. We introduce recursively a sequence $\left({\beta }_n\right)$ via ${\beta }_0\equiv 1$ and ${\beta }_n:=q_1\dots q_n,n=1,2,\dots$. Note then that

$$p_n=q_1\dots q_{n-1}{\overline{q}}_n={\beta }_{n-1}-{\beta }_n$$

This can be inverted as

$${\beta }_n=p_{n+1}+p_{n+2}+\dots ={\overline{F}}_{{\mathbf{t}}_1}\left(n\right)=1-F_{{\mathbf{t}}_1}\left(n\right)$$

(9)

where $F_{{\mathbf{t}}_1}\equiv F$ is the cumulative distribution function (cdf) of ${\mathbf{t}}_1$ and $\overline{F}\left(n\right):=1-F\left(n\right)$. Recalling that ${\beta }_n:=q_1\dots q_n,$ we finally have that Equation (6) can be inverted as

$$q_n=\frac{\overline{F}\left(n\right)}{\overline{F}(n-1)},n=1,2\dots$$

(10)

2.2. Reset Averse and Reset-Inclined Systems

One of the most defining traits in the random walk (2)–(4) is what we call propensity towards resetting, a measure of how likely is that the resetting mechanism is triggered as the time from the last reset increases. We say that a system is inclined towards resetting if such a probability grows as the distance to the origin increases: ${\overline{q}}_n<{\overline{q}}_{n+1},$ for all n. Intuitively, for a reset-inclined system, the greater the time since the last visit (alternatively, the farthest off) the more anxious to return to the origin becomes the random walk. If this probability decreases (respectively, remains unchanged), we say that the system is reset-averse or reset-neutral. Reset-neutral chains correspond to having $q_n=q_{n-1}\equiv q_1\in (0,1)$ for all n. This is the choice considered in [1]. In this case

$$Pr({\mathbf{t}}_1=n)=q_1^{n-1}(1-q_1),F\left(n\right)=1-q_1^n$$

(11)

Actually, we are interested in this property for large n. We say that a system is ultimately averse, neutral or, respectively, inclined towards resetting if, as the time from the last reset tends to infinity, the reset probability $\left(q_n\right)$ satisfies

$$\underset{n\to \infty }{lim}{q}_n:=\underset{n\to \infty }{lim}Pr\left(x_{t+n}\ne 0|x_t=0,x_{t+j}\ne 0,0<j<n\right)=\left\{\begin{array}{c}0,\left(\mathrm{inclined}\right)\hfill \\ q_{\infty }\in (0,1)\left(\mathrm{neutral}\right)\hfill \\ 1,\left(\mathrm{averse}\right)\hfill \end{array}\right.$$

(12)

The selection $q_n=q_1/n$ corresponds to an ultimately reset-inclined system. Here, we have ${lim}_{n\to \infty }{q}_n=0$ and

$$p_n=\frac{q_1^{n-1}}{(n-1)!}-\frac{q_1^n}{n!},\overline{F}\left(n\right)=\frac{q_1^n}{n!},n=1,2\dots$$

(13)

A simple calculation yields $<{\mathbf{t}}_1>=e^{q_1}\le e$, which is bounded with respect to the parameter $q_1$.

Finally, the choice $q_n=n/(n+1)$ corresponds to a reset-averse system. Here, the chain has power law decay tails:

$$p_n=\frac{1}{n(n+1)}\mathrm{and}\overline{F}\left(n\right)=\frac{1}{n+1}$$

(14)

The selections (13) and (14) reflect that the probability to commit an error that sends the walker to square one diminishes (increases) with every step. This may be put down to a capability to learn or, in contrast, to forget or grow tired with the distance to the origin. Equation (14) corresponds to $q_n=q_{n-1}\left(1+\frac{1}{n^2-1}\right)$—and hence to learning—while, if $q_n=q_{n-1}(1-\frac{1}{n})$, then Zipf law (13)$:q_n=q_1/n$ follows.

Equation (14) may also arise due to uncertainty in the relevant parameters. Suppose we accept the basic model (11) to hold but are ignorant of the value of parameter $q_1$. Besides, we accept that all values for $q_1$ are equally likely; in this situation, the parameter $q_1$ should be assumed to have Uniform $(0,1)$ distribution. Bayes theorem implies that the distribution at posteriori of ${\mathbf{t}}_1$ must be given by Equation (14):

$$Pr({\mathbf{t}}_1=n)={\int }_0^{1}Pr({\mathbf{t}}_1=n|q_1)d{q}_1={\int }_0^{1}d{q}_1{q}_1^{n-1}(1-q_1)=\frac{1}{n(n+1)}$$

(15)

We next show that the above behavior is ubiquitous, so the reset propensity is directly related with the tail’s behavior. Indeed, since the sequence $\overline{F}\left(n\right)$ is strictly monotone and $\overline{F}\left(n\right)\downarrow 0$ as $n\to \infty$, the Stolz–Cesáro theorem gives

$$q_{\infty }:=\underset{n\to \infty }{lim}{q}_n=\underset{n\to \infty }{lim}\frac{\overline{F}\left(n\right)}{\overline{F}(n-1)}=\underset{n\to \infty }{lim}\frac{p_n}{p_{n-1}}$$

(16)

Requiring $q_{\infty }\equiv e^{-\lambda }\in (0,1)$, we obtain that asymptotically $\left(p_n\right)$ must grow as

$$p_n\approx c{e}^{-\lambda n},c,\lambda >0,n\to \infty$$

(17)

which is the paradigmatic example of ultimately neutral systems. Note that such $\left(p_n\right)$ has medium tails. By contrast, tails of the form

$$p_n\approx c{e}^{-\lambda n^{\alpha }},n\to \infty \mathrm{where}c>0,\lambda >0,\alpha >0$$

(18)

give $q_{\infty }=1$ if $0<\alpha <1$ and $q_{\infty }=0$ if $\alpha >1$. The exponential case $\alpha =1$, i.e., the geometric distribution, marks the crossover between these cases.

Note that slowly, power-law decaying, sequences such as

$$p_n\approx c/n^{\alpha },c>0,\alpha >1,n\to \infty$$

(19)

also correspond to reset-averse systems ultimately. Thus, heavy tails of the sequence $\left(p_n\right)$ correspond to reset-averse systems while the opposite holds with medium and light (super-exponential) tails such as those in Equations (11) and (18).

Table 1. The table summarizes the propensity to resetting in terms of the decay of $p_n:=Pr({\mathbf{t}}_1=n)$ and the equilibrium distribution. In all cases $\lambda >0$.

${\mathit{p}}_{\mathit{n}\to \mathit{\infty }}$	${\mathit{F}}_{{\mathbf{t}}_1}\left(\mathit{n}\right)$	${\mathit{q}}_{\mathit{n}\to \mathit{\infty }}$	Propensity	Tails	$\mathbb{E}{\mathbf{t}}_1$	${\mathit{\pi }}_{\mathit{n}\to \mathit{\infty }}$
$O\left(e^{-\lambda n^{\alpha }}\right),\alpha >1$	$O(e^{-\lambda n^{\alpha }})$	0	$\mathrm{inclined}$	Super-exp.	<∞	$O\left(e^{-\lambda n^{\alpha }}\right)$
$O({(\frac{e^{-\lambda }}{n})}^n,$	$O({(\frac{e^{-\lambda }}{n})}^n,$	0	$\mathrm{inclined}$	Super-exp.	<∞	$O({(\frac{e^{-\lambda }}{n})}^n$
$O\left(e^{-\lambda n}\right)$	$O\left(e^{-\lambda n}\right)$	$\in (0,1)$	$\mathrm{neutral}$	$\exp .$	<∞	$O\left(e^{-\lambda n}\right)$
$O\left(e^{-\lambda n^{\alpha }}\right),0<\alpha <1$	$O\left(e^{-\lambda n^{\alpha }}\right),0<\alpha <1$	1	$\mathrm{averse}$	Sub-exp.	<∞	$O\left(e^{-\lambda n^{\alpha }}\right)$
$O(1/n^{\alpha }),\alpha >2$	$O(1/n^{\alpha -1}),\alpha >2$	1	$\mathrm{averse}$	Power-law	<∞	$O\left(n^{1-\alpha }\right)$
$O(1/n^{\alpha }),1<\alpha \le 2$	$O(1/n^{\alpha -1}),1<\alpha \le 2$	1	$\mathrm{averse}$	Power-law	=∞	—-

summarizes all the possibilities.

More complicated tails can be handled noting the behavior of ultimately averse, neutral or inclined reset systems under sums and products. We use $q_{\infty }:=\vartheta$ to denote that ${lim}_{n\to \infty }{q}_n\in (0,1)$ (thus, ${lim}_{n\to \infty }{q}_n=0,\vartheta$ or 1). Hence, with obvious notation, the sums and product rules for $q_{\infty }^{\left(1\right)},q_{\infty }^{\left(2\right)}$ read

$$0+0=0;0+\vartheta =\vartheta ;0+1=1;\vartheta +\vartheta =\vartheta ;\vartheta +1=1+1=1;$$

$$0·0=0·\vartheta =0·1=0;\vartheta ·\vartheta =\vartheta ·1=\vartheta ;1·1=1$$

where the symbol $\vartheta ·\vartheta =\vartheta$ is used to mean that if

$$\underset{n\to \infty }{lim}{q}_n^{\left(1\right)}\in (0,1),\underset{n\to \infty }{lim}{q}_n^{\left(2\right)}\in (0,1),\mathrm{then}\underset{n\to \infty }{lim}{q}_n^{\left(1\right)}{q}_n^{\left(2\right)}\in (0,1)$$

As an example, for $0<c<1$, consider the hybrid system

$$p_n=\frac{n\overline{\nu }+1}{n(n+1)}{\nu }^{n-1}=O\left(\frac{e^{-\lambda n}}{n}\right)$$

where $\nu :=e^{-\lambda },\lambda >0$. Here, $p_n\equiv p_n^{\left(1\right)}{p}_n^{\left(2\right)}$ and tails display mixed exponential and power-law decay. Hence, $q_{\infty }=q_{\infty }^{\left(1\right)}·q_{\infty }^{\left(2\right)}=\vartheta ·1=\vartheta$ correspond to an ultimately neutral system. This is corroborated by an exact evaluation of $q_n$. Equation (10) yields that

$$q_n=n\nu /(n+1)\mathrm{and}\underset{n\to \infty }{lim}{q}_n=\nu \in (0,1)$$

2.3. Comments on Markovianness

The fact that the system has a capability for learning (or to forget, see Equations (11), (13) and (14)) suggests the existence of some “memory” in the dynamics, a fact that cast doubt upon the Markovian nature of the model. Actually, there is some hidden memory, although not in $\left(x_t\right)$.

This apparent paradox has interesting implications and can be developed further as we now explain. At any time t, let $N_t$ be the process that counts the number of resets ${\mathbf{t}}_n$ “observed” in the time window $(0,t]$. Thus, $N_t=n$ if ${\mathbf{t}}_n\le t<{\mathbf{t}}_{n+1}$. Given that $N_t=n,n\le t$ any additional information may result in information relevant to predict its future: if additionally, say $N_{t-1}=n-1$, we infer that a reset has occurred exactly at t, a valuable information. Hence, $\left(N_t\right)$ is generically non-Markovian. The only exception is the system (11); here, $N_t$ follows a binomial distribution with success parameter ${\overline{q}}_1$:

$$\mathbb{P}\left(N_t=n\right)=\left(\genfrac{}{}{0pt}{}{t}{n}\right)·{\overline{q}}_1^n{q}_1^{t-n},n=0,1,\dots t$$

(20)

By contrast, suppose we know $x_t=i,0<i\le t$. This information amounts to saying that the previous reset occurred at ${\mathbf{t}}_n=t-i$ and hence pins down the history of the process after ${\mathbf{t}}_n$, viz $x_{t-i+j}=i+j,\forall j=0,1,\dots i$. By contrast, the history of the process previous to ${\mathbf{t}}_n$ remains unknown. Should additional information be provided, it would not help predict the future of x, since at ${\mathbf{t}}_n$ the process started anew; hence, only the history after ${\mathbf{t}}_n$ is relevant, but this is already known. Thus, we arrive at the counter-intuitive fact that $\left(N_t\right)$ needs not be Markovian but $\left(x_t\right)$ is. Likewise, the vector chain $(N_t,x_t)$ is Markovian.

3. Equilibrium Distribution

Here, we consider the large time distribution of the random walk. Call $x_{\infty }\equiv {lim}_{t\to \infty }{x}_t$ the limit of the process and ${\pi }_n:=Pr(x_{\infty }=n),n\in Z$ its distribution. When it exists it is also a stationary state, in the sense that if it has initially this distribution then it will not abandon it. $\left({\pi }_n\right)$ satisfies the system

$$\sum _{n\in Z}{g}_{nm}{\pi }_n={\pi }_m,m\in Z$$

(21)

where $\left(g_{nm}\right)$ is the transition probability matrix defined in Equation (3):

$$g_{nm}:=Pr\left(x_{t+1}=m|x_t=n\right)$$

(22)

To handle this, we divide the matrix in upper and lower parts, connected only by the column and rows with index 0, i.e.,

$$G=\left(\begin{array}{cc}{G}_{-}& 0\\ 0& G_{+}\end{array}\right)$$

(23)

where $G_{-}$ is essentially obtained from $G_{+}$ by reflection and $G_{+,nm},n,m=0,\infty$ reads (including the $0-$ column)

$$G_{+}=\left(\begin{array}{cccccc}{\overline{q}}_1& \rho q_1& 0& 0& 0& \dots \\ {\overline{q}}_2& 0& q_2& 0& 0& \dots \\ {\overline{q}}_3& 0& 0& q_3& 0& \dots \\ \dots & & \dots \\ {\overline{q}}_n& 0& \dots & 0& 0& q_n\end{array}\right)$$

(24)

By insertion, we find

$${\pi }_1=\rho q_1{\pi }_0,{\pi }_{-1}=\overline{\rho }{q}_1{\pi }_0$$

along with the recursive system

$${\pi }_{n+1}=q_{n+1}{\pi }_n,n\ge 1\mathrm{and}{\pi }_{n-1}=q_{\left|n\right|-1}{\pi }_n,n\le -1$$

(25)

Solving recursively, we find

$${\pi }_n=\rho {\pi }_0{q}_1{q}_2\dots q_n={\pi }_0\rho \overline{F}\left(n\right)\mathrm{and}{\pi }_{-n}={\pi }_0\overline{\rho }\overline{F}\left(n\right),n\ge 1$$

(26)

Normalization gives $1/{\pi }_0={\sum }_{n=0}^{\infty }n{p}_n\equiv <{\mathbf{t}}_1>\equiv \mu$ which requires $<{\mathbf{t}}_1>\equiv \mu <\infty$, i.e., $\left(p_n\right)$ must decay at least as $p_n\approx 1/n^r,r>2$. In this case, defining ${\rho }_n\equiv \rho {\mathbf{1}}_{n>0}+\overline{\rho }{\mathbf{1}}_{n<0}+{\delta }_{n0}$, the stationary distribution is

$${\pi }_n=({\rho }_n/\mu )\overline{F}\left(\right|n\left|\right),n=-\infty ,\dots ,\infty$$

(27)

The probability that the random walk has drifted to site n for large times decreases as $\overline{F}\left(n\right)$ does, see Table 1. Hence, for reset-averse systems, $\left({\pi }_n\right)$ displays heavy tails (which may even decay in a weak, algebraic way), and so there exists a non-negligible probability to find the system away from the origin at large times. This agrees with the “unwillingness” of the system to return to the origin. The opposite holds for reset-inclined systems where this probability will decrease quicker than exponentially. Concretely, for the cases (11) and (13), the stationary distribution is

$${\pi }_n={\rho }_n(1-q_1)q_1^{\left|n\right|}\mathrm{and}{\pi }_n={\rho }_n{e}^{q_1}\frac{q_1^{\left|n\right|}}{\left|n\right|!},n=-\infty ,\dots 0,\dots ,\infty$$

(28)

Finally, for system (14), there is neither an equilibrium nor a stationary distribution, indicating that the chain spreads out far from the origin and does not settle to an equilibrium.

Writing Equation (21) as ${\sum }_{n\ne m}{\pi }_m{g}_{nm}={\sum }_{n\ne m}{\pi }_n{g}_{mn}$, it states that the total probability flux from all states n into m, $n\ne m$, must equal the flux from state m into the rest of states. Hence, $\left({\pi }_n\right)$ satisfies a global balance equations, viz Equation (21). This does not imply that $\left({\pi }_n\right)$ is an equilibrium state, only a limit state; for an equilibrium distribution it must satisfy the far stronger “detailed balance” condition: ${\pi }_m{g}_{nm}\ne {\pi }_n{g}_{mn}$—which does not hold. This was to be expected since detailed balance guarantees time-reversibility of the dynamics, a trait that the system at hand clearly does not exhibit, as a simple inspection of the trajectories shows.

In the absence of any information, the maximum entropy principle yields that the distribution with maximum entropy should be chosen on the basis that it is the least-informative one. Within the class with fixed finite mean, it is well known that this corresponds to a geometric distribution. It follows from Equation (28) that this implies that the model satisfies Equation (11). We conclude that the reset neutral chain (11) satisfying $q_n=q_1$, for all n, and arbitrary $\rho$ has the remarkable properties of being the only selection that corresponds to a truly Markovian situation (i.e., for both $\left(x_t\right)$ and the arrival process $\left(N_t\right)$) and the one that gives the maximum entropy for fixed mean.

4. Escape Probabilities

In a classical study, W. Feller [37] showed that most recurrent properties of general diffusion processes can be codified in terms of two of the functions that define escape probabilities from an interval $(c,d),c<d$. Given that the process has started from a general $x_0,c<x_0<d$, Feller considers the “scale and speed functions”, defined as

$$s\left(x_0\right)=Pr({\tau }_d>{\tau }_c)\mathrm{and}m\left(x_0\right)\equiv <min\{{\tau }_c,{\tau }_d\}>$$

(29)

and shows that they solve certain differential equations (see [37] for an overview). Here, for any $a\in R$, we introduce the “hitting time” ${\tau }^a=inf\{t>0:x_t=a\}$ which represents the lapse of time necessary to travel from the starting value to a.

We perform a similar study here and determine, for given levels $a,b\in \mathbb{N},-b<0<a$, the probability that the random walk $\left(x_t\right)$ reaches a before having reached $-b$. Note that by translational invariance the case when $\left(x\right)$ starts from general $x_0$ is immediately reduced to that with $x_0=0$.

We start noting that when resets are switched off the only source of randomness lies in the first displacement of the random walk away from $x=0$; hence, $x_n=n$ for all n if $x_1=1$. In this case, ${\tau }_{a,b}$—the minimum time to hit either $a>0$ or $-b<0$—is a binary random variable that takes values a and b with probabilities $\rho$ and $\overline{\rho }$. Besides, ${Pr}_0({\tau }_a<{\tau }_b)=\rho$.

Obviously, ${\tau }_{a,b}$ will increase when a reset mechanism is introduced; it is, however, tempting to think that resets do not affect the escape probabilities, namely ${Pr}_0({\tau }_a<{\tau }_b)=\rho$ still holds. However this is not correct! To dispel such a misinterpretation, note that resets introduce a bias which favors the closest barrier against the farthest one. This is similar to the classical waiting time paradox where cycles with very large inter-reset times have a greater probability than smaller ones. Intuitively, if restarts occur very often, the possibility to reach the farthest barrier diminishes. We now determine this probability.

A very simple argument goes as follows. Consider the probability ${\ell }_a$ that the random walk $\left(x\right)$ reaches $a>0$ before having reached $-b$ when we know that $\left(x_t\right)$ hits a or $-b$ in the given cycle. The event that escape occurs at a given cycle, say, the first, is

$$E:=\{x_1>0,{\mathbf{t}}_1>a\}\cup \{x_1<0,{\mathbf{t}}_1>b\}\equiv E_1\cup E_2$$

(30)

with probability

$$\kappa \equiv Pr\left(E\right)=\rho \overline{F}\left(a\right)+\overline{\rho }\overline{F}\left(b\right)$$

(31)

Hence, the probability that escape occurs via the upper barrier can be evaluated as the probability of $E_1$ conditional on the event E having happened:

$${\ell }_a=Pr\left(\mathrm{escape}\mathrm{via}a\right|E)=Pr\left(E_1\right|E_1\cup E_2)=\frac{\rho \overline{F}\left(a\right)}{\kappa }$$

(32)

The reasoning when escape occurs at a general given cycle is a bit more involved but does not change the result.

Denote ${\ell }_a^0\equiv \rho$ the corresponding probability when no resets are introduced. Then,

$${\ell }_a\ge {\ell }_a^0\iff \overline{F}\left(b\right)\le \overline{F}\left(a\right)\iff b\ge a$$

(33)

which means that resets increase the probability to hit first the closest barrier, as expected. Further, when $a=b$ Equation (32) yields ${\ell }_a={\ell }_a^0$.

We thus have for the neutral chain (11), the reset-averse chain (13) and the reset-inclined chain (14), respectively

$${\ell }_a=\left\{\begin{array}{c}\rho {\left(\rho +\overline{\rho }{q}_1^{b-a}\right)}^{-1}\hfill \\ \rho b!{\left(b!\rho +\overline{\rho }a!q_1^{b-a}\right)}^{-1}\hfill \\ {\left(1+\frac{\overline{\rho }(a+1)}{\rho (b+1)}\right)}^{-1}\hfill \end{array}\right.=\left\{\begin{array}{c}\rho {\left(\rho +\overline{\rho }{q}_1^d\right)}^{-1}\hfill \\ \rho {\left(\rho +\overline{\rho }{q}_1^{d}a!/(a+d)!\right)}^{-1}\hfill \\ {\left(1+\frac{\overline{\rho }(a+1)}{\rho (a+d+1)}\right)}^{-1}\hfill \end{array}\right.$$

(34)

In the second equality, we introduce $d:=b-a$, which measures the departure from symmetry of the problem and suppose $b\ge a$ for ease of notation.

5. Escape Times

5.1. Symmetry Properties of First Passage Times

Denote for a moment as ${\tau }_{a,b}^{\rho }$ the FPT to either a or $-b$ when $Pr(x_{t+1}=1|x_t=0,x_{t+1}\ne 0)=\rho$. This quantity ${\tau }_{a,b}$ has a nice interpretation. Suppose the model (11) holds. Say a success has been scored every time a reset does not happen. Then, ${\tau }_{n,n}^1={\tau }_n^1$ is the time that takes to obtain $n\ge 1$ successes in a row provided the probability of individual success is $q_1$, a classical problem in probability. Obviously, if $n=1$ then ${\tau }_1^1$—the first time to reach level 1—must have a geometric distribution with parameter $q_1$. However, even with $n=2$, this problem has no easy solution, not even for the mean times.

We note the interesting relation between the asymmetric and symmetric cases.

• If $l_a$ is defined in Equation (32) and $l_a+l_b=1$ and $\mathbb{E}X\equiv <X>$ indicates the expected value of the random variable X, we have

  $$\mathbb{E}{\tau }_{a,b}^{\rho }=l_a\mathbb{E}{\tau }_{a,a}^{\rho }+l_b\mathbb{E}{\tau }_{b,b}^{\rho }$$

  (35)
• ${\tau }_{a,a}^{\rho }$ is independent of $\rho$. Besides, the distributions in the symmetric case and one-sided case are equal, namely, for any b

  $${\tau }_{a,a}^{\rho }={\tau }_{a,b}^1={\tau }_{a,\infty }^1\equiv {\tau }_a^1;{\tau }_{a,\infty }^{\rho }={\tau }_a^{\rho }$$

  (36)
  Indeed, when the interval is symmetric the escape time will not be influenced by whether resets favor upward or downward flights; hence, Equation (36) must hold. For the sake of comparison, we see that Equation (39) overestimates the time it takes to reach the boundaries.

A first approximation is given by $<{\tau }_{a,b}>\approx <\mathbf{N}>\times <{\mathbf{t}}_1>$ where $\mathbf{N}$ is the number of resets until escape. To warm up, we consider first the distribution of $\mathbf{N}$.

By independence of cycles, $\mathbf{N}$ has a geometric distribution with exit parameter $\kappa :=Pr\left(E\right)$ where E, $\kappa$ are defined in Equations (30), (31). Thus, we have $\mathbf{N}\sim$ Geom $\left(\kappa \right)$:

$$Pr(\mathbf{N}=n)=\kappa {(1-\kappa )}^{n-1},n=1,2\dots \mathrm{and}$$

(37)

$$<\mathbf{N}>=(1/\kappa ),<{\tau }_{a,b}>\approx <\mathbf{N}>\times <{\mathbf{t}}_1>=\frac{{\sum }_{n=1}^{\infty }n{p}_n}{\rho \overline{F}\left(a\right)+\overline{\rho }\overline{F}\left(b\right)}$$

(38)

In particular, for the symmetric case $a=b$,

$$<{\tau }_{a,b}>\approx \left(\sum _{n=1}^{\infty }n{p}_n\right)/\left(\sum _{n=a+1}^{\infty }{p}_n\right)\ge a+\left(\sum _{n=1}^{a}n{p}_n\right)/\left(\sum _{n=a+1}^{\infty }{p}_n\right)$$

(39)

Clearly, this approximation is only reasonable when the system needs a large number of resets prior to exit the interval, i.e., $\kappa \approx 0$.

5.2. Mean Exit Time

To study the exact time to hit a or b, we note that depending of what happens at the the first reset ${\mathbf{t}}_1$ there are five excluding and exhausting possibilities. These scenarios are:

• (S1) $x_1>0$ and ${\mathbf{t}}_1>a$;
• (S2) $x_1<0$ and ${\mathbf{t}}_1>b$;
• (S3) $x_1>0$ and ${\mathbf{t}}_1\le a$;
• (S4) corresponds to having $x_1<0$ and ${\mathbf{t}}_1\le b$;
• (S5) corresponds to $x_1=0$.

Under scenario (S1), $\left(x_t\right)$ hits a before it hits b with ${\tau }_{a,b}=a$. Under scenario (S2), $\left(x_t\right)$ hits b before a and ${\tau }_{a,b}=b$. Scenarios (S3) to (S5) refresh $\left(x_t\right)$ to the origin so the “race” starts again from scratch; hence ${\tau }_{a,b}={\mathbf{t}}_1+{\tau }_{a,b}^{\prime }$, where ${\tau }_{a,b}^{\prime }$ is the time that remains until exit once the new cycle starts. This implies that

$${\tau }_{a,b}=\left\{\begin{array}{c}a\mathrm{if}{x}_1>0,{\mathbf{t}}_1>a\hfill \\ b\mathrm{if}{x}_1<0,{\mathbf{t}}_1>b\hfill \\ {\mathbf{t}}_1+{\tau }_{a,b}^{\prime }\mathrm{if}2\le {\mathbf{t}}_1\le a,x_1>0\mathrm{or}2\le {\mathbf{t}}_1\le b,x_1<0\mathrm{or}{\mathbf{t}}_1=1\hfill \end{array}\right.$$

(40)

$$\mathrm{and}\mathbb{E}{\tau }_{a,b}=a\rho \overline{F}\left(a\right)+b\overline{\rho }\overline{F}\left(b\right)+$$

$$\rho \mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le a}\right)+\overline{\rho }\mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le b}\right)+(\overline{\rho }F\left(b\right)+\rho F\left(a\right))\mathbb{E}{\tau }_{a,b}^{\prime }$$

Here, ${\mathbf{1}}_A=1$ if the event A holds and is 0 otherwise. Thus, we finally obtain

$$\mathbb{E}{\tau }_{a,b}=\frac{\rho \left(a\overline{F}\left(a\right)+\mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le a}\right)\right)+\overline{\rho }\left(b\overline{F}\left(b\right)+\mathbb{E}(\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le b}\right)\right)}{\overline{\rho }\overline{F}\left(b\right)+\rho \overline{F}\left(a\right)}$$

(41)

If $b\to \infty$, then $b\overline{F}\left(b\right)\to 0$ and we recover the mean hitting time to level a as

$$\mathbb{E}{\tau }_a=a+\frac{1}{\rho \overline{F}\left(a\right)}\left(\mathbb{E}\left({\mathbf{t}}_1\right)-\rho \mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1>a}\right)\right)$$

(42)

Particularly interesting is the symmetric case $a=b$. Here,

$$\mathbb{E}{\tau }_{a,a}^{\rho }=\mathbb{E}{\tau }_a^1=a+\frac{\mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le a}\right)}{\overline{F}\left(a\right)}=a+\left(\sum _{n=1}^{a}n{p}_n\right)/\left(\sum _{n=a+1}^{\infty }{p}_n\right)$$

(43)

Note how this implies equation (35).

5.3. Distribution of the Exit Time

Finally, we consider the distribution of ${\tau }_{a,b}$. We evaluate its generating function

$$G_{}\left(z\right)=\sum _{n=1}^{\infty }{z}^{n}Pr({\tau }_{a,b}=n)$$

(44)

by using Equation (40). Here, $z\in C,\left|z\right|\le 1$. Recall that

$${\tau }_{a,b}=a{\mathbf{1}}_{x_1>0,{\mathbf{t}}_1>a}+b{\mathbf{1}}_{x_1<0,{\mathbf{t}}_1>b}+\left({\mathbf{t}}_1+{\tau }_{a,b}^{\prime }\right)$$

(45)

$${\mathbf{1}}_{2\le {\mathbf{t}}_1\le a,x_1>0}+{\mathbf{1}}_{2\le {\mathbf{t}}_1\le b,x_1<0}+{\mathbf{1}}_{{\mathbf{t}}_1=1}$$

Note also that

$$\mathbb{E}\left(z^{{\mathbf{t}}_1+{\tau }_{a,b}^{\prime }}{\mathbf{1}}_{2\le {\mathbf{t}}_1\le a,x_1>0}\right)=\mathbb{E}\left(z^{{\mathbf{t}}_1}{\mathbf{1}}_{2\le {\mathbf{t}}_1\le a,x_1>0}\right)\mathbb{E}\left(z^{{\tau }_{a,b}}\right)=\rho {\widehat{p}}^a\left(z\right)G_{{\tau }_{a,b}}\left(z\right)$$

where we define the truncated generating function ${\widehat{p}}^a\left(z\right)={\sum }_{k=1}^a{z}^k{p}_k$.

It follows from Equation (40) that $G_{{\tau }_{a,b}}\left(z\right)$ is the sum of the following terms

$$G_{{\tau }_{a,b}}\left(z\right)=E_1+E_2{G}_{{\tau }_{a,b}^{\prime }}\left(z\right)$$

(46)

where

$$E_1:=z^{a}Pr(x_1>0,{\mathbf{t}}_1>a)+z^{b}Pr(x_1<0,{\mathbf{t}}_1>b)=z^a\rho \overline{F}\left(a\right)+z^b\overline{\rho }\overline{F}\left(b\right),$$

$$E_2:=\mathbb{E}(z^{{\mathbf{t}}_1}{\mathbf{1}}_{{\mathbf{t}}_1\le a,x_1>0})+\mathbb{E}\left(z^{{\mathbf{t}}_1}{\mathbf{1}}_{{\mathbf{t}}_1\le b,x_1<0}\right)+\mathbb{E}\left({\mathbf{1}}_{{\mathbf{t}}_1=1}\right)$$

Thus finally, in Laplace space, the generating functions read

$$G_{{\tau }_{a,b}}\left(z\right)=\frac{z^a\rho \overline{F}\left(a\right)+z^b\overline{\rho }\overline{F}\left(b\right)}{1-\rho {\widehat{p}}^a\left(z\right)-\overline{\rho }{\widehat{p}}^b\left(z\right)}$$

(47)

Hence, the mass function of ${\tau }_{a,b}$ is

$$\mathbb{P}({\tau }_{a,b}=n)=\frac{\overline{F}\left(a\right)}{2\pi i}\oint dz\frac{G_{{\tau }_{a,b}}\left(z\right)}{z^{n+1}},n\ge 1$$

(48)

If either $b=a$ (symmetric case) or $\rho =1,b=1$ (one-sided case), it simplifies to

$$G_{{\tau }_{a,a}}\left(z\right)=\frac{z^a\overline{F}\left(a\right)}{1-{\widehat{p}}^a\left(z\right)}$$

(49)

$$\mathbb{P}({\tau }_{a,a}=n)=\frac{\overline{F}\left(a\right)}{2\pi i}\oint dz\frac{dz}{z^{n+1-a}(1-{\widehat{p}}^a\left(z\right))},n\ge a$$

(50)

The FPT to a is recovered letting $b\to \infty$; then, ${\widehat{p}}^b\left(z\right)\to \widehat{p}\left(z\right):={\sum }_{n=1}^{\infty }{z}^n{p}_n$ and

$$G_{{\tau }_a}\left(z\right)=\frac{z^a\rho \overline{F}\left(a\right)}{1-\rho {\widehat{p}}^a\left(z\right)-\overline{\rho }\widehat{p}\left(z\right)}$$

(51)

5.4. FPT under the Model (11)

If Equation (11) holds, the distribution of ${\tau }_{a,a}$ simplifies. The generating function and distribution of the exit time read ${\widehat{p}}^a\left(z\right)={\overline{q}}_{1}z(1-{\left(z{q}_1\right)}^a)/(1-z{q}_1)$ and

$$G_{{\tau }_{a,a}}\left(z\right)=\frac{{\left(q_{1}z\right)}^a(1-q_{1}z)}{1-z+q_1^a{\overline{q}}_1{z}^{a+1}}$$

(52)

Hence, when $a=1$ we recover $G_{{\tau }_{1,1}}\left(z\right)=q_{1}z/(1-{\overline{q}}_{1}z)$ corresponding to a geometric distribution. Note

$$Pr({\mathbf{t}}_1=n)=q_1^{n-1}(1-q_1),Pr({\tau }_{1,1}=n)={\overline{q}}_1^{n-1}{q}_1$$

(53)

For $a=2$, we have

$$G_{{\tau }_{2,2}}\left(z\right)=\frac{{\left(q_{1}z\right)}^2}{1-{\overline{q}}_{1}z-q_1{\overline{q}}_1{z}^2}$$

(54)

If $s_{\pm }:={\overline{q}}_1\pm \sqrt{{\overline{q}}_1^2+4q_1{\overline{q}}_1}$, this can be inverted as

$$\mathbb{P}({\tau }_{2,2}=n)=q_1^2\sum _{j=0}^{n-2}\left(\genfrac{}{}{0pt}{}{n-2-j}{j}\right)q_1^j{\overline{q}}_1^{n-2-j}=\frac{q_1^2\left(s_{+}^{n-1}-s_{-}^{n-1}\right)}{2^{n-2}(s_{+}-s_{-})}$$

(55)

Hence, summing an arithmetic-geometric series, we find if $\ell =1/q_1$

$$\mathbb{E}{\tau }_{a,a}=a+\frac{1}{q_1^a{\overline{q}}_1^2}\left(q_1(1-q_1^{a+1}-{\overline{q}}_1(a+1)q_1^{a+1}\right)=\frac{{\ell }^a-1}{\ell -1}$$

(56)

Let ${\xi }_a$ denote the number of trials until the first consecutive a successes occur in a sequence of Bernoulli trials (or unfair coin-tosses) with probability of individual success $q_1$. This problem does not have a simple answer except when $a=1$. Clearly, ${\xi }_1\sim \mathrm{Geom}\left(q_1\right)$.

To handle the case $a\ge 2$, we note that the distribution of ${\xi }_a$ is that of the FPT to level a provided $\rho =1$; it is recovered letting $b\to \infty$ (see Equation (36) ) and using Equation (52) as

$${\xi }_a={\tau }_{a,\infty }^1={\tau }_a^1={\tau }_{a,a}^{\rho }\mathrm{and}{G}_{\xi }\left(z\right)=\frac{{\left(q_{1}z\right)}^a(1-q_{1}z)}{1-z+q_1^a{\overline{q}}_1{z}^{a+1}}$$

(57)

The mean number of trials until the first consecutive a successes is

$$a+\frac{\mathbb{E}\left({\mathbf{t}}_1{\mathbf{1}}_{{\mathbf{t}}_1\le a}\right)}{\overline{F}\left(a\right)}$$

6. Discussion

We considered a discrete-time random walk $\left(x_t\right)$ which at random times is reset to the starting position and performs a deterministic motion between them. We discussed how to interpret the property that the system is averse, neutral or inclined towards resetting. We showed that such a behavior is critical for the existence and properties of the stationary distribution. We obtained double barrier probabilities, first passage times and the distribution of the escape time from intervals. We pointed out that the distribution of the FPT to level $k\ge 1$ solves a classical problem in probability, namely that of the number of trials required in an unfair coin-toss or Bernoulli trial to obtain k successes in a row.