2

Consider

$$P = \left( \begin{array}{ccccc} p_0 & 1-p_0 & 0 & 0 & \cdots \\ p_1 & 0 & 1-p_1 & 0 & \cdots \\ p_2 & 0 & 0 & 1-p_2 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \vdots \end{array} \right)$$

It is obvious all states are recurrent from the graph but how to prove it in the formula? something like show f(00)=1

Erik M
  • 2,391
AlieZ777777
  • 23
  • What do you mean by $f(oo)$? – amWhy Mar 02 '20 at 00:29
  • If the $p_i$ go to zero too fast, then the chain is not recurrent, even if they are never actually equal to zero. – Ian Mar 02 '20 at 00:37
  • Specifically, assuming for definiteness that $0<p_i<1$ for all $i$, the question is about whether $\prod_{i=1}^\infty (1-p_i)=0$ or not, which (by taking the logarithm and using some simple estimates for $\ln(x)$ near $x=1$) is equivalent to asking whether $\sum_{i=1}^\infty p_i$ is infinite or not. – Ian Mar 02 '20 at 00:42
  • i mean from definition if i is recurrent ,then pii(1)+pii(2)+....=infinite（where pii(j) means the probability of using j steps start from j and return to j ）and fii(1)+fii(2)....=1(where fii(j) means the probability of using j steps start from i and the first time return to i) could use these formulas to prove? – AlieZ777777 Mar 02 '20 at 03:40
  • supposing each $p_i \in (0,1)$ (i.e. not 1 or 0... and in particular not 1) then the answer is given here (with $\delta_{k+1} = p_k$ https://math.stackexchange.com/questions/3532080/prod-k-1-infty-1-1-2k-converge-to-zero/3532167#3532167 – user8675309 Mar 10 '20 at 18:59

2 Answers2

1

First off, this chain is irreducible, so either every state is recurrent or every state is transient.

Second, the probability that the chain started at $0$ reaches state $n$ before its next return to $0$ is $\prod_{i=0}^{n-1} (1-p_i)$. Thus the probability that it diverges to infinity before returning to $0$ is $\prod_{i=0}^\infty (1-p_i)$. Thus everything hinges on whether this product is $0$ or not. As usual with infinite products, we tend to study them by looking at their logarithm, so you are looking at whether $\sum_{i=1}^\infty \ln(1-p_i)$ is infinite or not. By a Taylor expansion argument, this is the same as asking whether $\sum_{i=1}^\infty p_i=\infty$ or not. If this sum is infinite, then the chain is not recurrent (because it will eventually fail to return to $0$ with probability 1, since it will get "infinitely many attempts" at a Bernoulli trial). If the sum is finite, then the chain is recurrent.

In terms of the $p_{i,i}$ and $f_{i,i}$ notation that you are acquainted with, you can calculate $f_{0,0}^{(n)}=p_{n-1} \prod_{j=0}^{n-2} (1-p_j)$, since you return to $0$ from $0$ in exactly $n$ steps if and only if you move to the right exactly $n-1$ times and then jump back to $0$.

To continue in this method, if you introduce $q_j=1-p_j$ then you are now looking at $\sum_{n=1}^N (1-q_{n-1}) \prod_{j=0}^{n-2} q_j=(1-q_0)+(1-q_1)q_0+(1-q_2)q_0 q_1+\dots$ which telescopes, becoming $1-\prod_{j=0}^{N-1} q_j$, so again the question of recurrence comes down to whether $\prod_{j=0}^\infty q_j=0$ or not. It is just that in the previous way of looking at the problem it was easier to compute $1-\sum_{k=1}^n f_{0,0}^{(k)}$ (i.e. the probability that the first return to $0$ doesn't occur in the first $n$ time steps) "by inspection", rather than actually evaluating the terms and summing them. (This is basically the same reason why the CDF of the geometric distribution is simpler than the PMF.)

Ian
  • 101,645
  • Why should we calculate weather π(1-pi)=0 or not? why the probability that the chain started at 0 reaches state n before its next return to 0 should be π(1-pi)? I think it is pi*π(1-pi) – AlieZ777777 Mar 02 '20 at 22:35
  • 1
    @AlieZ777777 The probability that the chain reaches state $n$ before returning to $0$ is $\prod_{i=0}^{n-1} (1-p_i)$, since I'm only specifying what happens in the first $n$ jumps, not what happens next. The probability that it doesn't go any further after that is $p_n \prod_{i=0}^{n-1} (1-p_i)$. – Ian Mar 02 '20 at 22:57
  • I could understand now. But why should we focus on The probability that the chain reaches state n before returning to 0? – AlieZ777777 Mar 02 '20 at 23:03
  • @AlieZ777777 It just makes the calculation easier, because if it reaches every state before returning to $0$ then it never returns to zero. Using $f_{0,0}$ you have to figure out whether $\sum_{n=1}^\infty p_{n-1} \prod_{j=0}^{n-2} (1-p_j)<1$ or not. – Ian Mar 02 '20 at 23:04
  • so when we calculate π(1-pi)=0 as n tends to infinite it means it would not reach every state so it would return to 0 and then every state are recurrent. Is it right? – AlieZ777777 Mar 02 '20 at 23:09
  • @AlieZ777777 In the case when $\prod_{i=0}^\infty (1-p_i)=0$, it is guaranteed to return to 0 so by the irreducibility every state is indeed recurrent. – Ian Mar 02 '20 at 23:10
  • sorry, could you explain a little bit about Taylor expansion？ beacuse ln(-x+1) =-x - x^2/2 -x^3/3 ...+(-1)x^n/n+.... so why we consider the sum of ln（1-pi）is equvialent to the sum of pi – AlieZ777777 Mar 02 '20 at 23:40
  • @AlieZ777777 The point is that $\ln(1-x)=\frac{-x}{1-\xi}$ where $\xi$ is between $0$ and $x$, so the sum of that will diverge exactly when the sum of $x$ itself diverges, since $1-\xi$ is bounded away from $0$ (since $x$ is bounded away from $1$). If $x$ is not bounded away from $1$ then it is very easy to see that the sum of the logs will diverge. – Ian Mar 02 '20 at 23:43
  • ln(-x+1) =-x - x^2/2 -x^3/3 ...+(-1)x^n/n+.... how to get ln(1−x)=−x/1−ξ where ξ is between 0 and x – AlieZ777777 Mar 02 '20 at 23:55
  • @AlieZ777777 The mean value theorem. – Ian Mar 03 '20 at 03:32
  • We can use that $1-x \leq exp(-x) \Rightarrow \prod_{k \geq 0}(1-p_k) \leq \prod_{k \geq 0}exp(-p_k) = exp(-\sum_{k\geq 0}(p_k)) \Rightarrow \sum_{k\geq 0}(p_k) = \infty \Rightarrow 0 \leq \prod_{k \geq 0}(1-p_k) \leq 0 $ – Ghylherme Patriota Nov 24 '22 at 00:02
0

If $\pi$ is a stationary distribution for $P$, that is, $\pi P=\pi$, then we must have \begin{align} \pi_0 &= \sum_{i=0}^\infty p_i\pi_i\tag1\\ \pi_i &= (1-\pi_{i-1})\pi_{i-1},\ i\geqslant 1\tag2 \end{align} Clearly $P$ cannot have a stationary distribution if $\sum_{i=0}^\infty p_i=\infty$. If the sum is finite, use the recurrence from $(2)$ to find an expression for $\pi_i$ in terms of $\pi_0$, then use $(1)$ to find $\pi_0$ explicitly and hence the rest of the $\pi_i$.

Math1000
  • 36,983
  • thanks but it didn't mention that this is a stationary distribution. should I prove it or just assume it? if we just get this assumption what about if it is not a stationary distribution? – AlieZ777777 Mar 02 '20 at 03:41
  • This Markov chain is positive recurrent if and only if it has a stationary distribution, which is the case if and only if $\sum_{i=0}^\infty p_i <\infty$. – Math1000 Mar 02 '20 at 06:14