Understanding invariant and stationary distributions for Markov chains

Question

I have 3 little questions regardings invariant and stationary probability distributions.

Let $E=\{a,b\}$ be a state space and $\textbf{Q}$ a transition matrix $$\textbf{Q}:=\begin{pmatrix}1-p&p \\ q &1-q\end{pmatrix}$$ with $0<p,q \leq 1$ and $p+q<2$.

Then a measure $\mu$ is invariant if $\mu \textbf{Q}=\mu$. Now this means the following must hold $$\begin{cases}\mu(a)+\mu(b)=1\\ p\cdot\mu(a)-q\cdot\mu(b)=0\end{cases}$$

1. Now I don't understand why $p\cdot\mu(a)-q\cdot\mu(b)=0$ must hold?

(If I have $3$ states would it be $p\cdot\mu(a)-q\cdot\mu(b) -0\cdot\mu(c)=0$?)

The invariant distribution is $$\mu(a)=\frac{q}{p+q}\quad \mu(b)=\frac{p}{p+q}$$ We have that $$\lim\limits_{n\to \infty}\textbf{Q}^n=\frac{1}{p+q}\begin{pmatrix}q&p \\ q &p\end{pmatrix}$$ So $\lim\limits_{n\to \infty}\textbf{Q}^n=(\mu,\mu)^T$

2. So the transition matrix is converging to the invariant probability distribution. Is the invariant probability always the stationary distribution (and vice versa)? It confuses me that we have invariant/stationary distributions defined as vectors, but now the stationary distribution is the matrix?

3. Does the limit matrix $\lim\limits_{n\to \infty}\textbf{Q}^n$ always have the same row? Meaning the first row of the limit matrix is also the second etc. (Here it is $\mu$ and again $\mu$)

Answer 1

Let $P(i\mid j)$ denote the conditional probability that the next state of the system will be $i$ given that it is in state $j$. With this the state transition matrix is $$\begin{bmatrix} P(a\mid a)& P(b\mid b)\\ P(a\mid b)& P(b\mid b) \end{bmatrix}=\begin{bmatrix} 1-p& p\\ q& 1-q \end{bmatrix}.$$

Given the state transition matrix and assuming that the probabilities that the system is in state $a$ or $b$ are $\mu(a)$ and $\mu(b)$, respectively, one can calculate the probability that the system willy in state $a$ is

$$\mu'(a)=P(a\mid a)\mu(a)+p(a\mid b)\mu(b)$$

The same way

$$\mu'(b)=P(b\mid a)\mu(a)+P(b\mid b)\mu(b).$$

If $\mu'(a)=\mu(a)$ and $\mu'(b)=\mu(b)$ then the these probabilities are the stationary state probabilities.

So, if one wants to calculate the stationary probabilities then the one has the following three equations for $\mu(a)$ and $$\mu(b)$::

$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$ $$\mu(b)=p\mu(a)+(1-q)\mu(b).$$ $$\mu(a)+\mu(b)=1.$$

Too many equations... But if the stationary probabilities exist then it is enough to take only two of the equations above:

$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$ $$\mu(a)+\mu(b)=1.\tag 1$$

From the first equation you get

$$0=q\mu(b)-p\mu(a).$$

This answers your first question.

---

The $n^{th}$ power of the state transition matrix is called the $n$ step transition matrix because it shows the state transition probabilities (in one) if we let the chain step ahead $n$ times.

If $p,q>0$ then the limit of the $n^{th}$ power of the state transition matrix tends to a matrix whose both rows equal the solutions (vector) of $(1)$. If you apply this limiting matrix to the vector of the stationary distributions then you get them back:

$$\frac1{p+q}[q \ p] \frac1{p+q}\begin{bmatrix} q&p\\ q& p \end{bmatrix}=\frac1{(p+q)^2} [q^2+pq\ \ qp+p^2] =\frac1{p+q}[ q \ p].$$

I expect that this latter relationship between the limit of the $n$-step transition matrix and the stationary solution enlightens you.