My Math Forum (http://mymathforum.com/math-forums.php)
-   -   3X3 matrix, Markov chain (http://mymathforum.com/advanced-statistics/326938-3x3-matrix-markov-chain.html)

 ilovepanda February 2nd, 2016 10:27 PM

3X3 matrix, Markov chain

I'm preparing an exam and I found this problem in the previous year's documents. I've been trying to solve it, thinking about it ever since.

Consider the following 3x3 matrix $\displaystyle P$:

$\displaystyle \begin{vmatrix} 0.5 & 0.3 & 0.2 \\ 0.2 & 0.2 & 0.6 \\ 0.4 & 0.4 & 0.2 \end{vmatrix}$

- Calculate $\displaystyle P^{100}$ analytically but quickly (show the steps - no Matlab)
- Assume this matrix corresponds to a (discrete time) markov chain where entry $\displaystyle P_{ij}$ shows the probability of going from state i to state j. What is the stationary distribution for this chain (if any)?
- How fast does the chain converge to the stationary distribution? (assume the initial distribution is (1,0,0) and calculate some difference metric between the initial distribution and the derived stationary distribution).

Firstly, let me tell you what I got so far.

- $\displaystyle P^{100}=P^{4}+P^{32}+P^{64} =$

$\displaystyle \begin{vmatrix} 0.376 & 0.301 & 0.32 \\ 0.376 & 0.301 & 0.32 \\ 0.376 & 0.301 & 0.32 \end{vmatrix}$

- Def: "The stationary distribution of a markov chain with transition matrix P is some vector, x, such that xP=x." Then we multiply x = [x1 x2 x3] with the matrix P from the beginning, and get the value of the vector (the stationary distribution): x=(0.377, 0.301, 0.32).

- The last question is the one causing trouble.

1) are my results correct so far?
2) how do I calculate convergence speed from (1,0,0) to (0.377, 0.301, 0.32)?
3) what are the difference metrics? I've only heard of "Total Variation Distance", but unfortunately, "I've only heard of it" is really all I know.

I would be tremendously grateful to any guidance/help you could provide me. Thanks!

 All times are GMT -8. The time now is 10:29 AM.