My Math Forum (http://mymathforum.com/math-forums.php)
-   -   help in markov chain (http://mymathforum.com/advanced-statistics/26380-help-markov-chain.html)

 legendoulis April 3rd, 2012 09:35 AM

help in markov chain

Hi i have an exercise and have a trouble with it if anyone can help me i would appreciate it plz help if you can.

This is the exercise:

Suppose the weather in a city in the (n) day is described from a Markov chain and it depend on the (n-1) day as follows:

1. if in (n-1) day rains then in (n) day rains with probability 60% or will be sunshine with probability 40%

2. if in (n-1) day has sunshine then in (n) day will have sunshine with probability 60% or will have clouds with probability 40%.

3. if in (n-1) day have clouds then in (n) day will rain with probability 20% , will have sunshine with probability 20% , will have clouds with probability 55% or will snow with probability 5%.

4. if in (n-1) day snows then in (n) day will have clouds with probability 90% or will snow with probability 10%

The exercise asks for the following:

1. Markov chain transition matrix
2. Solve the balanced equations and calculate the steady-state probabilities .

 legendoulis April 3rd, 2012 09:55 AM

Re: help in markov chain

i cant make both the Markov chain transition matrix and Solve the balanced equations and calculate the steady-state probabilities .

 soroban April 3rd, 2012 02:21 PM

Re: help in markov chain

Hello, legendoulis!

Quote:
 Suppose the weather in a city on day n is described by a Markov chain [color=beige]. . [/color]and it depends on the (n - 1) day as follows: 1. If on day (n - 1), it rains, then on day n, [color=beige]. . [/color]it rains with probability 60%, or will be fair with probability 40% 2. If on day (n-1), it is fair, then on day n, [color=beige]. . [/color]it will be fair with probability 60%, or be cloudy with probability 40%. 3. If on day (n - 1), it is cloudy, then on day n, [color=beige]. . [/color]it will rain with probability 20%, will be fair with probability 20%, [color=beige]. . [/color]will be cloudy with probability 55%, or will snow with probability 5%. 4. If on day (n - 1), it snows, then on day n, [color=beige]. . [/color]it will be cloudy with probability 90%, or will snow with probability 10%. The exercise asks for the following: [color=beige]. . [/color](1) Markov chain transition matrix [color=beige]. . [/color](2) Solve the balanced equations and calculate the steady-state probabilities .

$\text{W\!e have this information:}$

[color=beige]. . [/color]$\begin{array}{c|cccc|} & \text{rain} & \text{fair} & \text{cloudy} & \text{snow} \\ \hline\\ \\
\text{rain} & 0.60 & 0.40 & 0 & 0 \\ \\
\text{fair} & 0 & 0.60 & 0.40 & 0 \\ \\
\text{cloudy} & 0.20 & 0.20 & 0.55 & 0.05 \\ \\
\text{snow} & 0 & 0 & 0.90 & 0.10 \end{array}$

$\text{(1) The transition matrix is: }\:P \;=\;\begin{pmatrix}0.60 &0.40=&0 \\ \\ 0=&0.60=&0.40=&0 \\ \\ 0.20=&0.20=&0.55=&0.05 \\ \\ 0=&0.90=&0.10 \end{pmatrix}=$

$\text{(2) To find the steady-state probabilities, we set up:}$

[color=beige]. . [/color]$(a,\,b,\,c,\,d)\,\begin{pmatrix}0.60=&0.40=&0 \\ \\ 0=&0.60=&0.40=&0 \\ \\ 0.20=&0.20=&0.55=&0.05 \\ \\ 0=&0.90=&0.10 \end{pmatrix} \;=\;(a,\,b,\,c,\,d)$

$\text{This gives us four equations:}$

[color=beige]. . [/color]$\begin{Bmatrix}0.60a\,+\,0.20c &=& a \\ \\ 0.40a\,+\,0.60b\,+\,0.20c &=& b \\ \\
0.40b\,+\,0.55c\,+\,0.90d &=& c \\ \\ 0.05c\,+\,0.1d &=& d \end{Bmatrix}\;\;\;\Rightarrow\;\;\;\begin{Bmatrix }-0.40a\,+\,0.20c &=& 0 \\ \\ 0.40a\,-\,0.40b\,+\,0.20c &=& 0 \\ \\ 0.40b\,-\,0.45c\,+\,0.90d &=& 0 \\ \\ 0.05c\,-\,0.90d &=& 0 \end{Bmatrix}$

$\text{Eliminating decimals, we have: }\:\begin{Bmatrix}-2a\,+\,c=&[1] \\ \\ 2a\,-\,2b\,+\,c=&[2] \\ \\ 8b\,-\,9c\,+\,18d=&[3] \\ \\ c\,+\,18d=&[4] \end{Bmatrix}=$

$\text{But these four equations are }dependent.$
$\text{We must disregard one of them, say [4], and include: }\:a\,+\,b\,+\,c\,+\,d \:=\:1$

$\text{So we have this system of equations: }\:\begin{Bmatrix}a\,+\,b\,+\,c\,+\,d &=& 1 \\ \\
-2a \,+\,c &=& 0 \\ \\ 2a\,-\,2b\,+\,c &=& 0 \\ \\ 8b\,-\,9c\,+\,18d &=& 0 \end{Bmatrix}$

$\text{Solve the augmented matrix: }\:\left|\begin{array}{cccc|c} 1&1&1&1&1 \\ \\-2 & 0 & 1 & 0 & 0 \\ \\ 2 & -2 & 1 & 0 & 0 \\ 0 & 8 & -9 & 18 & 0 \end{array}\right|$

[color=beige]. . [/color]$\text{and we get: }\:(a,\,b,\,c,\,d) \;=\;\left(\frac{1}{10},\:\frac{1}{5},\:\frac{1}{5 },\:\frac{1}{2}\right)$

 legendoulis April 3rd, 2012 03:48 PM

Re: help in markov chain

thank you very very match soroban for the answer.

I have one more thing:

We have to simulate the Markov chain that we created i will explain:

You can start the chain from whichever situation you want.
1. In every step of the simulation we must create a random number .
2. So when we finish we use the transition matrix to go to the next situation
and so on
The simulation must have time equal to 1080 steps of the previous (1. and 2.)

 legendoulis April 4th, 2012 11:28 AM

Re: help in markov chain

We need to find the Markov chain diagram from the data that we have

 All times are GMT -8. The time now is 05:49 AM.