My Math Forum help in markov chain

 April 3rd, 2012, 09:35 AM #1 Newbie   Joined: Apr 2012 Posts: 4 Thanks: 0 help in markov chain Hi i have an exercise and have a trouble with it if anyone can help me i would appreciate it plz help if you can. This is the exercise: Suppose the weather in a city in the (n) day is described from a Markov chain and it depend on the (n-1) day as follows: 1. if in (n-1) day rains then in (n) day rains with probability 60% or will be sunshine with probability 40% 2. if in (n-1) day has sunshine then in (n) day will have sunshine with probability 60% or will have clouds with probability 40%. 3. if in (n-1) day have clouds then in (n) day will rain with probability 20% , will have sunshine with probability 20% , will have clouds with probability 55% or will snow with probability 5%. 4. if in (n-1) day snows then in (n) day will have clouds with probability 90% or will snow with probability 10% The exercise asks for the following: 1. Markov chain transition matrix 2. Solve the balanced equations and calculate the steady-state probabilities . Thanks in advance
 April 3rd, 2012, 09:55 AM #2 Newbie   Joined: Apr 2012 Posts: 4 Thanks: 0 Re: help in markov chain i cant make both the Markov chain transition matrix and Solve the balanced equations and calculate the steady-state probabilities .
April 3rd, 2012, 02:21 PM   #3
Math Team

Joined: Dec 2006
From: Lexington, MA

Posts: 3,267
Thanks: 408

Re: help in markov chain

Hello, legendoulis!

Quote:
 Suppose the weather in a city on day n is described by a Markov chain [color=beige]. . [/color]and it depends on the (n - 1) day as follows: 1. If on day (n - 1), it rains, then on day n, [color=beige]. . [/color]it rains with probability 60%, or will be fair with probability 40% 2. If on day (n-1), it is fair, then on day n, [color=beige]. . [/color]it will be fair with probability 60%, or be cloudy with probability 40%. 3. If on day (n - 1), it is cloudy, then on day n, [color=beige]. . [/color]it will rain with probability 20%, will be fair with probability 20%, [color=beige]. . [/color]will be cloudy with probability 55%, or will snow with probability 5%. 4. If on day (n - 1), it snows, then on day n, [color=beige]. . [/color]it will be cloudy with probability 90%, or will snow with probability 10%. The exercise asks for the following: [color=beige]. . [/color](1) Markov chain transition matrix [color=beige]. . [/color](2) Solve the balanced equations and calculate the steady-state probabilities .

$\text{W\!e have this information:}$

[color=beige]. . [/color]$\begin{array}{c|cccc|} & \text{rain} & \text{fair} & \text{cloudy} & \text{snow} \\ \hline\\ \\
\text{rain} & 0.60 & 0.40 & 0 & 0 \\ \\
\text{fair} & 0 & 0.60 & 0.40 & 0 \\ \\
\text{cloudy} & 0.20 & 0.20 & 0.55 & 0.05 \\ \\
\text{snow} & 0 & 0 & 0.90 & 0.10 \end{array}$

$\text{(1) The transition matrix is: }\:P \;=\;\begin{pmatrix}0.60 &0.40=&0 \\ \\ 0=&0.60=&0.40=&0 \\ \\ 0.20=&0.20=&0.55=&0.05 \\ \\ 0=&0.90=&0.10 \end{pmatrix}=$

$\text{(2) To find the steady-state probabilities, we set up:}$

[color=beige]. . [/color]$(a,\,b,\,c,\,d)\,\begin{pmatrix}0.60=&0.40=&0 \\ \\ 0=&0.60=&0.40=&0 \\ \\ 0.20=&0.20=&0.55=&0.05 \\ \\ 0=&0.90=&0.10 \end{pmatrix} \;=\;(a,\,b,\,c,\,d)$

$\text{This gives us four equations:}$

[color=beige]. . [/color]$\begin{Bmatrix}0.60a\,+\,0.20c &=& a \\ \\ 0.40a\,+\,0.60b\,+\,0.20c &=& b \\ \\
0.40b\,+\,0.55c\,+\,0.90d &=& c \\ \\ 0.05c\,+\,0.1d &=& d \end{Bmatrix}\;\;\;\Rightarrow\;\;\;\begin{Bmatrix }-0.40a\,+\,0.20c &=& 0 \\ \\ 0.40a\,-\,0.40b\,+\,0.20c &=& 0 \\ \\ 0.40b\,-\,0.45c\,+\,0.90d &=& 0 \\ \\ 0.05c\,-\,0.90d &=& 0 \end{Bmatrix}$

$\text{Eliminating decimals, we have: }\:\begin{Bmatrix}-2a\,+\,c=&[1] \\ \\ 2a\,-\,2b\,+\,c=&[2] \\ \\ 8b\,-\,9c\,+\,18d=&[3] \\ \\ c\,+\,18d=&[4] \end{Bmatrix}=$

$\text{But these four equations are }dependent.$
$\text{We must disregard one of them, say [4], and include: }\:a\,+\,b\,+\,c\,+\,d \:=\:1$

$\text{So we have this system of equations: }\:\begin{Bmatrix}a\,+\,b\,+\,c\,+\,d &=& 1 \\ \\
-2a \,+\,c &=& 0 \\ \\ 2a\,-\,2b\,+\,c &=& 0 \\ \\ 8b\,-\,9c\,+\,18d &=& 0 \end{Bmatrix}$

$\text{Solve the augmented matrix: }\:\left|\begin{array}{cccc|c} 1&1&1&1&1 \\ \\-2 & 0 & 1 & 0 & 0 \\ \\ 2 & -2 & 1 & 0 & 0 \\ 0 & 8 & -9 & 18 & 0 \end{array}\right|$

[color=beige]. . [/color]$\text{and we get: }\:(a,\,b,\,c,\,d) \;=\;\left(\frac{1}{10},\:\frac{1}{5},\:\frac{1}{5 },\:\frac{1}{2}\right)$

 April 3rd, 2012, 03:48 PM #4 Newbie   Joined: Apr 2012 Posts: 4 Thanks: 0 Re: help in markov chain thank you very very match soroban for the answer. I have one more thing: We have to simulate the Markov chain that we created i will explain: You can start the chain from whichever situation you want. 1. In every step of the simulation we must create a random number . 2. So when we finish we use the transition matrix to go to the next situation and so on The simulation must have time equal to 1080 steps of the previous (1. and 2.)
 April 4th, 2012, 11:28 AM #5 Newbie   Joined: Apr 2012 Posts: 4 Thanks: 0 Re: help in markov chain We need to find the Markov chain diagram from the data that we have

 Tags chain, markov

,

,

### markov chains with levels forums

Click on a term to search for related topics.
 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post dr_romix Advanced Statistics 2 October 16th, 2012 06:36 PM 450081592 Advanced Statistics 1 March 6th, 2012 04:17 PM butabi Advanced Statistics 1 February 12th, 2012 03:20 PM CraigSager Advanced Statistics 5 March 23rd, 2011 10:10 AM Turloughmack Linear Algebra 0 February 7th, 2011 05:15 AM

 Contact - Home - Forums - Cryptocurrency Forum - Top