My Math Forum (http://mymathforum.com/math-forums.php)
-   -   Markov chain or another solution? (http://mymathforum.com/advanced-statistics/342695-markov-chain-another-solution.html)

 Falsetto November 6th, 2017 06:36 PM

Markov chain or another solution?

Ok, here's the problem.

I am rolling 2 fair dice repeatedly and I want to know what the chance is of rolling both a total of 5 and a total of 9 before rolling a total of 7.

Let y equal the probability of a 5 or 9 or 7 being rolled. There are 4 ways each to roll a 5 and 9 and 6 ways to roll a seven for a total of 14/36 combinations.

I calculate P(5|y) = 1/9 / 7/18 or 1/9 * 18/7 = 18/63 = 2/7

P(9|y) must equal P(5|y) so also 2/7

P(7|y) = 1 - (P(5|y) + P(9|y)) = 3/7

So far so good?

Now I'm not sure if I'm on the right track next or not but . . .

If I imagine this problem as having 5 states:-

a) Having made 0 of the numbers yet.
b) Having rolled a 5 but neither a 9 nor 7.
c) Having rolled a 9 but neither a 5 nor 7.
d) Having succeeded in rolling both a 5 and a 9 without the 7 (won).
e) Having rolled a 7 and failed (lost).

I then made a transition matrix from that data.

a b c d e
a 0 2/7 2/7 0 3/7

b 0 2/7 0 2/7 3/7

c 0 0 2/7 2/7 3/7

d 0 0 0 1 0

e 0 0 0 0 1

(EXCUSE THE FORMATTING)

Since I always start in state 'a' I multiplied an initial distribution vector of
[1 0 0 0 0] by the above matrix raised to the power of 50 to try and find an approximation of the steady state probabilities.

This told me that P(5 and 9 before 7) = 0.2286 (ish)

Anyone make any sense of that and tell me if I got the answer right? If yes, could I have done it in an easier way? If not, what should I have done?

Thanks in advance for your help folks!

 romsek November 6th, 2017 08:27 PM

This is how I did it.

Consider sequences of rolls that lead to a win in $n$ rolls.

Note $n \geq 2$

Roll $n$ will either be a $5$ or a $9$

Rolls $1 \text{ through }(n-1)$ will not be roll $n$ and will contain no $7$

Now consider roll sequences of length $n-1$ that satisfy this.

There will be at least $1$, and up to $n-1$ rolls that are either $5$ or $9$ whichever isn't roll $n$, and none of these will be $7$.
Otherwise they can be any roll.

so we end up with

$P[\text{win}] = 2 \sum \limits_{n=2}^\infty \dfrac 1 9\left(\sum \limits_{k=1}^{n-1}\dbinom{n-1}{k}\left(\dfrac 1 9\right)^k\left(\dfrac{11}{18}\right)^{n-1-k}\right)$

Digesting this a bit.

The $2$ in front is because roll $n$ can be either $5$ or $9$, and the probability is identical for both.

The $\dfrac 1 9$ is the probability of roll $n$ being the $5$ (or the $9$)

What's inside the parens is the sum of probabilities that there are $k$ occurrences of $5$ or $9$ whichever isn't roll $n$

This can be simplified to

$P[\text{win}] = 2 \sum\limits_{n=2}^\infty \left(\left(\dfrac{13}{18}\right)^{n-1} - \left(\dfrac{11}{18}\right)^{n-1}\right)$

and then finally to

$P[\text{win}] =\dfrac{8}{35}$

I leave it to you to digest all of this and reproduce the simplifications in more detail.

ps. $p=\dfrac {8}{35}$ matches my sim of this very well so I'm pretty confident in this answer.

 Falsetto November 6th, 2017 08:43 PM

THANKYOU!!

Exactly what i was hoping to find. And also the answer i had was correct as well :)

But a formula like yours is so much better. I will study it in detail tomorrow when i wake up.

Thanks for your time and help. Really appreciate it!

 Falsetto November 15th, 2017 05:58 PM

Trying to digest what you did I figured out that there is a lot I still don't know.

I have had a look into geometric sequences and that is helping me get started. Are there any other topics you would recommend me learning more about to gain a better grasp of what's going on here? Haha

Also, in playing with the problem I found another way to work it out I think.

Since we only care about 5,9 and 7.

Can I say that P(5 or 9 | 5,9,7) = 8/14 and then the probability of the next one is p(5 | 5,7) = 4/10

8/14 * 4/10 = 32/140 = 8/35 again.

Think this only works is a case like this though where the P(n) is the same for both. How would your formula above change for a problem like rolling 4 and 5 before 7?

 romsek November 15th, 2017 09:47 PM

Quote:
 Originally Posted by Falsetto (Post 584331) Trying to digest what you did I figured out that there is a lot I still don't know. I have had a look into geometric sequences and that is helping me get started. Are there any other topics you would recommend me learning more about to gain a better grasp of what's going on here? Haha Also, in playing with the problem I found another way to work it out I think. Since we only care about 5,9 and 7. Can I say that P(5 or 9 | 5,9,7) = 8/14 and then the probability of the next one is p(5 | 5,7) = 4/10 8/14 * 4/10 = 32/140 = 8/35 again. Think this only works is a case like this though where the P(n) is the same for both. How would your formula above change for a problem like rolling 4 and 5 before 7?
Your solution is the way to go. Very nicely thought out.

 All times are GMT -8. The time now is 11:02 PM.