My Math Forum (http://mymathforum.com/math-forums.php)

 butabi February 12th, 2012 04:31 AM

Markov chain

A boy and a girl move to the same two-bar-town on the same day. Each night, boy visits one bar, starting the first night with bar 1 and continues by selecting a bar for the next night according to a Markov chain with transition matrix P below. Similarly, girl visits one bar a night,starting with bar 2 and selecting the next bar according to Q below:

$P= \left[ \begin{array}{cc}
0.7 & 0.3 \\
0.3 & 0.7 \end{array} \right] \; Q= \left[ \begin{array}{cc}
0.4 & 0.6 \\
0.6 & 0.4 \end{array} \right]$

Hint: The progress of boy and girl finding each other can be modelled as a single Markov chain where only one (!) state is absorbing.
a) Find the probability that boy visits bar 1 and girl visits bar 2 on the nth night.
b) Let N denote the number of the night when girl meets boy. Compute the expected number of nights E[N] it takes for boy and girl to meet. Hint: Either compute the distributionof N, or develop an invariance equation for E[N].
c) Find the probability that they meet in bar 1.
d) Find the distribution of the time of their meeting (distribution of N).