Example 1.28

Assume , for which we have the following transition diagram

What is of interest to us is the eigen value of . Using basic concepts we have det(P-l)=0, i.e., , hence , which implies  and , i.e., for , , all eigen values are ≤1.
In case we are interested to find , we have  and as   .

Suppose we are at step 0, and we may be interested to find the expected time until we return to step 0. Hence if T is the time until first return, then
,   ,  ,

 

 

 

 

Thus

Note in case you start at state 1, then the expected time until return is 3.5.