Module 1:Concepts of Random walks, Markov Chains, Markov Processes
Lecture 3:Markov Chains
(i) ,
(ii)
Thus we have
The above result can be obtained if we note that . Now this fact that is true for this case as , else we have to rework the whole problem.
Again going back to , we see that for being a large value we have when , i.e., . Thus we have , and again using Stirling's formula or approximation, which is , we have the right hand side as , when . But if we have to find , , …., then the sum, i.e., , which is the property of transient state and notrecurrentstate.
, 
. Now this fact that 
is true for this case as 
, else we have to rework the whole problem.
, we see that for 
being a large value we have 
when 
, i.e., 
. Thus we have
, and again using Stirling's formula or approximation, which is 
, we have the right hand side as 
, when 
. But if we have to find 
, 
, …., then the sum, i.e., 
, which is the property of 
