Module 12:Application of stochastic processes in areas of engineering and management science
Lecture 39:Application of Markov chain in trying to judge the efficiency of algorithms in OR
The question which is important for us to answer is, what happens when the search is probabilistic or stochastic. Let us consider the example for which we have and , where and . Let denote the number of transitions one needs to go from state to state 1. So given the initial condition we have and with the initial condition that , we can obtain all the successive expected values.
Now if where , then we can prove
(i)
(ii)
(iii) For large , follows a Poisson process with a mean of
and 
, where 
and 
. Let 
denote the number of transitions one needs to go from state 
to state 1. So given the initial condition we have 
and with the initial condition that 
, we can obtain all the successive expected values.
where 
, then we can prove
, 
follows a Poisson process with a mean of 
