Module 1:Concepts of Random walks, Markov Chains, Markov Processes
Lecture 3:Markov Chains
Assignments 1.4
Let , where , then prove that
Assignments 1.5
Consider a Markov chain and it has number of states, then prove (a)If a state can be reached from state , then it can be reached in steps or less. (b)If is a recurrent state, then there exits such that , the probability that first return to state occurs after transition is
, where 
, then prove that
number of states, then prove
can be reached from state 
, then it can be reached in 
steps or less.
is a recurrent state, then there exits 
such that 
, the probability that first return to state 
occurs after 
transition is 
