Module 1:Concepts of Random walks, Markov Chains, Markov Processes
Lecture 1:Introduction to Stochastic Process
Claim:Given (or its probability) and , then the whole probabilistic structure of the process is determined.
Proof: We need to compute ,
Thus:
i.e., we use the concept of
Now
Similarly we have:
Using this we can find the . Finally the probability of starting from the initial position will be given by the problem or from the practical situation, based on which we have formulated our problem.
(or its probability) and 
, then the whole probabilistic structure of the process is determined.
, 
. Finally the probability of starting from the initial position will be given by the problem or from the practical situation, based on which we have formulated our problem.
