Module 2 : Markov Processes and Markov Chains

Lecture 2 : Markov Chains, Birth-Death Processes, Flow Balance


pk(t) = P{X(t)=k}

 

                                                          = Probability that system is in state k at time t


State Transition Diagram for a Birth-Death Process

Note that when the system is in state k, we assume that the arrival rate to the system is λk and the departure rate from the system is μk. Therefore, in a time interval Δt, the system will go from state k to state k +1 with probability λkΔt and will go from state k to state k -1 with probability μkΔt. This allows us to write the state transition equations given in the next slide.

The state transitions from time t to t + Δt will then be governed by the following equations -


For Δt0, we then get
(2.1)