In an irreducible, aperiodic, homogenous Markov Chain, the limiting state probabilities p_j = P{state j} always exist and these are independent of the initial state probability distribution

                                                                                        and

either

All states are transient, or all states are recurrent null - in this case, the state probabilities pj 's are zero for all states and no stationary state distribution will exist.

or	All states are recurrent positive - in this case a stationary distribution giving the equilibrium state probabilities exists and is given by pj=1/ Mj ∀ j .

Interpreting this for queueing, we will find that -

(a) When the queue is in equilibrium, all its states will be recurrent positive with a stationary state distribution which will be ergodic as well. That implies that if we let the queue operate for a long time T and find that overall, it spent time T_j in state j then the probability of finding the queue in state j will be the same as T_j /T. It is also important to note that all states (including state 0, i.e. the state when the queue is empty) will have non-zero probabilities of occurrence. This is the reason why one sometimes makes the statement that a queue in equilibrium is guaranteed to be empty some time or the other! It is also important to note that, in this case, the state distribution probability of the queue will be the same regardless of the initial state with which the queue is started.

(b) If the queue is overloaded (i.e. maximum service rate is less than the arrival rate) then the queue will not be in stable equilibrium. In this case, all its states will be transient as the overall tendency of the queue will be that the number in the queue will keep increasing - it will tend to infinity for a queue with infinite buffer or the buffer will become completely full if it has only a finite buffer. Moreover, the probability of finding the queue empty will be zero.

The stationary distribution of the states (i.e. the equilibrium state probabilities ) of an irreducible, aperiodic, homogenous Markov Chain (which will therefore also be ergodic ), may be found by solving a set of simultaneous linear equations and a normalization condition, as given below.

∀j

Balance Equations

Normalization Condition

If system has N states, j = 0,1, ....., (N -1) ), then we need to solve forp₀ , p₁ ,….., p_{N -1} using the Normalization Condition and any (N -1) equations from the N Balance Equations.

Note that the Normalization Condition has to be used as the N Balance Equations will not be linearly independent!