Classification of States for a Markov Chain

The states of a Markov Chain may be classified as being either Recurrent or Transient as shown in the figure given below. Recurrent states may be further divided into Recurrent Null or Recurrent Non-null. A state which is Recurrent Non-null is also referred to as being Positive Recurrent. It may be noted that the all the states of a queue will be Positive Recurrent under equilibrium conditions.

	fj (n) = P{system returns to state j exactly n steps after leaving state j }

f_j = P{system returns to state j some time after leaving state j }

M _j = Mean recurrence time for state j

      = Mean number of steps to return to state j after leaving state j

State j is periodic with respect to a ( a >1), if the only possible steps in which state j may occur are a , 2 a , 3 a ............... .

In that case, the recurrence time for state j has period a . State j is said to be aperiodic if a =1

A recurrent state is said to be ergodic if it is both positive recurrent and aperiodic .

An ergodic Markov chain will have all its states as ergodic .

An Aperiodic , Irreducible, Markov Chain with a finite number of states will always be ergodic .

The states of an Irreducible Markov Chain are either all transient, or all recurrent null or all recurrent positive. If the chain is periodic, then all states have the same period a .