Important Observation from the results of the earier two slides -

In a homogenous Markov Chain, the distribution of time spent in a state is (a) Geometric for a Discrete Time Markov Chain (b) Exponential for a Continuous Time Markov Chain

The Semi-Markov Process/Chain relaxes this condition but still has the one step memory property

Semi-Markov Process/Chain In these processes, the distribution of time spent in a state can have any arbitrary distribution but the one-step memory feature of the Markovian property is retained, i.e. the system state at the n+1th time instant depends only on the nth time instant but the distribution of time spent in a particular state can be arbitrary.

Discrete-Time Markov Chains

The sequence of random variables X₁, X₂ , .......... forms a Markov Chain if for all n (n=1, 2, ..........) and all possible values of the random variables, we have that -

P{ X_n =j | X₁ =i₁ ...........X_n-1 =i_n-1 }=P{ X_n =j | X_n-1 =i_n-1 }

Note that this once again, illustrates the one step memory of the process since the probability of the system state at the n^th time instant depends only on the system state at the (n-1)^th instant and not on any of the earlier time instants.