Example 6.1

Pick a non-trivial  and  matrices and calculate , etc. On can also attempt to find, , etc.

Consider an arbitrary but fixed state . For that define ;.
Then  is the probability (given we are in state  at time ), of the first attainment of state  occuring at the  transition, which may be called the first passage time.

By definition we have ,  and , then one can claim that:
, where , , .

Diagrammatically it can be shown in Figure 6.3.

Figure 6.3: A simple stochastic process

Contrast this with the first order Markov chain.