Recurrence

Consider an arbitrary state, , which is fixed, such that for all integers, , we define: , which is basically the probability that after having started from the  state it comes back to  state for the first time ONLY at the  transition. The definition makes it clear that  and  for , and we also define Now let us consider the simple illustrations (Figure 1.19 and Figure 1.20).

Figure 1.19: Movement from  state back to  state considering that there may or may not be any reoccurrence or visits to  state during the time period

Figure 1.20: Movement from  state back to  state considering that there may or may not be any reoccurrences or visits at  state in between

From both the figures (Figure 1.19 and Figure 1.20) it is clear that  and  and the first return to state  occurs at  transition. If this return is denoted by event , then the events   are mutually exclusive.

Now the probability of the event that the first return is at the  transition is  which we already know. Now for the remaining  transitions we will only deal with those such that  holds true. In case it does not, we will not consider that.

So we have

for   and we also have

Hence

 as