Note/Remark

• Recurrence and periodicity are class property, which means that all states in an equivalence class are either recurrent or non-recurrent.
• The expected number of returns to state , given that  is given by , hence a state  is recurrent iff the expected number of returns is infinite.

Let us define two power series

(a) , where

(b) , where

With these definitions we can now make the following claim which is



 

Now we also know that , i.e.,

Hence we have: , i.e.,