Stochastic Process    

Since arrivals may come to a queue at random instants and since the service provided by the servers will also generally be random, we would need to consider arrival processes and departure processes to be random processes when studying a queue or a queuing network.

A Stochastic Process (also called a random process) is a family of random variables indexed by the parameter t in T. For a given choice of the time instant t, different realizations of the stochastic process will generate random values of X at the selected time instant t. Alternatively one can view the stochastic process to be one which generates a random function of time for every realization of the process.

We consider the the stochastic process X(t) to take on the random values X(t₁)=x₁,......, X(t_n)=x_n ,.......at times t₁,....,t_n,....... The random variables x₁,........, x_n,........ are then specified by specifying their joint distribution. One can also choose the time points t₁,....,t_n,....... where the process is actually examined in different ways.

For our purposes, we need to consider a specific type of random process where past history of the process can be neglected if one knows the current state of the process. This property is referred to as the Markov property. Apart from the fact that this property can be naturally observed in many processes, assuming this also makes the analysis reasonably tractable in the case where we want to study queues and queueing networks.

Markov Processes                                

The stochastic process X(t) is referred to as a Markov Process if it satisfies the Markovian Property (i.e. memoryless property). This states that -

P{X(t_n+1)=x_n+1 | X(t_n)=x_n ...... X(t₁)=x₁} = P{X(t_n+1)=x_n+1 | X(t_n)=x_n}

for any choice of time instants t_i , i =1,……, n where t_j > t_k for j>k .

Note that we use P(A) to denote the probability of event A and P(A | B) as the probability of the event A given that event B has happened, i.e. the event (A | B) .

This is referred to as the Memoryless property as the state of the system at future time t_n+1 is only decided by the system state at the current time t_n and does not depend on the state at the earlier time instants t₁ ,…….., t_n-1