Let be the estimate of the random variable based on the random variable .Clearly is a function of .We have to find best estimate be in some meaningful sense. Observe that

• is the unknown random variable



• is the estimate of.



• is the estimation error.



• is the mean of the square error.
  
  One meaningful criterion is to minimize with respect to and the corresponding estimation principle is called the minimum mean-square error principle . Such a function which we want to minimize is called a cost function in optimization theory . For finding we have to minimize the cost function .