3.3.1 Determination of MVUE using sufficient statistics

There are two ways in which one may derive the MVUE based on the sufficient statistics, T(x):

1. Method I: Let be any unbiased estimator of θ. Then
   is the MVUE.
2. Method II: Find some function g such that = g(T(x)) is an unbiased estimator of θ, i.e., E(g(T(x)) = θ, then is the MVUE.

The basis of above mentioned approaches of finding the MVUE lies in the Rao-Blackwell-Lehmann-Scheffe (RBLS) theorem which states that = E( |T(x)) is:

• a valid estimator for θ
• unbiased
• of a variance var() less or equal to the variance of , for all θ
• the MVUE if the sufficient statistics, T(x) is complete

Note that the sufficient statistics, T(x), is complete if there is only one function g(T(x) that is unbiased. That is, if h(T(x)) is another unbiased estimator (E(h(T(x))) = θ) then we must have g = h if T(x) is complete.

The property of completeness of a sufficient statistics depends on its PDF which in turn is determined by the PDF of data. To validate that a sufficient statistic is complete is in general quite difficult. But for many practical cases, in particularly, for exponential family of PDFs the completeness of sufficient statistic holds.

3.3.2 Exponential family of distributions

A study of the properties of probability distributions that have sufficient statistics of the same dimension as the parameter space regardless of the sample size led to the development of what is called the exponential family of distributions. The common members of this family are:

• Binomial distribution
• Exponential distribution
• Gamma distribution
• Goemetric distribution
• Normal distribution
• Rayleigh distribution

On the other hand, there are some distributions which do not belong to the exponential family of PDFs. The examples of those are:

• Uniform distribution
• Cauchy distribution
• Weibull (unless shape parameter is known)
• Laplace (unless mean parameter is zero)

The one-parameter members of the exponential family have probability density or probability mass function of the form

where η(θ) and A(θ) are some function of θ, T(x) is function of data x, and h(x) is purely a function of data, i.e., it does not involve θ.

Suppose that x = {x[0],x[1],…,x[N - 1]} are i.i.d. samples from a member of the exponential family with the parameter θ, then the joint PDF can be expressed as

From this it is apparent by the factorization theorem that T(x) = ∑_n=0^N-1T(x[n]) is a sufficient statistics.

In case of the multi-parameter members of the exponential family, the joint PDF or PMF with parameter set θ = [θ₁,θ₂,…,θ_d]^T can be expressed as

Thus, we have a vector of natural sufficient statistics as

3.3.3 Example

3.3.4 Example