Module 4 : Maximum Likelihood Estimation (MLE)

Lecture 11 : MLE in General Cases

4.3.1 MLE for Transformed Parameters

The MLE of the transformed parameter, α = g(θ) is given by:

αˆ=  g(ˆθ)

where ˆθ is the MLE of θ. If g is not one-to-one function (i.e., not invertible) then ˆα is obtained as the MLE of transformed likelihood function, pT (x; α), which is defined as:

pT(x;α ) = {θ:mαa=xg(θ)}p(x;θ )

4.3.2 Example

4.3.3 MLE for General Linear Model

Consider the general linear model of the form:

x = Hθ + w

where H is a known N × p matrix, x is an N × 1 observation vector with N samples, and w is N × 1 noise vector with PDF N(0,C). The PDF of the observed data is:

                1           [  1                        ]  p(x;θ) =  ----N----1----exp  - -(x - H  θ)TC -1(x - H θ)            (2 π)2 det2(C )       2

and the MLE of θ is found by differentiating the log-likelihood which can be shown to yield:

∂-ln-p(x;θ-)   ∂(H-θ-)T  -1      ∂θ      =    ∂θ   C   (x - H θ)

which upon simplification and setting to zero becomes:

HT C - 1(x -  H θ) = 0

and this yields the MLE of θ as:

 ˆ      T  -1   -1  T  -1  θ =  (H  C   H )  H  C   x

which turns out to be same as the MVU estimator.