**4.2 The Maximum Likelihood Method**

The method of maximum likelihood is only applicable if the form of the
theoretical distribution from which the sample is taken is known. For
most measurements in physics, this is either the Gaussian or Poisson
distribution. But, to be more general, suppose we have a sample of n
independent observations *x*_{1}, *x*_{2},
. . . ,*x*_{n}, from a theoretical
distribution *f*(*x* | )
where is the parameter to be
estimated. The method then consists of calculating the *likelihood*
function,

which can be recognized as the probability for observing the sequence
of values *x*_{1}, *x*_{2}, . . .,
*x*_{n}. The principle now states that this
probability is a maximum for the observed values. Thus, the parameter
must be such that *L* is a
maximum. If *L* is a regular function,
can
be found by solving the equation,

If there is more than one parameter, then the partial derivatives of *L*
with respect to each parameter must be taken to obtain a system of
equations. Depending on the form of *L*, it may also be easier to
maximize the logarithm of *L* rather than *L* itself. Solving
the equation

then yields results equivalent to (30). The solution,
, is known as
the maximum likelihood *estimator* for the parameter
. In order to
distinguish the estimated value from the true value, we have used a
caret over the parameter to signify it as the estimator.

It should be realized now that
is also a random
variable, since it
is a function of the *x _{i}*. If a second sample is taken,
will have a
different value and so on. The estimator is thus also described by a
probability distribution. This leads us to the second half of the
estimation problem: What is the error on
? This is given by the
standard deviation of the estimator distribution We can calculate this
from the likelihood function if we recall that

This is a general formula, but, unfortunately, only in a few simple cases can an analytic result be obtained. An easier, but only approximate method which works in the limit of large numbers, is to calculate the inverse second derivative of the log-likelihood function evaluated at the maximum,

If there is more than one parameter, the matrix of the second derivatives must be formed, i.e.,

The diagonal elements of the inverse matrix then give the approximate variances,

A technical point which must be noted is that we have assumed that
the mean value of is the
theoretical . This is a
desirable, but not
essential property for an estimator, guaranteed by the maximum
likelihood method only for infinite *n*. Estimators which have this
property are *non-biased*. We will see one example in the following
sections in which this is not the case. Equation (32), nevertheless,
remains valid for all ,
since the error desired is the deviation from
the true mean irrespective of the bias.

Another useful property of maximum likelihood estimators is
invariance under transformations. If *u* =
*f*(), then the best estimate
of *u* can be shown to be =
*f*().

Let us illustrate the method now by applying it to the Poisson and Gaussian distributions.