Next Contents Previous

15. THE LEAST-SQUARES METHOD

Until now we have been discussing the situation in which the experimental result is N events giving precise values x1, ... , xN where the xi may or may not, as the case may be, be all different.

From now on we shall confine our attention to the case of p measurements (not p events) at the points x1, ... , xp. The experimental results are (y1 ± sigma1), ... ,(yp ± sigmap). One such type of experiment is where each measurement consists of Ni events. Then yi = Ni and is Poisson-distributed with sigmai = sqrt[Ni]. In this case the likelihood function is

Equation

and

Equation

We use the notation ybar(alphai; x) for the curve that is to be fitted to the experimental points. The best-fit curve corresponds to alphai = alphai*. In this case of Poisson-distributed points, the solutions are obtained from the M simultaneous equations

Equation

If all the Ni >> 1, then it is a good approximation to assume each yi is Gaussian-distributed with standard deviation sigmai. (It is better to use Nbari rather than Ni for sigmai2 where Nbari can be obtained by integrating ybar(x) over the ith interval.) Then one can use the famous least squares method.

The remainder of this section is devoted to the case in which yi are Gaussian-distributed with standard deviations sigmai. See Fig. 4. We shall now see that the least-squares method is mathematically equivalent to the maximum likelihood method. In this Gaussian case the likelihood function is

Equation 23     (23)

where

Equation 24     (24)

Figure 4

Figure 4. ybar(x) is a function of known shape to be fitted to the 7 experimental points.

The solutions alphai = alphai* are given by minimizing S(alpha) (maximizing w):

Equation 25     (25)

This minimum value of S is called S*, the least squares sum. The values of alphai which minimize are called the least-squares solutions. Thus the maximum-likelihood and least-squares solutions are identical. According to Eq. (11), the least-squares errors are

Equation

Let us consider the special case in which ybar(alphai; x) is linear in the alphai:

Equation

(Do not confuse this f (x) with the f (x) on page 2.)

Then

Equation 26     (26)

Differentiating with respect to alphaj gives

Equation 27     (27)

Define

Equation 28     (28)

Then

Equation

In matrix notation the M simultaneous equations giving the least-squares solution are

Equation 29     (29)

is the solution for the alpha*'s. The errors in alpha are obtained using Eq. 11. To summarize:

Equation 30     (30)

Equation (30) is the complete procedure for calculating the least squares solutions and their errors. Note that even though this procedure is called curve-fitting it is never necessary to plot any curves. Quite often the complete experiment may be a combination of several experiments in which several different curves (all functions of the alphai) may be jointly fitted. Then the S-value is the sum over all the points on all the curves. Note that since w(alpha*) decreases by ½ unit when one of the alphaj has the value (alphai* ± Deltaalphaj), the S-value must increase by one unit. That is,

Equation

Example 5 Linear regression with equal errors

ybar(x) is known to be of the form ybar(x) = alpha1 + alpha2x. There are p experimental measurements (yj ± sigma).Using Eq. (30) we have

Equation

These are the linear regression formulas which are programmed into many pocket calculators. They should not be used in those cases where the sigmai are not all the same. If the sigmai are all equal, the errors

Equation

or

Equation

Example 6 Quadratic regression with unequal errors

The curve to be fitted is known to be a parabola. There are four experimental points at x = - 0.6, - 0.2, 0.2, and 0.6. The experimental results are 5 ± 2, 3 ± 1, 5 ± 1, and 8 ± 2. Find the best-fit curve.

Equation
Equation
Equation
Equation

ybar(x) = (3.685 ± 0.815) + (3.27 ± 1.96)x + (7.808 ± 4.94)x2 is the best fit curve. This is shown with the experimental points in Fig. 5.

Figure 5

Figure 5. This parabola is the least squares fit to the 4 experimental points in Example 6.

Example 7

In example 6 what is the best estimate of y at x = 1? What is the error of this estimate?

Solution: Putting x = 1 into the above equation gives

Equation

Deltay is obtained using Eq. 12.

Equation

Setting x = 1 gives

Equation

So at x = 1, y = 14.763 ± 5.137.

Least Squares When the yi are Not Independent

Let

Equation

be the error matrix-of the y measurements. Now we shall treat the more general case where the off diagonal elements need not be zero; i.e., the quantities yi are not independent. We see immediately from Eq. 11a that the log likelihood function is

Equation

The maximum likelihood solution is found by minimizing

Equation
where
Equation
Generalized least squares sum

Next Contents Previous