Next Contents Previous

2.7. Orthogonal series estimators

Orthogonal series estimators approach the density estimation problem from quite a different point of view. They are best explained by a specific example. Suppose that we are trying to estimate a density f on the unit interval [0, 1]. The idea of the orthogonal series method is then to estimate f by estimating the coefficients of its Fourier expansion.

Define the sequence phinu(x) by


Then, by standard mathematical analysis, f can be represented as the Fourier series sumnu=0infty fnu phinu, where, for each nu geq 0,

Equation 2.6 (2.6)

For a discussion of the sense in which f is represented by the series, see, for example, Kreider et al. (1966).

Suppose X is a random variable with density f. Then (2.6) can be written


and hence a natural, and unbiased, estimator of fnu based on a sample X1,..., Xn from f is


Unfortunately, the sum sumnu=0infty hat fnu phinu will not be a good estimate of f, but will `converge' to a sum of delta functions at the observations; to see this, let

Equation 2.7 (2.7)

where delta is the Dirac delta function. Then, for each nu,


and so the hat fnu are exactly the Fourier coefficients of the function omega.

In order to obtain a useful estimate of the density f, it is necessary to smooth omega by applying a low-pass filter to the sequence of coefficients hat fnu. The easiest way to do this is to truncate the expansion sum hat fnu phinu at some point. Choose an integer K and define the density estimate hat f by

Equation 2.8 (2.8)

The choice of the cutoff point K determines the amount of smoothing.

A more general approach is to taper the series by a sequence of weights lambdanu, which satisfy lambdanu rightarrow 0 as nu rightarrow infty, to obtain the estimate


The rate at which the weights lambdanu converge to zero will determine the amount of smoothing.

Other orthogonal series estimates, no longer necessarily confined to data lying on a finite interval, can be obtained by using different orthonormal sequences of functions. Suppose a(x) is a weighting function and (psinu) is a series satisfying, for µ and nu geq 0,


For instance, for data resealed to have zero mean and unit variance, a(x) might be the function e-x2/2 and the psinu multiples of the Hermite polynomials; for details see Kreider et al. (1966).

The sample coefficients will then be defined by


but otherwise the estimates will be defined as above; possible estimates are

Equation 2.9 (2.9)


Equation 2.10 (2.10)

The properties of estimates obtained by the orthogonal series method depend on the details of the series being used and on the system of weights. The Fourier series estimates will integrate to unity, provided lambda0 = 1, since


and hat f0 will always be equal to one. However, except for rather special choices of the weights lambdanu, hat f cannot be guaranteed to be non-negative. The local smoothness properties of the estimates will again depend on the particular case; estimates obtained from (2.8) will have derivatives of all orders.

Next Contents Previous