Annu. Rev. Astron. Astrophys. 1992. 30: 653-703 Copyright © 1993 by Annual Reviews. All rights reserved

2.2 Analysis Procedures

The question of the best strategy in analyzing microwave background radiation observations is not fundamental but it is important because the data are exceedingly difficult to obtain and, as of this writing, no clear signal due to the microwave background radiation fluctuations has been reported. It is therefore reasonable to push statistical analysis to its limits in extracting information from these data, and parametric methods, which can be pushed much further than nonparemetric methods, are almost always used. Once fluctuations have been measured with good signal-to-noise ratio, questions of statistical procedures will largely disappear and work will focus on the actual distribution of fluctuations.

Considerable disagreement exists about the best techniques to use in statistical analyses of these types of data. There are two main points of view, ``frequentist'', and ``Bayesian'', described briefly below. We refer the reader to Readhead et al (1989) and references therein for more detailed discussions. Despite apparent similarities, and many common terms and definitions, there is a fundamental difference between these two points of view. Frequentist methods are absolute, in the sense that the probability of obtaining a certain set of data is evaluated by considering all datasets that might have been obtained, given our assumptions about the sky and knowledge of instrumental errors. Bayesian methods, on the other hand, are relative, in that the absolute probability of obtaining a certain set of data is of little interest. What matters is only the relative probability as the assumptions about the sky are varied. Frequentist methods tend to go awry with unlikely datasets, because the frequentist credo is good long-run performance, and what happens to unlikely datasets doesn't make much difference in the evaluation of long-run performance. Bayesian methods avoid this difficulty, because they do not consider long-run performance.

2.2.1 BAYESIAN ANALYSIS We denote the probability density for a measurement T ± by p(T, ), and the variance of the distribution of microwave background radiation fluctuations on the sky by T²_mbr. For Gaussian fluctuations the probability density is entirely determined by W(k) and we have

(4)

In observations of n fields we have the set of observations {T_i}, and the likelihood function, L({T_i}|T_mbr), is defined by the joint density function for the n fields. In the case that the n fields are uncorrelated we have:

(5)

When the fields are correlated the joint density function must include the correlation matrix (Kendall & Stuart 1977 vol 1). From Bayes's formula, the probability density of T_mbr is

(6)

where p(T_mbr), called the prior probability density, represents our knowledge of T_mbr before the observations. p(T_mbr|{T_i}) is called the posterior probability density. Since we have, in fact, no knowledge of T we must assume a prior probability density. The usual practice is to make some such simple assumption as p(T_mbr) = c for T_mbr >> 0. Note that p(T_mbr) = 0 for T_mbr < 0.

2.2.2 THE LIKELIHOOD RATIO METHOD Likelihood ratio tests are the most commonly used frequentist method in the analysis of microwave background data (e.g. Boynton & Partridge 1973). Given a set of measurements whose distribution is known in terms of the (assumed) parameter T_mbr, we must find T_mbr. The test is simply a criterion by which we decide between an hypothesis H (e.g., that T_mbr 15 µK), and an alternative hypothesis K (e.g., that T_mbr > 15 µK). The level of significance or size of the test is the probability of rejecting H if H is true (a so-called Type I error). The power of the test, , is the probability of rejecting H if K is true. Thus 1- is the probability of accepting H when K is true (a so-called Type II error). Clearly we would like a test for which is low, and is high. It can be shown that a test based on the statistic S, defined as the ratio of the likelihoods of the observations T_i under H and under K, maximizes the power for any given size of the test.

Such tests are called powerful for obvious reasons, but it is important to remember that maximizing the power does not guarantee that the power is high. Likelihood ratio tests have low power, in fact, on data where ² per degree of freedom, ², is much less than unity.

2.2.3 APPLICATION TO OBSERVATIONS Both of the above approaches have been used in analyzing observations of the microwave background radiation and over the last few years much print has been devoted to discussions of the ``best'' approach. We adopt the pragmatic view that each method of analysis has its strengths and weaknesses and is sensitive in different ways to peculiarities in the data. In general, we prefer the Bayesian method, because it is better-behaved when the dataset has <sup>2</sup> (reduced <sup>2</sup>) much less than unity - which can happen if measurement errors are overestimated or if the data are simply atypical (Readhead et al 1989). In such cases the frequentist ``likelihood- ratio'' method can give misleading results if the power of the test is low (Lasenby 1981, Cottingham 1987, Lawrence et al 1988, Readhead et al 1989). For well-behaved datasets, with ² 1, however, both frequentist and Bayesian methods give comparable results. In our opinion, how a method is used matters more than which one is used.

Given the importance that we attach to observations of the microwave background radiation, it seems wise to make use of all the available statistical tools, and to base the interpretation of the observations upon those aspects upon which different methods agree. Indeed, disagreement amongst statistical methods is a sure sign of statistical peculiarities in the data. Nevertheless statistical analyses can easily obscure, rather than illuminate, the observations themselves. For this reason we reproduce below, wherever possible, the actual T measurements with their errors, since these convey, more clearly than tables of 95% confidence limits, the real progress in this field over the last decade.