The above discussion concerned analysis of data which was in hand. Another important application of statistical techniques is to forecast the outcomes of future experiments, which is increasingly expected by funding agencies wishing to compare the capabilities of competing proposals. A Figure of Merit (FoM) is defined for each experiment, and used to rank experiments. More ambitiously, the same techniques can be used to optimize the design of a survey in order to maximize the likely science return (e.g. Ref. [43]).

The leading approach presently is the Fisher information matrix
approach, introduced to cosmology in Ref.
[44]
and popularized especially when adopted by the initial report of the
Dark Energy Task Force
[45].
The Fisher matrix measures the second derivatives of the likelihood
around its maximum, and in a gaussian approximation is used to estimate
the expected uncertainty in parameters around some selected fiducial
model. For instance, the DETF FoM considers the uncertainty on the two
parameters of the dark energy equation of state defined by the form
*w* = *w*_{0}
+ (1 − *a*)*w*_{a}, taking the fiducial model to
be ΛCDM (*w*_{0} = −1, *w*_{a} =
0), defining the FoM to be the inverse area of the 95% confidence
region. The likelihood is determined via a model of how well the
instrument will perform, possibly through analysis of a simulated data
stream.

An important caveat is the following: an experiment capable of reducing
the volume of permitted parameter space by say a factor of 10 should in
no way be considered as having a 90% chance of measuring those
parameters as different from some special value. An example is whether
upcoming data can exclude the cosmological constant model in favour of
dark energy. What the Fisher FoM shows is the reduction in allowed
parameter volume *provided that the dark energy model is
correct*. However that assumption is actually what we wish to test;
it is perfectly possible that it is the cosmological constant model
which is correct and then no amount of improvement to the uncertainty
will rule it out. Once again, to develop a full picture we need to
consider multiple models, and hence model selection statistics.

**5.2. Model selection approaches**

Model selection forecasting, pioneered for cosmology in Refs. [36, 46], instead forecasts the ability of upcoming experiments to carry out model comparisons. Let's restrict to the simplest case of two models, one nested within another, though the generalization to other circumstances is straightforward.

The very simplest model selection question is to assume that the nested model (say ΛCDM) is true, and ask whether a given experiment would be able to rule out the more complex alternative. Data is simulated only for the ΛCDM model, as with the Fisher analysis, and the Bayes factor between ΛCDM and the parametrized dark energy model calculated. This shows how strongly the experiment will rule out the dark energy model if it is wrong.

More generally, one may wish to consider the dark energy model as the correct one, and ask whether the simpler model can be excluded. This is more complex, as the outcome depends on the actual parameters of the dark energy model (known as the fiducial parameters), and hence has to be considered as a function of them. This leads to the concept of the Bayes factor plot, showing the expected Bayes factor as a function of fiducial parameters [46]. A suitable FoM may be to minimize the area in fiducial parameter space in which the wrong model cannot be ruled out by the proposed experiment. Alternatively, one can study the distribution of the Bayes factor weighted by present knowledge of the parameters, to predict the probability distribution of expected outcomes of the experiment [36].

Examples of cosmological model selection forecasts can be found in Refs. [36, 46, 38].