Adapted from P. Coles, 1999, The Routledge Critical Dictionary of the New Cosmology, Routledge Inc., New York. Reprinted with the author's permission. To order this book click here:

From observations of the cosmic microwave background radiation, we know that the Universe was almost (but not quite) homogeneous when this radiation was last in contact with matter, about 300,000 years after the initial Big Bang. But we also know that the Universe around us today (perhaps 15 billion years later) is extremely inhomogeneous: matter is organised into galaxies, clusters of galaxies, superclusters, and so on in a complex hierarchy of large-scale structure. On sufficiently large scales the Universe does indeed begin to look homogeneous (as required by the cosmological principle), but there is clearly a great deal of structure around us that was not present at the stage of the thermal history of the Universe probed by the microwave background.

The need to explain how the lumpy Universe we see today emerged from the relatively featureless initial state of the early Universe calls for a theory of structure formation. There is a standard picture of how this might have happened, and it is based on the relatively simple physics of the gravitational Jeans instability. Since gravity is an attractive force, small fluctuations in the density from place to place get progressively amplified as the Universe evolves, eventually turning into the large structures we observe at the present time. Constructing a complete theory based on this idea is, however, far from straightforward, and no completely successful theory has yet emerged. To see how this has happened, it is instructive to consider the history of structure formation based on gravitational instability.

The first to tackle the problem of gravitational instability in an expanding cosmological model within the framework of general relativity was Evgeny Lifshitz in 1946. He studied the evolution of small fluctuations in the density of a Friedmann model with perturbation theory, using techniques similar to those still used today. The relativistic setting produces entirely similar results to the standard Jeans theory (which was obtained using only Newtonian gravity, and in a static background). Curiously, it was not until 1957 that the evolution of perturbations in a matter-dominated Friedmann model was investigated in Newtonian theory, by William Bonnor. In some ways the relativistic cosmological theory is more simple that the Newtonian analogue, which requires considerable mathematical subtlety.

These foundational studies were made at a time when the existence of the cosmic microwave background radiation was not known. There was no generally accepted cosmological model within which to frame the problem of structure formation, and there was no way to test the gravitational instability hypothesis for the origin of structure. Nevertheless, it was still clear that if the Universe was evolving with time (as Hubble's law indicated), then it was possible, in principle, for structure to have evolved by some mechanism similar to the Jeans process. The discovery of the microwave background in the 1960s at last gave theorists a favoured model in which to study this problem: the Big Bang theory. The existence of the microwave background in the present implied that there must have been a period in which the Universe consisted of a plasma of matter and radiation in thermal equilibrium. Under these physical conditions there are a number of processes, due to viscosity and thermal conduction in the radiative plasma, which could have influenced the evolution of a perturbation with a wavelength less than the usual Jeans length. The pioneering work by Joseph Silk, Jim Peebles, Yakov Zel'dovich and others between 1967 and 1972 represented the first attempts to derive a theory of galaxy and structure formation within the framework of modern cosmology.

At this time there was a rival theory in which galaxies were supposed to have formed as a result of primordial cosmic turbulence: that is, by large-scale vortical motions rather than the longitudinal adiabatic waves that appear in gravitational instability models. The vortical theory, however, rapidly fell from fashion when it was realised that it should lead to large fluctuations in the temperature of the microwave background on the sky. In fact, this point about the microwave background was then, and is now, important in all theories of galaxy formation. If structure grows by gravitational instability, it is, in principle, possible to reconcile the present highly inhomogeneous Universe with a past Universe which was much smoother. The microwave back-ground seemed to be at the same temperature in all directions to within about one part in a hundred thousand, indicating a comparable lack of inhomogeneity in the early Universe. If gravitational instability were the correct explanation for the origin of structure, however, there should be some fluctuations in the microwave background temperature. This initiated a search, which met with success in the 1990s, for fluctuations in the cosmic microwave background.

In the 1970s, the origin of cosmic protostructure was modelled as two-component systems containing baryonic material and radiation. Two fundamental modes of perturbations can exist in such a two-component system: adiabatic perturbations, in which the matter fluctuations and radiation fluctuations are coupled together, and isothermal perturbations, in which the matter component is disturbed but the radiation component is uniform. These two kinds of perturbation evolve in a very different way, and this led to two distinct scenarios for structure formation:

  1. The adiabatic scenario, in which the first structures to form are extremely massive, in the range 1012 to 1014 solar masses (the sizes of clusters of galaxies). This is because small-scale fluctuations in this model are eliminated in the early stages of their evolution by a process known as Silk damping. Galaxies then form by successive processes of fragmentation of these large objects. For this reason, the adiabatic scenario is also called a top-down model.

  2. The isothermal scenario, in which the first structures, protoclouds or protogalaxies, are formed on a much smaller mass scale, of around 105 or 106 solar masses (similar to the mass of a globular cluster). Structures on larger scales than this are formed by successive merger of these smaller objects in a process known as hierarchical clustering. For this reason, the isothermal scenario is described as a bottom-up model.

During the 1970s there was a vigorous debate between the adherents of these two pictures, roughly divided between the Soviet school led by Zel'dovich which favoured the adiabatic scenario, and the American school which favoured hierarchical clustering. Both these models were eventually abandoned: the former because it predicted larger fluctuations in the cosmic microwave back-ground radiation than were observed, and the latter because no reasonable mechanism could be found for generating the required isothermal fluctuations.

These difficulties opened the way for the theories of the 1980s. These were built around the hypothesis that the Universe is dominated by (non-baryonic) dark matter in the form of weakly interacting massive particles (WIMPs). The WIMPs are collisionless elementary particles, perhaps massive neutrinos with a mass of around 10 eV, or some other more exotic species produced presumably at higher energies - perhaps the photino predicted by supersymmetry theory. These models had three components: baryonic material, non-baryonic material made of a single type of WIMP particle, and radiation. Again, as in the two-component system, there were two fundamental perturbation modes: these were curvature perturbations (essentially the same as the previous adiabatic modes) and isocurvature perturbations. In the first mode, all three components are perturbed together, and there is therefore a net perturbation in the energy density and hence a perturbation in the curvature of spacetime. In the second type of perturbation, however, the net energy density is constant, so there is no perturbation to the spatial curvature.

The fashionable models of the 1980s can also be divided into two categories along the lines of the top-down/bottom-up categories mentioned on the previous page. Here the important factor is not the type of initial perturbation, because no satisfactory way has been constructed for generating isocurvature fluctuations, just as was the case for the isothermal model. What counts in these models is the form of the WIMP. The two competing models were:

  1. The hot dark matter (HDM) model, which is in some sense similar to the old adiabatic baryon model. This model starts with the assumption that the Universe is dominated by collisionless particles with a very large velocity dispersion (hence the `hot dark matter'). The best candidate for an HDM particle would be the massive 10 eV neutrino mentioned above. In this model a process of free streaming occurs in the early Universe that erases structure on scales all the way up to the scale of a supercluster (greater than 1015 solar masses). Small-scale structure thus forms by the fragmentation of much larger-scale objects, as before.

  2. The cold dark matter (CDM) model, which has certain similarities to the old isothermal picture. This model starts with the assumption that the Universe is, again, dominated by collisionless WIMPs, but this time with a very small velocity dispersion (hence the `cold'). This can occur if the particles decouple when they are no longer relativistic (typical examples are supersymmetric particles such as gravitinos and photinos) or if they have never been in thermal equilibrium with the other components (e.g. the axion). Small-scale structure survives in this model, but it is slightly suppressed, relative to the large-scale fluctuations, by the Meszaros effect. Structure formation basically proceeds hierarchically, but the hierarchy develops extremely rapidly.

Detailed calculations have shown that the fluctuations in the cosmic microwave background produced by these models are significantly lower than in the old baryonic models. This is essentially because the WIMP particles do not couple to the radiation field directly via scattering, so it is possible for there to be fluctuations in the WIMP density that are not accompanied by fluctuations in the temperature of the radiation. The HDM model fell out of favour in the early 1980s because it proved difficult to form objects early enough. The presence of quasars at redshifts greater than 4 requires superclusters to have already been formed by that epoch if small-scale structures are to form by fragmentation. The CDM model then emerged as the front-runner for most of the 1980s.

So far in this discussion we have concentrated only on certain aspects of the Jeans instability phenomenon, but a complete model that puts this into a cosmological context requires a number of different ingredients to of the underlying Friedmann model (the density parameter, Hubble parameter and cosmological constant) need to be fixed. Secondly, the relative amounts of baryons and WIMPs need to be decided. And thirdly, the form and statistical properties of the primordial density fluctuations that start the whole instability process off need to be specified.

The standard CDM model that emerged in the mid-1980s served a very useful purpose because it established that the underlying cosmology was a flat Friedmann model with a Hubble constant of 50 kilometres per second per megaparsec, and no cosmological constant. The density of CDM particles was assumed to dominate all other species (so that Omega = 1, due entirely to WIMPs). Finally, in accord with developments in the theory of the inflationary Universe that were happening at the same time, the initial fluctuations were assumed to be adiabatic fluctuations with the characteristic scale-free power spectrum predicted by most models of inflation.

Unfortunately, subsequent measurements of large-scale structure in the galaxy distribution from redshift surveys and, perhaps most importantly, the ripples seen by the COBE satellite, have effectively ruled out the CDM model. However, the early successes of CDM, and the fact that it fits all the data to within a factor of two or so, suggest that its basic premises may be correct. Various possible variations on the CDM theme have been suggested that might reconcile the basic picture with observations. One idea, which produces a hybrid scenario, is to have a mixture of hot and cold particles (this is often called the CHDM model). The most popular version of this model has a density parameter in CDM particles of about 0.7, and an HDM particle density of around 0.3. This model is reasonably successful in accounting for large-scale structure, but it still has problems on the scales of individual galaxies. It also requires an awkward fine-tuning to produce two particles of very different masses with a similar cosmological density. Unless a definite physical model can be advanced to account for this coincidence, the CHDM model must be regarded as rather unlikely.

It has also generally been assumed for most of the 1980s and 1990s that the Universe has to be very nearly flat, as suggested by the inflationary Universe models. This appeared to be a good idea before the COBE discovery of the ripples because the larger the total density of the Universe, the faster the fluctuations would grow. This means that a given level of structure now produces lower-temperature fluctuations on the microwave sky in a high-density Universe than in a low-density one. A low-density CDM model with a density parameter of around 0.3 to 0.4 actually matches all available data fairly well, and this may also be consistent with the measured amounts of cosmological dark matter. If we really want a low density and a flat universe, we can also add a cosmological constant. An alternative is to tinker with the initial fluctuation spectrum so that it is no longer scale-free, but tilted (i.e. the index of the power spectrum is no longer 1).

Whether one of these variants eventually emerges as a clear winner remains to be seen. All the variations on the CDM theme actually produce large-scale structure that looks qualitatively similar: detailed testing of the various models involves running complicated N-body simulations on supercomputers and comparing the results with statistical analyses of large-scale galaxy surveys (see the Figure). One of the problems with this approach is that, while the large-scale structure is relatively easy to predict, the same is not true for individual galaxies. On large scales the gravitational behaviour of the WIMPs completely dominates, and this is quite easy to compute, but on small scales the baryonic material comes into its own, introducing hydrodynamical, radiative and dissipative effects into the picture. In particular,

Structure formation. Results of an N-body computation of the clustering of material expected in a Universe dominated by cold dark matter. The characteristic pattern of sheets, filaments and knots is in qualitative agreement with observations of large-scale structure.