astro-ph/00004075
EFI-2000-13
Science is rarely tidy. We ultimately seek a unified explanatory framework characterized by elegance and simplicity; along the way, however, our aesthetic impulses must occasionally be sacrificed to the desire to encompass the largest possible range of phenomena (i.e., to fit the data). It is often the case that an otherwise compelling theory, in order to be brought into agreement with observation, requires some apparently unnatural modification. Some such modifications may eventually be discarded as unnecessary once the phenomena are better understood; at other times, advances in our theoretical understanding will reveal that a certain theoretical compromise is only superficially distasteful, when in fact it arises as the consequence of a beautiful underlying structure.
General relativity is a paradigmatic example of a scientific theory of impressive power and simplicity. The cosmological constant, meanwhile, is a paradigmatic example of a modification, originally introduced [] to help fit the data, which appears at least on the surface to be superfluous and unattractive. Its original role, to allow static homogeneous solutions to Einstein's equations in the presence of matter, turned out to be unnecessary when the expansion of the universe was discovered [], and there have been a number of subsequent episodes in which a nonzero cosmological constant was put forward as an explanation for a set of observations and later withdrawn when the observational case evaporated. Meanwhile, particle theorists have realized that the cosmological constant can be interpreted as a measure of the energy density of the vacuum. This energy density is the sum of a number of apparently unrelated contributions, each of magnitude much larger than the upper limits on the cosmological constant today; the question of why the observed vacuum energy is so small in comparison to the scales of particle physics has become a celebrated puzzle, although it is usually thought to be easier to imagine an unknown mechanism which would set it precisely to zero than one which would suppress it by just the right amount to yield an observationally accessible cosmological constant.
This checkered history has led to a certain reluctance to consider further invocations of a nonzero cosmological constant; however, recent years have provided the best evidence yet that this elusive quantity does play an important dynamical role in the universe. This possibility, although still far from a certainty, makes it worthwhile to review the physics and astrophysics of the cosmological constant (and its modern equivalent, the energy of the vacuum).
There are a number of other reviews of various aspects of the cosmological constant; in the present article I will outline the most relevant issues, but not try to be completely comprehensive, focusing instead on providing a pedagogical introduction and explaining recent advances. For astrophysical aspects, I did not try to duplicate much of the material in Carroll, Press and Turner [], which should be consulted for numerous useful formulae and a discussion of several kinds of observational tests not covered here. Some earlier discussions include [,,], and subsequent reviews include [,,]. The classic discussion of the physics of the cosmological constant is by Weinberg [], with more recent work discussed by [,]. For introductions to cosmology, see [,,].
Einstein's original field equations are
| (1) |
| (2) |
| (3) |
The energy-momentum sources may be modeled as a perfect fluid,
specified by an energy density r and isotropic pressure p
in its rest frame. The energy-momentum tensor of such a fluid
is
| (4) |
| (5) |
| (6) |
Einstein was interested in finding static ([a\dot] = 0)
solutions, both due to his hope that general relativity would
embody Mach's principle that matter determines inertia,
and simply to account for the astronomical data as they
were understood at the time.1 A static universe
with a positive energy density is compatible with
(5) if the spatial curvature is positive (k = +1)
and the density is appropriately tuned; however,
(6) implies that [a\ddot] will never vanish in
such a spacetime if the pressure p is also nonnegative
(which is true for most forms of matter, and certainly
for ordinary sources such as stars and gas). Einstein
therefore proposed a modification of his equations, to
| (7) |
| (8) |
| (9) |
The discovery by Hubble that the universe is expanding eliminated the empirical need for a static world model (although the Einstein static universe continues to thrive in the toolboxes of theorists, as a crucial step in the construction of conformal diagrams). It has also been criticized on the grounds that any small deviation from a perfect balance between the terms in (9) will rapidly grow into a runaway departure from the static solution.
Pandora's box, however, is not so easily closed. The disappearance of the original motivation for introducing the cosmological constant did not change its status as a legitimate addition to the gravitational field equations, or as a parameter to be constrained by observation. The only way to purge L from cosmological discourse would be to measure all of the other terms in (8) to sufficient precision to be able to conclude that the L/3 term is negligibly small in comparison, a feat which has to date been out of reach. As discussed below, there is better reason than ever before to believe that L is actually nonzero, and Einstein may not have blundered after all.
The cosmological constant L is a dimensionful parameter with units of (length)-2. From the point of view of classical general relativity, there is no preferred choice for what the length scale defined by L might be. Particle physics, however, brings a different perspective to the question. The cosmological constant turns out to be a measure of the energy density of the vacuum - the state of lowest energy - and although we cannot calculate the vacuum energy with any confidence, this identification allows us to consider the scales of various contributions to the cosmological constant [,].
Consider a single scalar field f, with potential
energy V(f). The action can be written
| (10) |
| (11) |
| (12) |
| (13) |
| (14) |
It is not necessary to introduce scalar fields
to obtain a nonzero vacuum energy. The action for general
relativity in the presence of a ``bare'' cosmological
constant L0 is
| (15) |
Classically, then, the effective cosmological constant is the sum of a bare term L0 and the potential energy V(f), where the latter may change with time as the universe passes through different phases. Quantum mechanics adds another contribution, from the zero-point energies associated with vacuum fluctuations. Consider a simple harmonic oscillator, i.e. a particle moving in a one-dimensional potential of the form V(x) = 1/2w2 x2. Classically, the ``vacuum'' for this system is the state in which the particle is motionless and at the minimum of the potential (x = 0), for which the energy in this case vanishes. Quantum-mechanically, however, the uncertainty principle forbids us from isolating the particle both in position and momentum, and we find that the lowest energy state has an energy E0 = 1/2 (h/2p)w (where I have temporarily re-introduced explicit factors of (h/2p) for clarity). Of course, in the absence of gravity either system actually has a vacuum energy which is completely arbitrary; we could add any constant to the potential (including, for example, -1/2 (h/2p)w) without changing the theory. It is important, however, that the zero-point energy depends on the system, in this case on the frequency w.
A precisely analogous situation holds in field theory.
A (free) quantum field can be thought of as a collection of an
infinite number of harmonic oscillators in momentum space.
Formally, the zero-point energy of such an infinite collection
will be infinite. (See [,] for further
details.) If, however, we discard the very high-momentum
modes on the grounds that we trust our theory only up to
a certain ultraviolet momentum cutoff kmax, we
find that the resulting energy density is of the form
| (16) |
The net cosmological constant, from this point of view, is the
sum of a number of apparently disparate contributions,
including potential energies from scalar fields and zero-point
fluctuations of each field theory degree of freedom, as well
as a bare cosmological constant L0. Unlike the last
of these, in the first two cases we can at least make
educated guesses at the magnitudes. In the Weinberg-Salam
electroweak model, the phases of broken and unbroken
symmetry are distinguished by a potential energy
difference of approximately MEW ~ 200 GeV
(where 1 GeV = 1.6×10-3 erg); the
universe is in the broken-symmetry phase during our
current low-temperature epoch, and is believed to have
been in the symmetric phase at sufficiently high temperatures
in the early universe. The effective cosmological constant
is therefore different in the two epochs; absent some
form of prearrangement, we would naturally expect a
contribution to the vacuum energy today of order
| (17) |
| (18) |
| (19) |
As we will discuss later, cosmological observations imply
| (20) |
From the Friedmann equation
(5) (where henceforth we take the effects of a
cosmological constant into account by including the vacuum energy
density rL into the total density r), for any
value of the Hubble parameter H there is a critical value
of the energy density such that the spatial geometry is flat
(k = 0):
| (21) |
| (22) |
| (23) |
In general, the energy density r will include
contributions from various distinct components.
From the point of view of cosmology, the relevant feature of
each component is how its energy density evolves as the universe
expands. Fortunately, it is often (although not always) the
case that individual components i have very simple equations
of state of the form
| (24) |
| (25) |
| (26) |
| (27) |
| (28) |
The simplest example of a component of this form is a set
of massive particles with negligible relative velocities,
known in cosmology as ``dust'' or simply ``matter''.
The energy density of such particles is given by their
number density times their rest mass; as the universe
expands, the number density is inversely proportional to the
volume while the rest masses are constant, yielding
rM µ a-3. For relativistic particles,
known in cosmology as ``radiation'' (although any relativistic
species counts, not only photons or even strictly massless
particles), the energy density is the number density times
the particle energy, and the latter is proportional to a-1
(redshifting as the universe expands); the radiation energy
density therefore scales as rR µ a-4.
Vacuum energy does not change as the universe expands, so
rL µ a0; from (26) this implies a
negative pressure, or positive tension, when the vacuum energy is
positive. Finally, for some purposes it is
useful to pretend that the -ka-2R0-2 term in (5)
represents an effective ``energy density in curvature'',
and define rk º -(3k/8pGR02)a-2. We can define a
corresponding density parameter
| (29) |
| (30) |
| (31) |
The ranges of values of the Wi's which are allowed in principle (as opposed to constrained by observation) will depend on a complete theory of the matter fields, but lacking that we may still invoke energy conditions to get a handle on what constitutes sensible values. The most appropriate condition is the dominant energy condition (DEC), which states that Tmnlm ln ³ 0, and Tmn lm is non-spacelike, for any null vector lm; this implies that energy does not flow faster than the speed of light []. For a perfect-fluid energy-momentum tensor of the form (4), these two requirements imply that r+ p ³ 0 and |r| ³ |p|, respectively. Thus, either the density is positive and greater in magnitude than the pressure, or the density is negative and equal in magnitude to a compensating positive pressure; in terms of the equation-of-state parameter w, we have either positive r and |w| £ 1 or negative r and w = -1. That is, a negative energy density is allowed only if it is in the form of vacuum energy. (We have actually modified the conventional DEC somewhat, by using only null vectors lm rather than null or timelike vectors; the traditional condition would rule out a negative cosmological constant, which there is no physical reason to do.)
There are good reasons to believe that the energy density in radiation today is much less than that in matter. Photons, which are readily detectable, contribute Wg ~ 5×10-5, mostly in the 2.73 °K cosmic microwave background [,,]. If neutrinos are sufficiently low mass as to be relativistic today, conventional scenarios predict that they contribute approximately the same amount []. In the absence of sources which are even more exotic, it is therefore useful to parameterize the universe today by the values of WM and WL, with Wk = 1 - WM - WL, keeping the possibility of surprises always in mind.
One way to characterize a specific
Friedmann-Robertson-Walker model is by the values of
the Hubble parameter and the various energy densities
ri. (Of course, reconstructing the history of such a
universe also requires an understanding of the microphysical
processes which can exchange energy between the different
states.) It may be difficult, however, to directly measure the
different contributions to r, and it is therefore useful
to consider extracting these quantities from the behavior of
the scale factor as a function of time. A traditional measure
of the evolution of the expansion rate is the deceleration
parameter
| (32) |
Notice that positive-energy-density sources with n > 2 cause the universe to decelerate while n < 2 leads to acceleration; the more rapidly energy density redshifts away, the greater the tendency towards universal deceleration. An empty universe (W = 0, Wk = 1) expands linearly with time; sometimes called the ``Milne universe'', such a spacetime is really flat Minkowski space in an unusual time-slicing.
In the remainder of this section we will explore the behavior
of universes dominated by matter and vacuum energy, W = WM + WL = 1-Wk. According to
(32), a positive cosmological constant accelerates the
universal expansion, while a negative cosmological constant
and/or ordinary matter tend to decelerate it. The relative
contributions of these components change with time; according
to (28) we have
| (33) |
Given WM, the value of WL for which
the universe will expand forever is given by
| (34) |
| (35) |
The dynamics of universes with W = WM +WL are summarized in Figure (), in which the arrows indicate the evolution of these parameters in an expanding universe. (In a contracting universe they would be reversed.)
Figure |
This is not a true phase-space plot, despite the superficial similarities. One important difference is that a universe passing through one point can pass through the same point again but moving backwards along its trajectory, by first going to infinity and then turning around (recollapse).
Figure (1) includes three fixed points, at (WM, WL) equal to (0, 0), (0, 1), and (1, 0). The attractor among these at (1, 0) is known as de Sitter space - a universe with no matter density, dominated by a cosmological constant, and with scale factor growing exponentially with time. The fact that this point is an attractor on the diagram is another way of understanding the cosmological constant problem. A universe with initial conditions located at a generic point on the diagram will, after several expansion times, flow to de Sitter space if it began above the recollapse line, and flow to infinity and back to recollapse if it began below that line. Since our universe has undergone a large number of e-folds of expansion since early times, it must have begun at a non-generic point in order not to have evolved either to de Sitter space or to a Big Crunch. The only other two fixed points on the diagram are the saddle point at (WM, WL) = (0, 0), corresponding to an empty universe, and the repulsive fixed point at (WM, WL) = (1, 0), known as the Einstein-de Sitter solution. Since our universe is not empty, the favored solution from this combination of theoretical and empirical arguments is the Einstein-de Sitter universe. The inflationary scenario [,,] provides a mechanism whereby the universe can be driven to the line WM+WL = 1 (spatial flatness), so Einstein-de Sitter is a natural expectation if we imagine that some unknown mechanism sets L = 0. As discussed below, the observationally favored universe is located on this line but away from the fixed points, near (WM, WL) = (0.3, 0.7). It is fair to conclude that naturalness arguments have a somewhat spotty track record at predicting cosmological parameters.
The lookback time from the present day to an object at
redshift z* is given by
| (36) |
Figure |
There are analytic approximation formulas which estimate (36) in various regimes [,,], but generally the integral is straightforward to perform numerically.
In a generic curved spacetime, there is no preferred notion of
the distance between two objects. Robertson-Walker spacetimes
have preferred foliations, so it is possible to define sensible
notions of the distance between comoving objects - those whose
worldlines are normal to the preferred slices. Placing ourselves
at r = 0 in the coordinates defined by (2), the
coordinate distance r to another comoving object is independent
of time. It can be converted to a physical distance at any
specified time t* by multiplying by the scale factor R0a(t*),
yielding a number which will of course change as the universe
expands. However, intervals along spacelike slices are not
accessible to observation, so it is typically more convenient to
use distance measures which can be extracted from observable
quantities. These include the luminosity distance,
| (37) |
| (38) |
| (39) |
| (40) |
| (41) |
The proper-motion distance between sources at redshift z1
and z2 can
be computed by using ds2 = 0 along a light ray, where ds2
is given by (2). We have
| (42) |
The comoving volume element in a Robertson-Walker universe
is given by
| (43) |
| (44) |
The introduction of a cosmological constant changes the relationship between the matter density and expansion rate from what it would be in a matter-dominated universe, which in turn influences the growth of large-scale structure. The effect is similar to that of a nonzero spatial curvature, and complicated by hydrodynamic and nonlinear effects on small scales, but is potentially detectable through sufficiently careful observations.
The analysis of the evolution of structure is greatly abetted by the fact that perturbations start out very small (temperature anisotropies in the microwave background imply that the density perturbations were of order 10-5 at recombination), and linearized theory is effective. In this regime, the fate of the fluctuations is in the hands of two competing effects: the tendency of self-gravity to make overdense regions collapse, and the tendency of test particles in the background expansion to move apart. Essentially, the effect of vacuum energy is to contribute to expansion but not to the self-gravity of overdensities, thereby acting to suppress the growth of perturbations [,].
For sub-Hubble-radius perturbations in a cold dark matter
component, a Newtonian analysis suffices. (We may of course
be interested in super-Hubble-radius modes, or the evolution
of interacting or relativistic particles, but the simple
Newtonian case serves to illustrate the relevant physical
effect.) If the energy density in dynamical matter
is dominated by CDM, the linearized Newtonian
evolution equation is
| (45) |
| (46) |
It has been suspected for some time now that there are good reasons to think that a cosmology with an appreciable cosmological constant is the best fit to what we know about the universe [,,,,,,,,]. However, it is only very recently that the observational case has tightened up considerably, to the extent that, as the year 2000 dawns, more experts than not believe that there really is a positive vacuum energy exerting a measurable effect on the evolution of the universe. In this section I review the major approaches which have led to this shift.
The most direct and theory-independent way to measure the cosmological constant would be to actually determine the value of the scale factor as a function of time. Unfortunately, the appearance of Wk in formulae such as (42) renders this difficult. Nevertheless, with sufficiently precise information about the dependence of a distance measure on redshift we can disentangle the effects of spatial curvature, matter, and vacuum energy, and methods along these lines have been popular ways to try to constrain the cosmological constant.
Astronomers measure distance in terms of the ``distance modulus''
m-M, where m is the apparent magnitude of the source and M
its absolute magnitude. The distance modulus is related to the
luminosity distance via
| (47) |
Recently, significant progress has been made by using Type Ia supernovae as ``standardizable candles''. Supernovae are rare - perhaps a few per century in a Milky-Way-sized galaxy - but modern telescopes allow observers to probe very deeply into small regions of the sky, covering a very large number of galaxies in a single observing run. Supernovae are also bright, and Type Ia's in particular all seem to be of nearly uniform intrinsic luminosity (absolute magnitude M ~ -19.5, typically comparable to the brightness of the entire host galaxy in which they appear) []. They can therefore be detected at high redshifts (z ~ 1), allowing in principle a good handle on cosmological effects [,].
Figure |
Figure |
Figure |
Figure |
The fact that all SNe Ia are of similar intrinsic luminosities fits well with our understanding of these events as explosions which occur when a white dwarf, onto which mass is gradually accreting from a companion star, crosses the Chandrasekhar limit and explodes. (It should be noted that our understanding of supernova explosions is in a state of development, and theoretical models are not yet able to accurately reproduce all of the important features of the observed events. See [,,] for some recent work.) The Chandrasekhar limit is a nearly-universal quantity, so it is not a surprise that the resulting explosions are of nearly-constant luminosity. However, there is still a scatter of approximately 40% in the peak brightness observed in nearby supernovae, which can presumably be traced to differences in the composition of the white dwarf atmospheres. Even if we could collect enough data that statistical errors could be reduced to a minimum, the existence of such an uncertainty would cast doubt on any attempts to study cosmology using SNe Ia as standard candles.
Fortunately, the observed differences in peak luminosities of SNe Ia are very closely correlated with observed differences in the shapes of their light curves: dimmer SNe decline more rapidly after maximum brightness, while brighter SNe decline more slowly [,,]. There is thus a one-parameter family of events, and measuring the behavior of the light curve along with the apparent luminosity allows us to largely correct for the intrinsic differences in brightness, reducing the scatter from 40% to less than 15% - sufficient precision to distinguish between cosmological models. (It seems likely that the single parameter can be traced to the amount of 56Ni produced in the supernova explosion; more nickel implies both a higher peak luminosity and a higher temperature and thus opacity, leading to a slower decline. It would be an exaggeration, however, to claim that this behavior is well-understood theoretically.)
Following pioneering work reported in [], two independent groups have undertaken searches for distant supernovae in order to measure cosmological parameters. Figure (3) shows the results for m-M vs. z for the High-Z Supernova Team [,,,], and Figure (4) shows the equivalent results for the Supernova Cosmology Project [,,]. Under the assumption that the energy density of the universe is dominated by matter and vacuum components, these data can be converted into limits on WM and WL, as shown in Figures (5) and (6).
It is clear that the confidence intervals in the WM-WL plane are consistent for the two groups, with somewhat tighter constraints obtained by the Supernova Cosmology Project, who have more data points. The surprising result is that both teams favor a positive cosmological constant, and strongly rule out the traditional (WM, WL) = (1,0) favorite universe. They are even inconsistent with an open universe with zero cosmological constant, given what we know about the matter density of the universe (see below).
Given the significance of these results, it is natural to ask what level of confidence we should have in them. There are a number of potential sources of systematic error which have been considered by the two teams; see the original papers [,,] for a thorough discussion. The two most worrisome possibilities are intrinsic differences between Type Ia supernovae at high and low redshifts [,], and possible extinction via intergalactic dust [,,,,]. (There is also the fact that intervening weak lensing can change the distance-magnitude relation, but this seems to be a small effect in realistic universes [,].) Both effects have been carefully considered, and are thought to be unimportant, although a better understanding will be necessary to draw firm conclusions. Here, I will briefly mention some of the relevant issues.
As thermonuclear explosions of white dwarfs, Type Ia supernovae can occur in a wide variety of environments. Consequently, a simple argument against evolution is that the high-redshift environments, while chronologically younger, should be a subset of all possible low-redshift environments, which include regions that are ``young'' in terms of chemical and stellar evolution. Nevertheless, even a small amount of evolution could ruin our ability to reliably constrain cosmological parameters []. In their original papers [,,], the supernova teams found impressive consistency in the spectral and photometric properties of Type Ia supernovae over a variety of redshifts and environments (e.g., in elliptical vs. spiral galaxies). More recently, however, Riess et al. [] have presented tentative evidence for a systematic difference in the properties of high- and low-redshift supernovae, claiming that the risetimes (from initial explosion to maximum brightness) were higher in the high-redshift events. Apart from the issue of whether the existing data support this finding, it is not immediately clear whether such a difference is relevant to the distance determinations: first, because the risetime is not used in determining the absolute luminosity at peak brightness, and second, because a process which only affects the very early stages of the light curve is most plausibly traced to differences in the outer layers of the progenitor, which may have a negligible affect on the total energy output. Nevertheless, any indication of evolution could bring into question the fundamental assumptions behind the entire program. It is therefore essential to improve the quality of both the data and the theories so that these issues may be decisively settled.
Other than evolution, obscuration by dust is the leading concern about the reliability of the supernova results. Ordinary astrophysical dust does not obscure equally at all wavelengths, but scatters blue light preferentially, leading to the well-known phenomenon of ``reddening''. Spectral measurements by the two supernova teams reveal a negligible amount of reddening, implying that any hypothetical dust must be a novel ``grey'' variety. This possibility has been investigated by a number of authors [,,,,]. These studies have found that even grey dust is highly constrained by observations: first, it is likely to be intergalactic rather than within galaxies, or it would lead to additional dispersion in the magnitudes of the supernovae; and second, intergalactic dust would absorb ultraviolet/optical radiation and re-emit it at far infrared wavelengths, leading to stringent constraints from observations of the cosmological far-infrared background. Thus, while the possibility of obscuration has not been entirely eliminated, it requires a novel kind of dust which is already highly constrained (and may be convincingly ruled out by further observations).
According to the best of our current understanding, then, the supernova results indicating an accelerating universe seem likely to be trustworthy. Needless to say, however, the possibility of a heretofore neglected systematic effect looms menacingly over these studies. Future experiments, including a proposed satellite dedicated to supernova cosmology [], will both help us improve our understanding of the physics of supernovae and allow a determination of the distance/redshift relation to sufficient precision to distinguish between the effects of a cosmological constant and those of more mundane astrophysical phenomena. In the meantime, it is important to obtain independent corroboration using other methods.
The discovery by the COBE satellite of temperature anisotropies
in the cosmic microwave background [] inaugurated a new
era in the determination of cosmological parameters.
To characterize the temperature fluctuations on the sky, we
may decompose them into spherical harmonics,
| (48) |
| (49) |
Figure |
Although the dependence of the Cl's on the parameters can be
intricate, nature has chosen not to test the patience of
cosmologists, as one of the easiest features to measure - the
location in l of the first ``Doppler peak'', an increase in
power due to acoustic oscillations - provides one of the most
direct handles on the cosmic energy density, one of the most
interesting parameters. The first
peak (the one at lowest l) corresponds to the angular scale
subtended by the Hubble radius HCMB-1 at the time when the
CMB was formed (known variously as ``decoupling'' or ``recombination''
or ``last scattering'') [].
The angular scale at which we observe this peak is tied to the
geometry of the universe: in a negatively (positively)
curved universe, photon paths diverge (converge), leading to
a larger (smaller) apparent angular size as compared to a
flat universe. Since the scale HCMB-1 is set
mostly by microphysics, this geometrical effect is dominant,
and we can relate the spatial curvature as characterized
by W to the observed peak in the CMB spectrum via
[,,]
| (50) |
Figure |
Figure 7 shows a summary of data as of 1998, with various experimental results consolidated into bins, along with two theoretical models. Since that time, the data have continued to accumulate (see for example [,]), and the near future should see a wealth of new results of ever-increasing precision. It is clear from the figure that there is good evidence for a peak at approximately lpeak ~ 200, as predicted in a spatially-flat universe. This result can be made more quantitative by fitting the CMB data to models with different values of WM and WL [,,,,], or by combining the CMB data with other sources, such as supernovae or large-scale structure [,,,,,,,]. Figure 8 shows the constraints from the CMB in the WM-WL plane, using data from the 1997 test flight of the BOOMERANG experiment []. (Although the data used to make this plot are essentially independent of those shown in the previous figure, the constraints obtained are nearly the same.) It is clear that the CMB data provide constraints which are complementary to those obtained using supernovae; the two approaches yield confidence contours which are nearly orthogonal in the WM-WL plane. The region of overlap is in the vicinity of (WM, WL) = (0.3, 0.7), which we will see below is also consistent with other determinations.
Many cosmological tests, such as the two just discussed, will constrain some combination of WM and WL. It is therefore useful to consider tests of WM alone, even if our primary goal is to determine WL. (In truth, it is also hard to constrain WM alone, as almost all methods actually constrain some combination of WM and the Hubble constant h = H0/(100 km/sec/Mpc); the HST Key Project on the extragalactic distance scale finds h = 0.71 ±0.06 [], which is consistent with other methods [], and what I will assume below.)
For years, determinations of WM based on dynamics of galaxies and clusters have yielded values between approximately 0.1 and 0.4 - noticeably larger than the density parameter in baryons as inferred from primordial nucleosynthesis, WB = (0.019 ±0.001)h-2 » 0.04 [,], but noticeably smaller than the critical density. The last several years have witnessed a number of new methods being brought to bear on the question; the quantitative results have remained unchanged, but our confidence in them has increased greatly.
A thorough discussion of determinations of WM requires a review all its own, and good ones are available [,,,,]. Here I will just sketch some of the important methods.
The traditional method to estimate the mass density of the universe is to ``weigh'' a cluster of galaxies, divide by its luminosity, and extrapolate the result to the universe as a whole. Although clusters are not representative samples of the universe, they are sufficiently large that such a procedure has a chance of working. Studies applying the virial theorem to cluster dynamics have typically obtained values WM = 0.2 ±0.1 [,,]. Although it is possible that the global value of M/L differs appreciably from its value in clusters, extrapolations from small scales do not seem to reach the critical density []. New techniques to weigh the clusters, including gravitational lensing of background galaxies [] and temperature profiles of the X-ray gas [], while not yet in perfect agreement with each other, reach essentially similar conclusions.
Rather than measuring the mass relative to the luminosity
density, which may be different inside and outside clusters,
we can also measure it with respect to the baryon density
[],
which is very likely to have the same value in clusters as
elsewhere in the universe, simply because there is no way
to segregate the baryons from the dark matter on such large
scales. Most of the baryonic mass is in the hot intracluster
gas [], and the fraction fgas
of total mass in this form can be measured either by
direct observation of X-rays from the gas
[] or by distortions
of the microwave background by scattering off hot electrons
(the Sunyaev-Zeldovich effect) [], typically
yielding 0.1 £ fgas £ 0.2.
Since primordial nucleosynthesis provides a determination
of WB ~ 0.04, these measurements imply
| (51) |
Another handle on the density parameter in matter comes from properties of clusters at high redshift. The very existence of massive clusters has been used to argue in favor of WM ~ 0.2 [], and the lack of appreciable evolution of clusters from high redshifts to the present [,] provides additional evidence that WM < 1.0.
The story of large-scale motions is more ambiguous. The peculiar velocities of galaxies are sensitive to the underlying mass density, and thus to WM, but also to the ``bias'' describing the relative amplitude of fluctuations in galaxies and mass [,]. Difficulties both in measuring the flows and in disentangling the mass density from other effects make it difficult to draw conclusions at this point, and at present it is hard to say much more than 0.2 £ WM £ 1.0.
Finally, the matter density parameter can be extracted from measurements of the power spectrum of density fluctuations (see for example []). As with the CMB, predicting the power spectrum requires both an assumption of the correct theory and a specification of a number of cosmological parameters. In simple models (e.g., with only cold dark matter and baryons, no massive neutrinos), the spectrum can be fit (once the amplitude is normalized) by a single ``shape parameter'', which is found to be equal to G = WMh. (For more complicated models see [].) Observations then yield G ~ 0.25, or WM ~ 0.36. For a more careful comparison between models and observations, see [,,,].
Thus, we have a remarkable convergence on values for the
density parameter in matter:
| (52) |
The volume of space back to a specified redshift, given by (44), depends sensitively on WL. Consequently, counting the apparent density of observed objects, whose actual density per cubic Mpc is assumed to be known, provides a potential test for the cosmological constant [,,,]. Like tests of distance vs. redshift, a significant problem for such methods is the luminosity evolution of whatever objects one might attempt to count. A modern attempt to circumvent this difficulty is to use the statistics of gravitational lensing of distant galaxies; the hope is that the number of condensed objects which can act as lenses is less sensitive to evolution than the number of visible objects.
In a spatially flat universe,
the probability of a source at redshift zs being lensed, relative
to the fiducial (WM = 1, WL = 0) case,
is given by
| (53) |
Figure |
As shown in Figure (9), the probability rises dramatically as WL is increased to unity as we keep W fixed. Thus, the absence of a large number of such lenses would imply an upper limit on WL.
Analysis of lensing statistics is complicated by uncertainties in evolution, extinction, and biases in the lens discovery procedure. It has been argued [,] that the existing data allow us to place an upper limit of WL < 0.7 in a flat universe. However, other groups [,] have claimed that the current data actually favor a nonzero cosmological constant. The near future will bring larger, more objective surveys, which should allow these ambiguities to be resolved. Other manifestations of lensing can also be used to constrain WL, including statistics of giant arcs [], deep weak-lensing surveys [], and lensing in the Hubble Deep Field [].
There is a tremendous variety of ways in which a nonzero cosmological constant can manifest itself in observable phenomena. Here is an incomplete list of additional possibilities; see also [,,].
In Section (1.3) we discussed the large
difference between the magnitude of the
vacuum energy expected from zero-point
fluctuations and scalar potentials, rLtheor ~ 2×10110 erg/cm3,
and the value we apparently observe,
rL(obs) ~ 2×10-10 erg/cm3
(which may be thought of as an upper limit, if we wish
to be careful). It is somewhat unfair to characterize this
discrepancy as a factor of 10120, since energy density
can be expressed as a mass scale to the fourth power.
Writing rL = Mvac4,
we find Mvac(theory) ~ MPl ~ 1018 GeV and
Mvac(obs) ~ 10-3 eV, so a more fair
characterization of the problem would be
| (54) |
Although the mechanism which suppresses the naive value of the vacuum energy is unknown, it seems easier to imagine a hypothetical scenario which makes it exactly zero than one which sets it to just the right value to be observable today. (Keeping in mind that it is the zero-temperature, late-time vacuum energy which we want to be small; it is expected to change at phase transitions, and a large value in the early universe is a necessary component of inflationary universe scenarios [,,].) If the recent observations pointing toward a cosmological constant of astrophysically relevant magnitude are confirmed, we will be faced with the challenge of explaining not only why the vacuum energy is smaller than expected, but also why it has the specific nonzero value it does.
Although initially investigated for other reasons, supersymmetry (SUSY) turns out to have a significant impact on the cosmological constant problem, and may even be said to solve it halfway. SUSY is a spacetime symmetry relating fermions and bosons to each other. Just as ordinary symmetries are associated with conserved charges, supersymmetry is associated with ``supercharges'' Qa, where a is a spinor index (for introductions see [,,]). As with ordinary symmetries, a theory may be supersymmetric even though a given state is not supersymmetric; a state which is annihilated by the supercharges, Qa |yñ = 0, preserves supersymmetry, while states with Qa |yñ ¹ 0 are said to spontaneously break SUSY.
Let's begin by considering ``globally supersymmetric'' theories,
which are defined in flat spacetime (obviously an
inadequate setting in which to discuss the cosmological
constant, but we have to start somewhere). Unlike most
non-gravitational field theories, in supersymmetry
the total energy of a state has an absolute meaning; the
Hamiltonian is related to the supercharges in a straightforward
way:
| (55) |
| (56) |
So the vacuum energy of a supersymmetric state in a globally
supersymmetric theory will vanish. This represents rather less
progress than it might appear at first sight, since: 1.) Supersymmetric
states manifest a degeneracy in the mass spectrum of bosons and
fermions, a feature not apparent in the observed world; and 2.)
The above results imply that non-supersymmetric states have a
positive-definite vacuum energy. Indeed, in a state where
SUSY was broken at an energy scale MSUSY, we would
expect a corresponding vacuum energy rL ~ MSUSY4. In the real world, the fact that
accelerator experiments have not
discovered superpartners for the known particles of the Standard
Model implies that
MSUSY is of order 103 GeV or higher. Thus, we
are left with a discrepancy
| (57) |
As mentioned, however, this analysis is strictly valid only
in flat space. In curved spacetime, the global transformations
of ordinary supersymmetry are promoted to the position-dependent
(gauge) transformations of supergravity. In this context the
Hamiltonian and supersymmetry generators play different
roles than in flat spacetime, but it is still possible to
express the vacuum energy in terms of a scalar field potential
V(fi, [`(f)]j). In supergravity V depends
not only on the superpotential W(fi), but also on a
``Kähler potential'' K(fi, [`(f)]j), and the
Kähler metric Ki[`j] constructed from the
Kähler potential by Ki[`j] = ¶2 K/¶fi ¶[`(f)]j. (The basic role
of the Kähler metric is to define the kinetic term
for the scalars, which takes the form gmnKi[`j]¶mfi ¶n[`(f)]j.)
The scalar potential is
| (58) |
| (59) |
Unlike supergravity, string theory appears to be a consistent and well-defined theory of quantum gravity, and therefore calculating the value of the cosmological constant should, at least in principle, be possible. On the other hand, the number of vacuum states seems to be quite large, and none of them (to the best of our current knowledge) features three large spatial dimensions, broken supersymmetry, and a small cosmological constant. At the same time, there are reasons to believe that any realistic vacuum of string theory must be strongly coupled []; therefore, our inability to find an appropriate solution may simply be due to the technical difficulty of the problem. (For general introductions to string theory, see [,]; for cosmological issues, see [,]).
String theory is naturally formulated in more than four spacetime dimensions. Studies of duality symmetries have revealed that what used to be thought of as five distinct ten-dimensional superstring theories - Type I, Types IIA and IIB, and heterotic theories based on gauge groups E(8)×E(8) and SO(32) - are, along with eleven-dimensional supergravity, different low-energy weak-coupling limits of a single underlying theory, sometimes known as M-theory. In each of these six cases, the solution with the maximum number of uncompactified, flat spacetime dimensions is a stable vacuum preserving all of the supersymmetry. To bring the theory closer to the world we observe, the extra dimensions can be compactified on a manifold whose Ricci tensor vanishes. There are a large number of possible compactifications, many of which preserve some but not all of the original supersymmetry. If enough SUSY is preserved, the vacuum energy will remain zero; generically there will be a manifold of such states, known as the moduli space.
Of course, to describe our world we want to break all of the supersymmetry. Investigations in contexts where this can be done in a controlled way have found that the induced cosmological constant vanishes at the classical level, but a substantial vacuum energy is typically induced by quantum corrections []. Moore [] has suggested that Atkin-Lehner symmetry, which relates strong and weak coupling on the string worldsheet, can enforce the vanishing of the one-loop quantum contribution in certain models (see also [,]); generically, however, there would still be an appreciable contribution at two loops.
Thus, the search is still on for a four-dimensional string theory vacuum with broken supersymmetry and vanishing (or very small) cosmological constant. (See [] for a general discussion of the vacuum problem in string theory.) The difficulty of achieving this in conventional models has inspired a number of more speculative proposals, which I briefly list here.
Of course, string theory might not be the correct description of nature, or its current formulation might not be directly relevant to the cosmological constant problem. For example, a solution may be provided by loop quantum gravity [], or by a composite graviton []. It is probably safe to believe that a significant advance in our understanding of fundamental physics will be required before we can demonstrate the existence of a vacuum state with the desired properties. (Not to mention the equally important question of why our world is based on such a state, rather than one of the highly supersymmetric states that appear to be perfectly good vacua of string theory.)
The anthropic principle [,] is essentially the idea that some of the parameters characterizing the universe we observe may not be determined directly by the fundamental laws of physics, but also by the truism that intelligent observers will only ever experience conditions which allow for the existence of intelligent observers. Many professional cosmologists view this principle in much the same way as many traditional literary critics view deconstruction - as somehow simultaneously empty of content and capable of working great evil. Anthropic arguments are easy to misuse, and can be invoked as a way out of doing the hard work of understanding the real reasons behind why we observe the universe we do. Furthermore, a sense of disappointment would inevitably accompany the realization that there were limits to our ability to unambiguously and directly explain the observed universe from first principles. It is nevertheless possible that some features of our world have at best an anthropic explanation, and the value of the cosmological constant is perhaps the most likely candidate.
In order for the tautology that ``observers will only observe conditions which allow for observers'' to have any force, it is necessary for there to be alternative conditions - parts of the universe, either in space, time, or branches of the wavefunction - where things are different. In such a case, our local conditions arise as some combination of the relative abundance of different environments and the likelihood that such environments would give rise to intelligence. Clearly, the current state of the art doesn't allow us to characterize the full set of conditions in the entire universe with any confidence, but modern theories of inflation and quantum cosmology do at least allow for the possibility of widely disparate parts of the universe in which the ``constants of nature'' take on very different values (for recent examples see [,,,,,,]). We are therefore faced with the task of estimating quantitatively the likelihood of observing any specific value of L within such a scenario.
The most straightforward anthropic constraint on the vacuum
energy is that it must not be so high that galaxies never
form []. From the discussion in Section
(2.4), we know that overdense regions do not
collapse once the cosmological constant begins to dominate
the universe; if this happens before the epoch of galaxy formation,
the universe will be devoid of galaxies, and thus of stars and
planets, and thus (presumably) of intelligent life. The
condition that WL(zgal) £ WM(zgal) implies
| (60) |
However, it is better to ask what is most likely value of
WL, i.e. what is the value that would
be experienced by the largest number of observers
[,]? Since a universe with
WL 0/WM0 ~ 1 will have many
more galaxies than one with WL 0/WM0 ~ 100, it is quite conceivable that most observers will
measure something close to the former value. The probability
measure for observing a value of rL can be
decomposed as
| (61) |
Thus, if one is willing to make the leap of faith required to believe that the value of the cosmological constant is chosen from an ensemble of possibilities, it is possible to find an ``explanation'' for its current value (which, given its unnaturalness from a variety of perspectives, seems otherwise hard to understand). Perhaps the most significant weakness of this point of view is the assumption that there are a continuum of possibilities for the vacuum energy density. Such possibilities correspond to choices of vacuum states with arbitrarily similar energies. If these states were connected to each other, there would be local fluctuations which would appear to us as massless fields, which are not observed (see Section ). If on the other hand the vacua are disconnected, it is hard to understand why all possible values of the vacuum energy are represented, rather than the differences in energies between different vacua being given by some characteristic particle-physics scale such as MPl or MSUSY. (For one scenario featuring discrete vacua with densely spaced energies, see [].) It will therefore (again) require advances in our understanding of fundamental physics before an anthropic explanation for the current value of the cosmological constant can be accepted.
The importance of the cosmological constant problem has engendered a wide variety of proposed solutions. This section will present only a brief outline of some of the possibilities, along with references to recent work; further discussion and references can be found in [,,].
One approach which has received a great deal of attention is the famous suggestion by Coleman [], that effects of virtual wormholes could set the cosmological constant to zero at low energies. The essential idea is that wormholes (thin tubes of spacetime connecting macroscopically large regions) can act to change the effective value of all the observed constants of nature. If we calculate the wave function of the universe by performing a Feynman path integral over all possible spacetime metrics with wormholes, the dominant contribution will be from those configurations whose effective values for the physical constants extremize the action. These turn out to be, under a certain set of assumed properties of Euclidean quantum gravity, configurations with zero cosmological constant at late times. Thus, quantum cosmology predicts that the constants we observe are overwhelmingly likely to take on values which imply a vanishing total vacuum energy. However, subsequent investigations have failed to inspire confidence that the desired properties of Euclidean quantum cosmology are likely to hold, although it is still something of an open question; see discussions in [,].
Another route one can take is to consider alterations of the classical theory of gravity. The simplest possibility is to consider adding a scalar field to the theory, with dynamics which cause the scalar to evolve to a value for which the net cosmological constant vanishes (see for example [,]). Weinberg, however, has pointed out on fairly general grounds that such attempts are unlikely to work [,]; in models proposed to date, either there is no solution for which the effective vacuum energy vanishes, or there is a solution but with other undesirable properties (such as making Newton's constant G also vanish). Rather than adding scalar fields, a related approach is to remove degrees of freedom by making the determinant of the metric, which multiplies L0 in the action (15), a non-dynamical quantity, or at least changing its dynamics in some way (see [,,] for recent examples). While this approach has not led to a believable solution to the cosmological constant problem, it does change the context in which it appears, and may induce different values for the effective vacuum energy in different branches of the wavefunction of the universe.
Along with global supersymmetry, there is one other symmetry which would work to prohibit a cosmological constant: conformal (or scale) invariance, under which the metric is multiplied by a spacetime-dependent function, gmn ®el(x) gmn. Like supersymmetry, conformal invariance is not manifest in the Standard Model of particle physics. However, it has been proposed that quantum effects could restore conformal invariance on length scales comparable to the cosmological horizon size, working to cancel the cosmological constant (for some examples see [,,]). At this point it remains unclear whether this suggestion is compatible with a more complete understanding of quantum gravity, or with standard cosmological observations.
A final mechanism to suppress the cosmological constant, related to the previous one, relies on quantum particle production in de Sitter space (analogous to Hawking radiation around black holes). The idea is that the effective energy-momentum tensor of such particles may act to cancel out the bare cosmological constant (for recent attempts see [,,,]). There is currently no consensus on whether such an effect is physically observable (see for example []).
If inventing a theory in which the vacuum energy vanishes is difficult, finding a model that predicts a vacuum energy which is small but not quite zero is all that much harder. Along these lines, there are various numerological games one can play. For example, the fact that supersymmetry solves the problem halfway could be suggestive; a theory in which the effective vacuum energy scale was given not by MSUSY ~ 103 GeV but by MSUSY2/MPl ~ 10-3 eV would seem to fit the observations very well. The challenging part of this program, of course, is to devise such a theory. Alternatively, one could imagine that we live in a ``false vacuum'' - that the absolute minimum of the vacuum energy is truly zero, but we live in a state which is only a local minimum of the energy. Scenarios along these lines have been explored [,,]; the major hurdle to be overcome is explaining why the energy difference between the true and false vacua is so much smaller than one would expect.
Although a cosmological constant is an excellent fit to the current data, the observations can also be accommodated by any form of ``dark energy'' which does not cluster on small scales (so as to avoid being detected by measurements of WM) and redshifts away only very slowly as the universe expands [to account for the accelerated expansion, as per equation (32)]. This possibility has been extensively explored of late, and a number of candidates have been put forward.
One way to parameterize such a component X is by an effective equation of state, pX = wX rX. (A large number of phenomenological models of this type have been investigated, starting with the early work in [,]; see [,] for many more references.) The relevant range for wX is between 0 (ordinary matter) and -1 (true cosmological constant); sources with wX > 0 redshift away more rapidly than ordinary matter (and therefore cause extra deceleration), while wX < -1 is unphysical by the criteria discussed in Section 2.1 (although see []). While not every source will obey an equation of state with wX = constant, it is often the case that a single effective wX characterizes the behavior for the redshift range over which the component can potentially be observed.
Figure |
Current observations of supernovae, large-scale structure, gravitational lensing, and the CMB already provide interesting limits on wX [,,,,,,,,,,,], and future data will be able to do much better [,,,]. Figure (10) shows an example, in this case limits from supernovae and large-scale structure on wX and WM in a universe which is assumed to be flat and dominated by X and ordinary matter. It is clear that the favored value for the equation-of-state parameter is near -1, that of a true cosmological constant, although other values are not completely ruled out.
The simplest physical model for an appropriate dark
energy component is a single slowly-rolling scalar field,
sometimes referred to as ``quintessence''
[,,,,,,,,,,,].
In an expanding universe, a spatially homogeneous scalar with potential
V(f) and minimal coupling to gravity obeys
| (62) |
| (63) |
There are many reasons to consider dynamical dark energy as an alternative to a cosmological constant. First and foremost, it is a logical possibility which might be correct, and can be constrained by observation. Secondly, it is consistent with the hope that the ultimate vacuum energy might actually be zero, and that we simply haven't relaxed all the way to the vacuum as yet. But most interestingly, one might wonder whether replacing a constant parameter L with a dynamical field could allow us to relieve some of the burden of fine-tuning that inevitably accompanies the cosmological constant. To date, investigations have focused on scaling or tracker models of quintessence, in which the scalar field energy density can parallel that of matter or radiation, at least for part of its history [,,,,,,]. (Of course, we do not want the dark energy density to redshift away as rapidly as that in matter during the current epoch, or the universe would not be accelerating.) Tracker models can be constructed in which the vacuum energy density at late times is robust, in the sense that it does not depend sensitively on the initial conditions for the field. However, the ultimate value rvac ~ (10-3 eV)4 still depends sensitively on the parameters in the potential. Indeed, it is hard to imagine how this could help but be the case; unlike the case of the axion solution to the strong-CP problem, we have no symmetry to appeal to that would enforce a small vacuum energy, much less a particular small nonzero number.
Quintessence models also introduce new naturalness problems
in addition to those of a cosmological constant. These can be
traced to the fact that, in order for the field to be slowly-rolling
today, we require Ö{V¢¢(f0)} ~ H0; but this
expression is the effective mass of fluctuations in f, so
we have
| (64) |
Nevertheless, these naturalness arguments are by no means airtight, and it is worth considering specific particle-physics models for the quintessence field. In addition to the pseudo-Goldstone boson models just mentioned, these include models based on supersymmetric gauge theories [,], supergravity [,], small extra dimensions [,], large extra dimensions [,], and non-minimal couplings to the curvature scalar [,,,,,,,,]. Finally, the possibility has been raised that the scalar field responsible for driving inflation may also serve as quintessence [,,,], although this proposal has been criticized for producing unwanted relics and isocurvature fluctuations [].
There are other models of dark energy besides those based on nearly-massless scalar fields. One scenario is ``solid'' dark matter, typically based on networks of tangled cosmic strings or domain walls [,,,]. Strings give an effective equation-of-state parameter wstring = -1/3, and walls have wwall = -2/3, so walls are a better fit to the data at present. There is also the idea of dark matter particles whose masses increase as the universe expands, their energy thus redshifting away more slowly than that of ordinary matter [,] (see also []). The cosmological consequences of this kind of scenario turn out to be difficult to analyze analytically, and work is still ongoing.
Observational evidence from a variety of sources currently points to a universe which is (at least approximately) spatially flat, with (WM, WL) » (0.3, 0.7). The nucleosynthesis constraint implies that WB ~ 0.04, so the majority of the matter content must be in an unknown non-baryonic form.
Figure |
Nobody would have guessed that we live in such a universe. Figure (11) is a plot of WL as a function of the scale factor a for this cosmology. At early times, the cosmological constant would have been negligible, while at later times the density of matter will be essentially zero and the universe will be empty. We happen to live in that brief era, cosmologically speaking, when both matter and vacuum are of comparable magnitude. Within the matter component, there are apparently contributions from baryons and from a non-baryonic source, both of which are also comparable (although at least their ratio is independent of time). This scenario staggers under the burden of its unnaturalness, but nevertheless crosses the finish line well ahead of any competitors by agreeing so well with the data.
Apart from confirming (or disproving) this picture, a major challenge to cosmologists and physicists in the years to come will be to understand whether these apparently distasteful aspects of our universe are simply surprising coincidences, or actually reflect a beautiful underlying structure we do not as yet comprehend. If we are fortunate, what appears unnatural at present will serve as a clue to a deeper understanding of fundamental physics.
I wish to thank Greg Anderson, Tom Banks, Robert Caldwell, Gordon Chalmers, Michael Dine, George Field, Peter Garnavich, Jeff Harvey, Gordy Kane, Manoj Kaplinghat, Bob Kirshner, Lloyd Knox, Finn Larsen, Laura Mersini, Ue-Li Pen, Saul Perlmutter, Joe Polchinski, Ted Pyne, Brian Schmidt, and Michael Turner for numerous useful conversations, Patrick Brady, Deryn Fogg and Clifford Johnson for rhetorical encouragement, and Bill Press and Ed Turner for insinuating me into this formerly-disreputable subject.
1This account gives short shrift to the details of what actually happened; for historical background see [].