Which things about the world are accidental, which things are necessary? Philosophers have debated this metaphysical question for thousands of years (see, e.g., Leslie 1989), but it has become more than an abstract philosophical issue since the answer now influences the mathematical design of fundamental physical theory (see, e.g., Tegmark 1998). Within the confines of physics we can sharpen the question and can even hope to offer some provisional answers.
The question now has special currency because in modern fundamental theories, low-energy effective constants can preserve the symmetry of precise spatial uniformity over a large spatial volume - even a whole ``daughter universe'' - even while they adopt different values in different universes. In addition, inflationary cosmology offers a physical mechanism for creating a true statistical ensemble (a ``multiverse''; Rees 1997) where many possible values of the constants are realized. The truly fundamental equations may be the same everywhere in all universes but may not completely determine the values of all the effective, apparently ``fundamental'' constants at low energies in each one. The Theory of Everything currently under construction, even in its final form, may never provide a derivation from first principles of all the pure numbers controlling everyday phenomenology. These may instead be primarily determined by a kind of selection, dubbed the ``anthropic principle'' by Carter, the ``principle of complexity'' by Reeves, the ``principle of effectiveness'' by Rozenthal, such that the elementary building blocks of the universe allow for complex things to happen, such as the assembly of observers. We can seek clues to the flexible degrees of freedom in the ``final theory'' by looking for parameters of the effective low-energy theory (the Standard Model) with especially powerful effects: parameters whose small variation from their actual fortuitous values lead to major qualitative changes.
Since the reviews of Carr and Rees (1979) and Barrow and Tipler (1986), advances in both physics and astronomy have, amazingly, led to progress on the ancient riddle of chance and necessity, on very different fronts: at one extreme the very concrete circumstances about our local habitable environment and its detailed history; at the other extreme, the most abstract levels of physics. The natural history of the solar system and the Galaxy have revealed new couplings between biology and the astrophysical environment, as well as actual data on other solar systems. Inflationary multiverses (e.g., Vilenkin 1998b) now provide a physical framework to discuss different choices of physical vacuum which may allow some of the parameters of low-energy physics (which we try to identify) to be tuned by selection. At the same time, unified theories constrain some relations among the parameters to be fixed by symmetry. Remarkably, the freedom still available to tune parameters in Grand Unified Theories appears well matched to that required to select parameters which yield a complex phenomenology at low energy. Simple arguments suggest that one independent coupling constant and two out of the three light fermion masses (the down quark mass, and either the up quark or electron mass) may not be fixed by symmetry, which allows the fundamental theory enough flexibility to find a combination with a rich nuclear and chemical phenomenology; the other relationships among the 20 or more parameters of current standard theory can be fixed by symmetries of unification mathematics.
It is easy to guess wrong about selection effects and it is worth recalling the history of the Large Numbers Hypothesis. Dirac (1938) saw two of the large numbers of nature - the weakness of gravity and the low density of the universe - and concluded, incorrectly, that gravitational coupling depends on cosmic density. The correct insight (by Dicke, 1961) was that the density of the universe is determined by its age, and the age of the universe is mainly fixed by our own requirements, probably mainly to do with how long it takes stellar populations to synthesize the heavy nuclei needed for planets and life. The long timescales associated with stars ultimately derive from the weakness of gravity and the energy available from nuclear fusion. Once it is granted that our presence requires evolved stars, Dirac's coincidence can be derived from physical models of stars. Carter (1983) extended the argument to draw conclusions about the intrinsic timescales of biological evolution, some of which appear to be confirmed by modern astrobiology. Fossil evidence now confirms intricate couplings of biological and astronomical processes throughout the history of the Earth, and we have developed enough understanding to guess that highly complex life requires a rare combination of factors (Ward and Brownlee 1999).
It is also easy to discredit anthropic arguments. In the same way that Darwinian natural selection can be discredited by silly ``Just So Stories'' (How the Leopard Got His Spots, etc.), anthropic arguments are sometimes used indiscriminately; for example, when a theory of quantum cosmology essentially fails to predict anything, so that all the important features of the universe must be attributed to selection. Such extreme applications of anthropic reasoning undermine the essential goal of unification physics, to achieve an elegant mathematical explanation for everything. Yet one must bear in mind - dare we call it a Principle of Humility? - that at least some properties of the world might not have an elegant mathematical explanation, and we can try to guess which ones these are.