In this Chapter we have attempted to summarize, albeit briefly, the history and the state of the art of sky surveys. However, this is a very rapidly evolving field, and the reader is advised to examine the subsequent literature for the updates and descriptions of new surveys and their science.
In some fields, surveys completely dominate the observational approaches; for example, cosmology, either as a quest to describe the global properties of the universe, the nature of the dark energy, etc., or the history of structure and galaxy formation and evolution, is now tackled largely through large surveys, both from ground and space. Surveys discover cosmic explosions, extrasolar planets, and even new or predicted phenomena.
Sky surveys have transformed the ways in which astronomy is done, and pushed it from the relative data poverty to a regime of an immense data overabundance. They are the by far the largest generators of data in astronomy, and they have already enabled a lot of important science, and will undoubtedly continue to do so. They have also fostered the emergence of the Virtual Observatory framework and Astroinformatics as means of addressing both the challenges and the opportunities brought by the exponential data growth. They also represent a superb starting point for education and public outreach, e.g., with the Google Sky and the WorldWide Telescope (WWT; http://www.worldwidetelescope.org) sky browsers.
Surveys have also revitalized the role of small telescopes in the era of giant ones, both for the surveying itself, and for the immediate imaging and photometric follow-up (Djorgovski 2002).
Small telescopes do not imply a small science. Survey-based astronomy is inherently systemic, requiring a full hierarchy of observational facilities, since much of the survey-based science is in the follow-up studies of selected sources. Mutual leveraging of survey and follow-up telescopes, on-line archives and cyber-infrastructure, creates an added value for all of them.
There is, however, one significant bottleneck that we can already anticipate in the survey-driven science: the follow-up spectroscopy of interesting sources selected from imaging surveys. While there seems to be a vigorous ongoing and planned activity to map and monitor the sky in many ways and many wavelengths, spectroscopic surveys will be necessary in order to interpret and understand the likely overabundance of potentially interesting objects. This looming crisis may seriously limit the scientific returns from the ongoing and future surveys.
Another important lesson is that the cost of these data-intensive projects is increasingly dominated by the cost of software development, implementation, and maintenance. Nobody has ever underestimated the cost of software. Our community has to develop more effective ways of sharing and leveraging software efforts. This remains as one of the key motivations behind the VO and Astroinformatics.
In addition to their roles as scientific and technological catalysts, surveys have also started to change the sociology and culture of astronomy, by opening new modes of research, new kinds of problems to be addressed, requiring new skills, and new modes of scientific publishing and knowledge preservation. This cultural shift is both inevitable and profoundly transformational. Other sciences have undergone comparable or greater changes, driven by the ways in which problems are defined, and data are obtained and analyzed; biology is a good example.
Some sociological changes may be a mixed blessing. By their nature, surveys tend to require large teams, since that can help secure the necessary resources (funding, observing time) and the manpower. Many astronomers are uneasy about this trend towards the high-energy physics mode of research. Generating large data sets requires large-scale efforts. However, important discoveries are still being made at all scales, from individuals and small groups, to large collaborations. Proposed survey science tends to be a committee-designed science, and thus often safe, but unimaginative; actual survey science tends to be dominated by the unexpected uses of the data and surprises.
One important way in which surveys have changed astronomy is their role as an intermediary step between the sky and the scientist. The large information content of modern sky surveys enables numerous studies that go well beyond the original purposes. The traditional approach where we observe selected targets and make new discoveries using such primary observational data is still with us, and will remain. However, there is now a new way of observing the sky, through its representation in the survey archives, using software instruments. It is now possible to make significant observational advances and discoveries without ever going to a telescope. Thus we see a rise in prominence of archival research, which can be as cutting-edge as any observations with the world's largest telescopes and space observatories.
This type of research requires new computational science skills, from data farming (databases, their interoperability, web services, etc.) to data mining and knowledge discovery. The methods and the tools that work efficiently in the Megabyte to Gigabyte regime usually do not scale to Terabytes and Petabytes of data, let alone the greatly increased complexity of the modern data sets. Effective visualization tools and techniques for high-dimensionality parameter sets are another critical issue. We need new kinds of expertise for the new, data-rich and data-intensive astronomy in the 21st century. As the science evolves, so does its methodology: we need both new kinds of tools, and the people who know how to use them.
Unfortunately, we are currently neither training properly the new generations of researchers in these requisite skills, nor rewarding the career paths that bridge astronomy and ICT. The culture of academia changes slowly, and these educational and professional recognition issues may be among the key obstacles in our path towards the full scientific utilization of the great and growing data abundance brought by the modern sky surveys.
Astronomy is not alone among the sciences facing these challenges. Interdisciplinary exchanges in the context of e-Science, cyber-infrastructure, and science informatics can help us tackle these important issues more efficiently. All of them signal a growing virtualization of science, as most of our work moves into the cyberspace.
To end on a positive note, we are likely entering a new golden age of discovery in astronomy, enabled by the exponential growth of the ICT, and the resulting exponential growth of data rates, volumes, and complexity. Any science must rest on the data as its empirical basis, and sky surveys are increasingly playing a fundamental role in this regard in astronomy. We have really just started to exploit them, and the future will bring many new challenges and opportunities for discovery.
Acknowledgments:
We are indebted to many colleagues and collaborators over the years, especially the key members of the survey teams: Nick Weir, Usama Fayyad, Joe Roden, Reinaldo de Carvalho, Steve Odewahn, Roy Gal, Robert Brunner, and Julia Kennefick in the case of DPOSS; Eilat Glikman, Roy Williams, Charlie Baltay, David Rabinowitz, and the rest of the Yale team in the case of PQ; and Steve Larson, Ed Beshore, and the rest of the Arizona and Australia team in the case of CRTS. Likewise, we acknowledge numerous additional colleagues and collaborators in the Virtual Observatory and the Astroinformatics community, especially Alex Szalay, Jim Gray, Giuseppe Longo, Yan Xu, Tom Prince, Mark Stalzer, and many others. Several tens of excellent undergraduate research students at Caltech contributed to our work through the years, many of them supported by the Caltech's SURF program. And last, but not least, the staff of Palomar, Keck, and other observatories, who helped the data flow. Our work on sky surveys and their exploration has been supported in part by the NSF grants AST-0122449, AST-0326524, AST0407448, CNS-0540369, AST-0834235, AST-0909182 and IIS-1118041; the NASA grant 08AISR08-0085; and by the Ajax and Fishbein Family Foundations. Some of the figures in this paper have been produced using an immersive VR visualization software, supported in part by the NSF grant HCC-0917814. We thank numerous colleagues, and in particular H. Bond, G. Longo, M. Strauss, and M. Kurtz, whose critical reading improved the text. Finally, we thank The Editors for their saintly patience while waiting for the completion of this Chapter.