The story of the serendipidous discovery of the microwave background in 1965 is widely known, so I will only briefly summarize it here. A recent book by the historian of science Helge Kraugh (1996) is a careful and authoritative reference on the history of cosmology, from which much of the information in this section was obtained. Arno Penzias and Robert Wilson, two radio astronomers at Bell Labs in Crawford, New Jersey, were using a sensitive microwave horn radiometer originally intended for talking to the early Telstar telecommunications satellites. When Bell Labs decided to get out of the communications satellite business in 1963, Penzias and Wilson began to use the radiometer to measure radio emission from the Cassiopeia A supernova remnant. They detected a uniform noise source, which was assumed to come from the apparatus. But after many months of checking the antenna and the electronics (including removal of a bird's nest from the horn), they gradually concluded that the signal might actually be coming from the sky. When they heard about a talk given by P.J.E. Peebles of Princeton predicting a 10 K blackbody cosmological background, they got in touch with the group at Princeton and realized they had detected the cosmological radiation. At the time, Peebles was collaborating with Dicke, Roll, and Wilkinson in a concerted effort to detect the microwave background. The Princeton group wound up confirming the Bell Labs discovery a few months later. Penzias and Wilson published their result in a brief paper with the unassuming title of ``A measurement of excess antenna temperature at = 7.3 cm'' (Penzias and Wilson 1965); a companion paper by the Princeton group explained the cosmological significance of the measurement (Dicke et al 1965). The microwave background detection was a stunning success of the Hot Big Bang model, which to that point had been well outside the mainstream of theoretical physics. The following years saw an explosion of work related to the Big Bang model of the expanding universe. To the best of my knowledge, the Penzias and Wilson paper was the second-shortest ever to garner a Nobel Prize, awarded in 1978. (Watson and Crick's renowned double helix paper wins by a few lines.)
Less well known is the history of earlier probable detections of the microwave background which were not recognized as such. Tolman's classic monograph on thermodynamics in an expanding universe was written in 1934, but a blackbody relic of the early universe was not predicted theoretically until 1948 by Alpher and Herman, a by-product of their pioneering work on nucleosynthesis in the early universe. Prior to this, Andrew McKellar (1940) had observed the population of excited rotational states of CN molecules in interstellar absorption lines, concluding that it was consistent with being in thermal equilibrium with a temperature of around 2.3 Kelvin. Walter Adams also made similar measurements (1941). Its significance was unappreciated and the result essentially forgotten, possibly because World War II had begun to divert much of the world's physics talent towards military problems.
Alpher and Herman's prediction of a 5 Kelvin background contained no suggestion of its detectability with available technology and had little impact. Over the next decade, George Gamow and collaborators, including Alpher and Herman, made a variety of estimates of the background temperature which fluctuated between 3 and 50 Kelvin (e.g. Gamow 1956). This lack of a definitive temperature might have contributed to an impression that the prediction was less certain than it actually was, because it aroused little interest among experimenters even though microwave technology had been highly developed through radar work during the war. At the same time, the incipient field of radio astronomy was getting started. In 1955, Emile Le Roux undertook an all-sky survey at a wavelength of = 33 cm, finding an isotropic emission corresponding to a blackbody temperature of T = 3 ± 2 K (Denisse et al. 1957). This was almost certainly a detection of the microwave background, but its significance was unrealized. Two years later, T.A. Shmaonov observed a signal at = 3.2 cm corresponding to a blackbody temperature of 4 ± 3 K independent of direction (see Sharov and Novikov 1993, p. 148). The significance of this measurement was not realized, amazingly, until 1983! (Kragh 1996). Finally in the early 1960's the pieces began to fall in place: Doroshkevich and Novikov (1964) emphasized the detectability of a microwave blackbody as a basic test of Gamow's Hot Big Bang model. Simultaneously, Dicke and collaborators began searching for the radiation, prompted by Dicke's investigations of the physical consequences of the Brans-Dicke theory of gravitation. They were soon scooped by Penzias and Wilson's discovery.
As soon as the microwave background was discovered, theorists quickly realized that fluctuations in its temperature would have fundamental significance as a reflection of the initial perturbations which grew into galaxies and clusters. Initial estimates of the amplitude of temperature fluctuations were a part in a hundred; this level of sensitivity was attained by experimenters after a few years with no observed fluctuations. Thus began a quarter-century chase after temperature anisotropies in which the theorists continually revised their estimates of the fluctuation amplitude downwards, staying one step ahead of the experimenters' increasingly stringent upper limits. Once the temperature fluctuations were shown to be less than a part in a thousand, baryonic density fluctuations did not have time to evolve freely into the nonlinear structures visible today, so theorists invoked a gravitationally dominant dark matter component (structure formation remains one of the strongest arguments in favor of non-baryonic dark matter). By the end of the 1980's, limits on temperature fluctuations were well below a part in 104 and theorists scrambled to reconcile standard cosmology with this small level of primordial fluctuations. Ideas like late-time phase transitions at redshifts less than z = 1000 were taken seriously as a possible way to evade the microwave background limits (see, e.g., Jaffe et al. 1990). Finally, the COBE satellite detected fluctuations at the level of a few parts in 105 (Smoot et al. 1990), just consistent with structure formation in inflation-motivated Cold Dark Matter cosmological models. The COBE results were soon confirmed by numerous ground-based and balloon measurements, sparking the intense theoretical and experimental interest in the microwave background over the past decade.