Reaching for the Stars: Bringing the Cosmos to a Computer Near You

Jeremiah Ostriker is being honored as a Champion of Change for the vision he has demonstrated and for his commitment to open science.

I have been very lucky.

Growing up in a big city where the lights obscured the night sky, I became more and more curious about what was “out there,” above the bustle of urban life, above the US, far above the little speck of dust we call planet Earth. Books on the solar system of planets and on the world of stars that made up our Milky Way galaxy fascinated me. Big telescopes, on the ground and in the good climate of the Western part of our country, were beginning to peer into the wider universe of galaxies and cosmology. Hubble had discovered the expanding universe whose fate was now a serious scientific subject, no longer in the realm of philosophy or theology but a domain where measurement and calculation based on physical laws was possible.

At that time, astronomers applied for time on telescopes, took observations, and then carried the results to their home institutions where they sat in desk drawers until years of slow analysis allowed the results to be published in technical journals. In my first job as a student at Yerkes Observatory in Wisconsin, I sat for months with a mechanical calculator analyzing data (and making mistakes) for a senior astronomer.

Needless to say, all data was proprietary, and competition was fierce. If someone else published results before you did, it was a disaster, so cooperation between groups was unheard of. Science moved – but relatively slowly.

But, starting in the mid-1960s, roughly a half-century ago, revolutionary changes, driven largely by technology, changed the game, and progress accelerated at an extraordinary rate. New electronic detectors were far more efficient, and the information was recorded electronically, making the use of digital computers possible. Astronomy took off.

With a boost from “Moore’s Law” (the increase of computing power by a factor of two every 18 months) we went from bits to megabytes to terabytes in a few decades. Instead of looking at one spot or another on the sky and storing what we saw, we could now survey large areas; we could do it in several colors from the far infrared to the ultraviolet. And, we could do it again and again to see what was changing in the distant universe. The revolution was incredible.

Then telescopes were placed on satellites. The obscuring and blurring effects of our atmosphere were bypassed, and astonishing results could be obtained, regardless of the cloud cover, 24/7. Now we astronomers had leapt off the planet ourselves. Policies on the availability and distribution of the results had to change, and did change in a fashion as revolutionary as the results themselves.

The Sloan Digital Sky Survey exemplifies this transition in science. Astronomers, who had experience in cooperating with one another across boundaries since the Renaissance, led the way. It was initiated in conversations in Princeton in the late 1980s with Jim Gunn leading on the technical side and me on the organizational aspects. In the next several years a consortium was established with funds raised from private sources, universities, the Sloan Foundation, and several US government agencies to put a relatively small (2.5m) telescope at Apache Point Observatory in New Mexico and repeatedly scan the available sky in a dedicated mode. Over 40 institutions, both US and foreign, have taken active roles in the enterprise. The results have far surpassed the wildest early expectations.

Very early on in the endeavor, Principles of Operation were established that made open to all participants all of the data acquired by any of them – a startling change from most existing astronomical practice. It was further required that all data be archived and made available to the scientific world and to the public within two years after it was initially acquired.

This practice had an extraordinary, and intended, byproduct. It became to the advantage of every participant that instruments and data reduction techniques of the other scientists be maximally effective! Thus, cooperation among the various teams was enthusiastic and effective from the very beginning.

In operation since 2000, SDSS has archived 70 terabytes of data from 1/3 of the celestial sphere including over half a billion galaxies, stars, quasars and asteroids. It has resulted in over 5000 refereed publications that have been cited over 200,000 times. This is a record that tops any previous, ground based astronomical project – astonishing for results of a relatively small telescope - and due in no small part to the “Open Science” character of its charter and operations.

The successor program, the Large Synoptic Survey Telescope (LSST), is in the President’s 2014 budget. A much larger telescope at a superb Chilean site is planned to extend and enhance this policy of open distribution of data, with results that should greatly extend our knowledge and understanding of the universe around us. The story of this half-century revolution of cosmological discovery and comprehension is told in my recent book – “Heart of Darkness: Unraveling the Mysteries of the Invisible Universe.”

I have been lucky indeed to have lived through and participated in this exciting adventure.

Jeremiah Ostriker is a Professor of Astronomy at Columbia University.