eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Astronomers will soon face a rare challenge in their field: managing an overwhelming surge of observational data. This data explosion comes from a revolutionary digital telescope that captures a new section of the night sky roughly every 40 seconds, generating about 1,000 images per night.
Named the Vera C. Rubin Observatory, in honor of an American astronomer who helped prove the existence of dark matter, the telescope was jointly funded by the US National Science Foundation and the US Department of Energy. It is located atop the Cerro Pachón mountain in the Chilean Andes.
A telescope designed for data
The Rubin Observatory is unique in the amount of data it collects. While the telescope’s 28-foot-wide mirror isn’t especially impressive, it does boast the world’s largest digital camera, built with 3.2 billion pixels per image. That translates to approximately 6.4 billion bytes of data captured in every shot.
Even more remarkably, the telescope will collect data every day for 10 years, producing a staggering 60 million billion bytes of image data.
From storage to global distribution
Handling this volume of information presents several challenges.
The first is storage. To solve this, the Rubin Observatory built a high-tech data center capable of storing up to a month’s worth of images in the event of network outages. A 60-mile fiber-optic cable connects the observatory to La Serena, Chile, though the line is occasionally damaged.
From there, data is transmitted to the SLAC National Accelerator Laboratory, a US Department of Energy research center in Menlo Park, California, for advanced analysis. This process involves software that compares copies of the same image taken over three consecutive nights to identify changes.
How AI helps astronomers navigate the sky
Rubin’s software detects approximately 10,000 changes per image, resulting in nearly 10 million alerts each night. To make this data useful, the observatory partners with nine external organizations known as data brokers. These systems sort and classify changes, allowing astronomers with various specialties to access the observations most relevant to their work.
“It’s better to send that out to a global community of scientists with a lot of different skills and expertise to bring in their knowledge,” Leanne Guy, the observatory’s data management scientist, told The New York Times.
Data brokers employ several methods to sort the telescope’s data. One is classic machine-learning methods, which categorize events using preselected criteria. Another leverages neural networks and other modern deep-learning methods that create their own criteria for identifying different phenomena.
A new kind of astronomy
The Rubin Observatory marks the beginning of a new era in cosmic research. Astronomers will only need high-speed networks, cloud computing, and AI algorithms to perform research anytime, anywhere.
“We produce lots of data for everyone,” said William O’Mullane, the observatory’s associate director for data management, per The New York Times. “So this idea of coming to the telescope and making your observation doesn’t exist, right? Your observation was made already. You just have to find it.”
Explore eWeek’s deep dive into how AI is transforming the search for alien life across Earth-like exoplanets — and what it means for the future of space discovery.