In a noteworthy development for astrophysics and computational science, the Department of Energy's Argonne National Laboratory has used the world's fastest supercomputer, Frontier, to conduct the most extensive universe simulation to date. This milestone triumph was announced in an article from Oak Ridge National Laboratory, detailing the groundbreaking use of Frontier at their Oak Ridge Leadership Computing Facility. Researchers ran advanced cosmological hydrodynamics simulations at an unprecedented scale, setting a new standard for complex simulations that include both atomic and dark matter.
"There are two components in the universe: dark matter — which as far as we know, only interacts gravitationally — and conventional matter, or atomic matter," Salman Habib, division director for Computational Sciences at Argonne, told Oak Ridge National Laboratory. The simulations incorporate elements ranging from hot gas to star and galaxy formation, encompassing the vast timeline of the universe's expansion spanning billions of years. Previously, this level of comprehensive simulation was unattainable outside of gravity-only models, according to the Oak Ridge National Laboratory's article.
The significant computational leap was facilitated by the Hardware/Hybrid Accelerated Cosmology Code (HACC), initially designed for petascale machines and a Gordon Bell Prize finalist in 2012 and 2013. Over the past seven years, as part of ExaSky—a subset of the Exascale Computing Project (ECP)—Salman Habib and his team upgraded HACC for exascale-class supercomputers, a requirement of which was to perform about 50 times faster than the former fastest supercomputer at the time of the ECP's launch, Titan.
In the end, HACC's performance on Frontier's AMD Instinct™ MI250X GPUs-operated compute nodes was nearly 300 times faster than the reference run on Titan. "It's not only the sheer size of the physical domain, which is necessary to make direct comparison to modern survey observations enabled by exascale computing," said Bronson Messer, OLCF director of science, according to the Oak Ridge National Laboratory. "It's also the added physical realism of including the baryons and all the other dynamic physics that makes this simulation a true tour de force for Frontier."
Members of the HACC team who contributed to the successful simulation include Michael Buehlmann, JD Emberson, Katrin Heitmann, Patricia Larsen, Adrian Pope, Esteban Rangel, and Nicholas Frontiere, who was at the helm of the Frontier simulations. Prior to utilizing Frontier, HACC was tested on the Perlmutter supercomputer at Lawrence Berkeley National Laboratory's National Energy Research Scientific Computing Center and on the exascale-class Aurora supercomputer at Argonne Leadership Computing Facility.
The fusion of expert talent, advanced technology, and funding support underlines the importance of collaboration in science and research. Notably, the DOE’s Office of Science acknowledged as the largest supporter of basic research in the physical sciences in the United States, was instrumental in the project. For more information about these efforts and their impact on the understanding of the universe, visit the official Oak Ridge National Laboratory news release.
