Begun in 2017, ESnet6 was completed in 2022 — two years ahead of schedule and under budget. This was a significant feat, especially considering the hurdles posed by the global pandemic, disruptions in the supply chain, and the task of replacing a network while it was actively in use — not unlike taking apart and re-assembling an airplane full of passengers… in mid-flight.
ESnet6 was specifically designed to help the DOE research community and its international collaborators harness the deluge of data created by scientific research in the exascale and artificial intelligence era. The project had three simple yet ambitious goals. The first was to build a network designed to handle that exponential growth in data cost-effectively. In 2021, ESnet5 carried more than 1.1 exabytes of science data; by 2022, the total exceeded 1.3 exabytes. Traffic on ESnet has increased nearly by a factor of ten every 5.5 years. Second, to invest in resiliency and reliability that would ensure real-time, seamless performance of data analysis and enable new modalities, such as streaming terabits of data directly from instruments to supercomputers thousands of miles apart. Third, to build flexibility and a foundation strong enough to support new breakthroughs in machine learning, artificial intelligence, quantum information science, and the revolutions yet to come.
ESnet6 represents a transformational change in the way networks are built for research, with improved capacity, resiliency, and flexibility. With more than 46 Terabits per second of aggregate bandwidth deployed, it features a significant increase over prior generations of the network. This capacity boost enables scientists to more quickly move, process, analyze, visualize, share, and store the mountains of research data produced by experiments, modeling, and simulations.
But the new network does more than just increase capacity. For the first time, ESnet built and managed the entire network stack built over a dedicated, nationwide15,000-mile dark-fiber footprint. With ESnet6, our engineers have developed smart, programmable, and automated services uniquely built to support the multi-petabyte dataflows typical of scientific research today. In addition, they are future-proofed to manage the emerging exabyte data era, streaming data from instruments and high-impact “digital twins” that require predictability and low latency. New capabilities and services include customizable network services via a new automation platform, high-precision telemetry to improve network performance, improved overall network security, and a future programmable API platform for scientists to directly request custom network services. (Many of these innovations are discussed further in this annual report.)
ESnet6 was officially unveiled at a special event attended by DOE and Berkeley Lab leadership, as well as local, state, and federal government representatives, including U.S. Congresswoman Barbara Lee and Asmeret Asefaw Berhe, director of the DOE’s Office of Science. The event also featured keynote talks from Internet pioneer Vint Cerf, vice president and chief internet evangelist for Google and ESnet Policy Board member; and Ian Foster, distinguished fellow at Argonne National Laboratory and professor of computer science at the University of Chicago. Each provided insights into the impact ESnet6 is having on the scientific community and the global Internet. Additional speakers included Berkeley lab scientists Ann Almgren, senior scientist and the Applied Mathematics Department head, and David McCallen, senior scientist in the Energy Geosciences Division and professor at the University of Nevada, Reno. These researchers shared background on how ESnet impacts several of their exascale computing research projects, including wind energy and earthquake simulations.
Close collaborations with Lumen Technologies, Ciena, Infinera, Nokia, and Xilinx (AMD) provided ESnet with leading-edge equipment and expertise to help bring the new network into service. These partners provided key building blocks, including ESnet’s nationwide fiber optic cable footprint, optical switching and routing platforms, and an extreme-scale packet monitoring system.
The event also included a live demonstration of ESnet6’s new automation platform, which rapidly configured ESnet6 network paths to support the transfer of large science datasets across the country in under two minutes. In addition, a live, multi-terabyte transfer of earthquake simulation data was also performed between the Oak Ridge Leadership Computing Facility to the National Energy Research Scientific Computing Center in minutes.
ESnet’s improved capacity, resiliency, and flexibility make it faster, easier, and more efficient for scientists around the world to conduct and collaborate on ground-breaking research. As Vint Cerf said at the unveiling, “The Internet is about information and data flow. With the launch of ESnet6, we are empowering the scientific community with unprecedented new capabilities to help advance the search for life-saving and planet-saving discoveries.”
and under budget. In recognition of this unusual feat, the Department
of Energy (DOE) presented ESnet with a special Project Assessment
Award. Office of Project Assessment Director Kurt W. Fisher presented it
to ESnet Network Services Group Lead Kate Petersen Mace, who as the
project’s director accepted the award on behalf of the ESnet6 team.