ESnet’s mission compels us to share knowledge, experience, and findings with both the DOE ecosystem and the global R&E networking community, including commercial networking technology providers. We do so by technical consulting, publishing, and participating in DOE and other U.S. scientific planning endeavors as well as contributing keynote speeches, technical presentations, demonstrations, and workshops at numerous high-profile conferences annually.

Spotlight: Pushing Boundaries with SCinet at SC23 

For decades, ESnet has been a crucial supporter of the International Conference for High Performance Computing, Networking, Storage, and Analysis known simply as SC. Beyond providing network services for the conference itself, ESnet staff volunteer to help build and manage SCinet, a temporary network that pushes the boundaries of high-speed data transfer. SCinet not only powers the conference but also serves as a testing ground for groundbreaking technologies that advance HPC.

SCinet famously requires “a year to design, a month to build, a week to operate, and a day to teardown.” In 2023, SCinet achieved record speeds of up to 6.71 Terabits per second, up from 5.01 Tbps for SC22 — and ESnet provided up to 2.8 Tbps of that capacity. 

The almost 200 volunteers, from 9 countries and 113 institutions, were split into 18 teams. ESnet staff served in multiple leadership roles: as SCinet’s Executive Director, Executive Advisor, Technical Director, Wide Area Network (WAN) Co-Lead, Deputy WAN Lead, Co-Chair of the Routing Team, and IPv6 adviser. (ESnet also serves on the leadership team for the Women in IT Networking at SC program; see page XX.) The SCinet milestones for 2023 included:

• Next-Generation Internet Protocol: Rolled out IPv6, the latest version of the internet protocol, using DHCP option 108, NAT64 and DNS64 to ease the transition from IPv4.
• Blazing-Fast Wireless: Provided high-speed 5G wireless internet, through 350 access points, access across the conference venue.
• High-Speed Optical Networking: Utilized multiple 400G optical connections and the Internet2 commodity circuit to create ultra-fast links.
• Automated Network Management: Deployed sophisticated software to automate the configuration and management of network equipment from different vendors.
• Enhanced Security: Strengthened network security by routing traffic through firewalls and implementing real-time threat monitoring.
• Advanced Data Visualization: Utilized powerful tools like InfluxDB and Grafana to visualize and analyze network traffic patterns.

These technologies, combined with record-breaking bandwidth capacity, ensured a seamless and high-performance networking experience for conference attendees.

Key Conferences for ESnet

SC

At the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), ESnet plays primarily a technical role, supporting the fastest temporary network in the world, SCinet (see preceding page). However, ESnet staff also participate as panelists and speakers. In 2023, ESnetters spoke on continuum computing, the open-source ESnet SmartNIC, SENSE and Rucio/FTS/XRootD interoperation (page XX), the SciStream architecture and toolkit for data streaming, and long-distance, high-speed data transfer with EScp 0.7. 

TNC

The largest research and education networking conference, TNC is funded by the European Union via the GÉANT (GN4-3) Project. It attracts more than 800 participants from 70+ countries. At TNC23 in Albania, ESnet staff ran a workshop on the Workflow Orchestrator (page XX) and presented on other network automation projects and the Science DMZ. 

Internet2 Technology Exchange 

Hosted by Internet2, the TechX conference brings together the global R&E community. At the 2023 TechX conference in Minneapolis, ESnet’s presentation topics included AI-driven traffic engineering, traffic measurement and analysis, orchestration, High Touch services, and Zeek. 

NSF Cyber Security Summit

In 2023, the National Science Foundation’s (NSF’s) annual Cybersecurity Summit, organized by Trusted CI, was held for the first time at Berkeley Lab. ESnet staff members served on the program committee, led training sessions on Zeek, and delivered a plenary talk on cybersecurity for R&E networks. Additionally, ESnet actively participated in SAFER, an operational security group, organizing closed-door sessions that included engaging talks, hands-on exercises simulating real-world cyberattacks, and even a satirical presentation from the perspective of a ransomware recruiter.

OFC

ESnet participated in talks and demonstrations for OFC 2023, the Optical Fiber Communication Conference and Exhibition sponsored by Optica, IEEE Communication Society, and IEEE Photonics Society, as well as in building OFCnet, an unconventional high-speed network connecting the show floor to a research center in Chicago.

QuNET

SIGCOMM’s first-ever Workshop on Quantum Networks and Distributed Quantum Computing (QuNet) was held in New York in September 2023. ESnet researchers with dual appointments with QUANT-NET (page XX) contributed a paper to the proceedings.

GrafanaCON

ESnet Measurement & Analysis staff presented at GrafanaCON, an annual visualization conference hosted by Grafana Labs, sharing how ESnet uses and extends our Stardust data collection, analysis, and visualization platform to support science. 

ESCC Meetings 

The ESnet Site Coordinator Committee (ESCC) is made up of technical representatives from ESnet’s supported sites. They gathered twice in 2023, in Berkeley, California, and Gaithersburg, Maryland, to share insights and collaborate on the operational, network engineering, and security innovations and challenges that will shape the future of the ESnet infrastructure.

Technical Presentations and Publications

In 2023, ESnet staff published 7 major reports, 5 journal articles, and 20 conference papers, as well as delivered 99 technical presentations. (For a full list, with links where available, visit https://www.es.net/esnet-2023-pubs.) The following examples were selected to illustrate ESnet’s work in the applied research areas featured in this report.

IRI-related reports

Beginning in 2021 at the direction of the DOE Office of Science (SC), senior ASCR facility personnel have come together to explore the concept of an Integrated Research Infrastructure (IRI) that would “empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation.” 

The Integrated Research Infrastructure Architecture Blueprint Activity (IRI ABA) was launched in 2022 to create a reference framework that would guide a unified strategy for IRI across the entire SC. The IRI ABA engaged more than 150 experts from all 28 SC user facilities spanning 13 national laboratories, including several from ESnet, to address the technological, policy, and sociological challenges associated with the implementation of IRI. 

In addition to participating in the IRI working group (WG), working on IRI pathfinder projects, work streams, and the IRI testbed, ESnet staff contributed to the following landmark IRI reports:

Accelerating data movement

M. Goodrich, C. Timmer, V. Gyurjyan, D. Lawrence, G. Heyes, Y. Kumar, S. Sheldon, “ESnet/JLab FPGA Accelerated Transport,” IEEE Transactions on Nuclear Science, vol. 70, no. 6, pp. 1096-1101, June 2023.

AI-driven networking

N. Buraglio, D. Carder (presenters), “AI-Driven Traffic Engineering: HECATE,” Internet2 Technology Exchange (TechEx23), Minneapolis, MN, September 2023.

Community studies

W. Bhimji, D. Carder, E. Dart, J. Duarte, I. Fisk, R. Gardner, C. Guok, B. Jayatilaka, T. Lehman, et al., “Snowmass 2021 Computational Frontier CompF4 Topical Group Report Storage and Processing Resource Access,” Computing and Software for Big Science, April 2023. 

IPv6 adoption

M. Babik, M. Bly, N. Buraglio, T. Chown, D. Christidis, et al, “Overcoming Obstacles to IPv6 on WLCG,” 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023), May 2023. 

N. Buraglio, “Federal IPv6-only Impact to Private Industry,” Internet Engineering Task Force (IETF) 117, July 2023

In-network caching

A. Sim, E. Kissel, D. Hazen, C. Guok, “Experiences in Deploying In-Network Data Caches,” 26th International Conference on Computing in High Energy & Nuclear Physics (CHEP2023), May 2023. 

C. Sim, K. Wu, A. Sim, I. Monga, C. Guok, F. Wurthwein, D. Davila, H. Newman, J. Balcas, “Effectiveness and Predictability of In-Network Storage Cache for Scientific Workflows,” IEEE International Conference on Computing, Networking and Communication (ICNC 2023), Feb. 2023. 

Quantum networking 

I. Monga, E. Saglamyurek, E. Kissel, H. Haffner, W. Wu, “QUANT-NET: A Testbed for Quantum Networking Research Over Deployed Fiber,” QuNet’23: Proceedings of the 1st Workshop on Quantum Networks and Distributed Quantum Computing, September 10-14, 2023. 

J. Phillips, R. Tollefsen, B. You, E. Perego, E. Saglamyurek, I. Monga, H. Haeffner, et al., “Towards a Quantum Network Testbed Over Deployed Fiber with Trapped Ions,“ APS Division of Atomic, Molecular and Optical Physics Meeting Abstracts, 2023. 

E. Saglamyurek, W. Wu, I. Monga, A. Sipahigil, H. Haeffner, “Photon-Matter Quantum Interfaces for Scalable and Networked Quantum Computer,” DOE Basic Research Needs in Quantum Computing and Networking Workshop, Washington, D.C., July 11-13, 2023. 

Security

F. Bannat Wala, S. Campbell, M. Kiran, “Insights into DoH: Traffic Classification for DNS over HTTPS in an Encrypted Network,” SNTA ’23: Proceedings of the 2023 on Systems and Network Telemetry and Analytics, July 2023.