{"id":6692,"date":"2025-10-01T05:09:12","date_gmt":"2025-10-01T05:09:12","guid":{"rendered":"https:\/\/annualreports.es.net\/?page_id=6692"},"modified":"2025-10-15T04:30:30","modified_gmt":"2025-10-15T04:30:30","slug":"2024-operational-innovations","status":"publish","type":"page","link":"https:\/\/annualreports.es.net\/index.php\/2024-operational-innovations\/","title":{"rendered":"2024 Operational Innovations"},"content":{"rendered":"\n
\n
\n
ESnet\u2019s network is operated as a high-availability network 24 hours per day, 365 days per year, and significant resources are devoted to ensuring its continuous and secure availability. Our staff continues to work on enhancing the technical systems that benefit both our internal team and external production services, to ensure our performance and reliability continue to stay ahead of demand. \n\nIn 2024, ESnet\u2019s operational achievements included (but were not limited to) nearly quadrupling our total trans-Atlantic bandwidth<\/a>; improving our tools for gathering, analyzing, and studying network performance data; and further automating the setup of devices on the production network. The increase in network capacity was planned to handle growth in network traffic easily, and we continue to plan for anticipated larger increases in the coming years due to adoption of AI tools.<\/pre>\n\n\n\n
Strengthening Site and Data Center Resiliency<\/strong><\/h3>\n\n\n\n
\n
While ESnet is required to deliver \u201cthree 9s\u201d \u2014 99.9% availability \u2014 we routinely exceed that standard. In fact, in 2024 ESnet hit a 100% Availability milestone<\/a>, which was driven in large part by the Site Resiliency Program (SRP). The program\u2019s importance only continues to grow as DOE science becomes more distributed, collaborative, and reliant on cloud-based resources. 

The SRP seeks to align each site’s network resilience to its mission needs, supporting highly available, high-performance connectivity”. In 2024, three sites improved their resiliency scores, and progress continued on major multi-year upgrades, including the complex Oak Ridge area connectivity project. Many efforts will carry forward into 2025.

Beyond site connectivity, ESnet advanced operational resiliency through our High Availability Location (HAL) initiative. For over a decade, ESnet has maintained redundant data centers at Berkeley Lab in California and Brookhaven National Laboratory in New York. However, the cross-country separation incurred latency and required manual intervention during some outages.<\/p>\n\n\n\n
\"Person\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
An ESnet Data and Facilities team member replaces an EJFAT load balancer.\n<\/figcaption><\/figure>\n<\/div>\n\n\n\n
The HAL project addresses this by adding a San Jose facility that enables automated, real-time synchronization with Berkeley and dramatically reduces recovery times. In 2024, ESnet deployed secure overlay networks between the two sites and created a \u201cstretched\u201d VMware cluster, allowing services to migrate seamlessly between data centers with minimal downtime.<\/p>\n\n\n\n
In 2025, ESnet will begin transitioning production services into this environment, improving availability, reducing operational complexity, and strengthening the resilience of critical systems that support DOE\u2019s science mission.<\/p>\n\n\n\n
perfSONAR Expands Global Reach and Capabilities<\/strong><\/h3>\n\n\n\n
perfSONAR<\/a> \u2014 the p<\/strong>erformance S<\/strong>ervice-O<\/strong>riented N<\/strong>etwork monitoring AR<\/strong>chitecture \u2014 is a global platform for measuring and monitoring end-to-end network performance. ESnet is a core partner in the perfSONAR collaboration, with Internet2, Indiana University, G\u00c9ANT, the University of Michigan, and Brazil\u2019s RNP. Each contributes staff, computing, and network resources. Today, perfSONAR is deployed at over 2,000 locations worldwide. In 2024, major new deployments supported critical science projects:<\/p>\n\n\n\n