{"id":6669,"date":"2025-10-01T04:37:01","date_gmt":"2025-10-01T04:37:01","guid":{"rendered":"https:\/\/annualreports.es.net\/?page_id=6669"},"modified":"2025-10-15T04:28:49","modified_gmt":"2025-10-15T04:28:49","slug":"highlight-real-time-streaming","status":"publish","type":"page","link":"https:\/\/annualreports.es.net\/index.php\/highlight-real-time-streaming\/","title":{"rendered":"Highlight: Real-Time Streaming"},"content":{"rendered":"\n
\n
\n
\"Fiber<\/span>
\n

EJFAT Achieves Real-Time, Terabit-Scale Data Streaming<\/h1>\n<\/div><\/div>\n\n\n\n

In April 2024, scientists at Thomas Jefferson National Accelerator Facility (Jefferson Lab) and ESnet achieved a major breakthrough for the transmission and processing of scientific data: they streamed raw physics data at 100 gigabits per second from Virginia to California, where it was processed on the Perlmutter<\/em> supercomputer at NERSC and the results returned in real time \u2014 with no buffering, storage delays, or data loss.<\/p>\n\n\n\n

\n
\"Two
JEFFERSON NATIONAL ACCELERATOR FACILITY STAFF SCIENTIST VARDAN GYURJYAN AND ESNET RESEARCHER YATISH KUMAR GAVE AN EJFAT DEMO AT THE SC24 DOE BOOTH.
<\/figcaption><\/figure>\n\n\n\n

The test used the ESnet\u2013JLab FPGA Accelerated Transport (EJFAT) prototype, a hardware-accelerated load balancer designed to connect high-speed scientific instruments directly to remote DOE high-performance computing (HPC) facilities. EJFAT uses Field Programmable Gate Arrays (FPGAs) to shape and route data within nanoseconds, dynamically balancing workloads across thousands of compute cores.

In this demonstration, 40 Perlmutter nodes (over 10,000 cores) continuously processed replayed data from Jefferson Lab\u2019s CLAS12 detector, sending results back every 400 microseconds \u2014 reducing turnaround from the 14 hours typical of traditional file-based workflows to near-instant feedback. Later tests replicated the 100 Gbps throughput and achieved simultaneous streaming across ~20,000 cores at NERSC, Oak Ridge Leadership Computing Facility, and the FABRIC testbed, in anticipation of projected demands of next-generation experiments like the Electron-Ion Collider.<\/p>\n<\/div>\n\n\n\n

EJFAT\u2019s success marks the first production-level demonstration of fully remote, distributed data stream processing for real physics data across multiple DOE facilities \u2014 a critical step toward the Integrated Research Infrastructure (IRI) vision of seamlessly linking instruments, compute, and storage across the DOE complex. With its scalable, stateless design, EJFAT is quickly being adapted to other time-sensitive facilities \u2014 from light sources to fusion devices \u2014 enabling scientists to steer experiments in real time and accelerating discovery across disciplines.<\/p>\n\n\n\n

\n