Building a Data Pipeline to Accelerate Discovery

High-Performance Capabilities

The innovation behind DELERIA lies in its unique approach and architecture, co-designed by ESnet engineers. Specifically, forward buffers allow information to be easily collected from the experiment’s electronics and transmitted across ESnet to an offsite computing facility using a messaging protocol that allows high-speed data transfer without losing information. And the use of software containers simplifies deployment across multiple systems by compartmentalizing information to ensure consistent, quick, and scalable deployment.

Ultimately, these innovations enable scientists to run their experiments and computing functions faster while delivering results in real time. “DELERIA now lets us take advantage of the extensive and significant computational advantage at the remote facilities,” Crawford said. “Now we get to think about, ‘How can we do this even better?’” With DELERIA, data moves at about 40 gigabits every second across the ESnet testbed network. To put that speed in perspective, it’s like moving a two-hour HD Netflix movie every second.

Connecting GRETA to the distant supercomputing testbed is ESnet, a high-performance, high-speed network used by tens of thousands of scientific researchers across the DOE complex. After the many data points are analyzed by the OLCF supercomputers, it is sent back in a condensed readout to the scientists as they iterate on their work in real time. “Science is no longer something that is done only in the lab, it is all connected together,” said Eric Pouyoul, ESnet’s software engineer for DELERIA. “It doesn’t matter where the data is, because with ESnet, it can go anywhere.”

A Wide Range of Science Applications

The teams involved are keeping an eye on a bigger perspective beyond the GRETA project. “The goal is to make this available to other sciences,” said Pouyoul. “The way that the DELERIA computing system has been designed, architected, and implemented leads us to believe this is possible.”

The teams involved in this work have already identified many potential applications within and beyond the world of nuclear science. Equipped with real-time feedback on a fusion shot, scientists can update diagnostic parameters for the next shot; and for light source users, real-time analysis could guide experimental settings used to illuminate a sample instantaneously, enabling optimal utilization of time on the beamline.

Further reading:

ORNL article on DELERIA

###

Lawrence Berkeley National Laboratory (Berkeley Lab) is committed to groundbreaking research focused on discovery science and solutions for abundant and reliable energy supplies. The lab’s expertise spans materials, chemistry, physics, biology, earth and environmental science, mathematics, and computing. Researchers from around the world rely on the lab’s world-class scientific facilities for their own pioneering research. Founded in 1931 on the belief that the biggest problems are best addressed by teams, Berkeley Lab and its scientists have been recognized with 16 Nobel Prizes. Berkeley Lab is a multiprogram national laboratory managed by the University of California for the U.S. Department of Energy’s Office of Science.

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.