Adaptive Ultrasound Imaging with Physics-Informed NV-Raw2Insights-US AI

NVIDIA and Siemens Healthineers Develop AI for Raw Ultrasound Data

NVIDIA, in collaboration with researchers from Siemens Healthineers, has released a new AI model called NV-Raw2Insights-US that processes raw ultrasound sensor data directly, bypassing traditional image formation pipelines. This approach allows the system to correct for patient-specific variations in the speed of sound through tissue, a factor that is typically simplified with a constant value in conventional systems. The result is an adaptive imaging system capable of generating clearer, more focused ultrasound images by accounting for unique patient anatomy in real time.

The Technical Stack

The core technical challenge addressed by this work is accessing and processing the high-bandwidth raw channel data from clinical-grade scanners before it is compressed into an image. The demonstration architecture uses a custom hardware and software stack to intercept this data stream using a method called "Data over DisplayPort." The key components of the system include:

• An Altera Agilex-7 FPGA running NVIDIA Holoscan Sensor Bridge (HSB) IP captures raw data from a Siemens ACUSON Sequoia scanner's DisplayPort output.
• The HSB packetizes the data and transmits it over Ethernet to an NVIDIA IGX edge AI system.
• The NV-Raw2Insights-US model performs accelerated inference on a Blackwell-class GPU to produce a patient-specific sound-speed estimate.
• This estimate is streamed back to the ultrasound scanner to enable improved focusing in the live imaging feed.

Impact on Medical Imaging

This architecture signals a significant step toward software-defined medical imaging, where advanced capabilities can be deployed through software updates rather than complete hardware replacement. By establishing a pipeline for raw sensor data into GPU memory using platforms like NVIDIA Holoscan, it creates a modular foundation for future AI development. This allows for the seamless integration of new diagnostic models into the same workflow, establishing a scalable path toward AI-native systems that learn from fundamental physics rather than just post-processed images.

By creating a hardware bridge to intercept raw sensor data from existing high-value medical equipment, NVIDIA is positioning its Holoscan and IGX platforms as a critical AI upgrade layer for the healthcare industry, enabling software-defined capabilities on legacy systems.