Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging

Magdalena Wysocki*, Mohammad Farid Azampour*, Christine Eilers, Benjamin Busam, Mehrdad Salehi, Nassir Navab

MIDL 2023 (Oral Presentation)

* equal contribution

OpenReview PDF Code (GitHub) Data

Ultra-NeRF overview figure — Ultra-NeRF fuses overlapping sweeps via physics-enhanced implicit neural representations for view-consistent ultrasound synthesis.

Abstract

We present a physics-enhanced implicit neural representation for ultrasound imaging that learns tissue properties from overlapping sweeps. Recent INR approaches encode three-dimensional scenes from two-dimensional ultrasound frames but ignore the strong view-dependent appearance changes that arise from ultrasound physics. We tackle this limitation with a ray-tracing-based neural renderer that models direction-dependent reflections and attenuation, enabling realistic synthesis from novel views. We validate the approach on simulated liver data and tracked spine phantom acquisitions, demonstrating sharper geometry and improved volume compounding for unseen views—the first INR framework to explicitly address view-dependent ultrasound synthesis.

Highlights

Physics-inspired neural rendering couples ray tracing with attenuation-aware appearance terms.
Learned INR spans multiple calibrated sweeps, producing geometrically consistent B-mode images.
Enables compounding from novel viewpoints for both synthetic liver simulations and robotic phantom scans.

Datasets & sampling

Ultra-NeRF data overview — Multi-sweep acquisitions provide calibrated poses from diverse viewing angles over the same anatomy.

Sampled rays illustration — Equidistant sampling of rays forms the basis for the differentiable rendering pipeline.

Citation

If you reference this work, please cite:

@inproceedings{wysocki2023ultranerf,
  title={Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging},
  author={Magdalena Wysocki and Mohammad Farid Azampour and Christine Eilers and Benjamin Busam and Mehrdad Salehi and Nassir Navab},
  booktitle={Medical Imaging with Deep Learning (MIDL)},
  year={2023}
}