Super-resolution ultrasound imaging (SRUS) reconstructs microvasculature beyond the acoustic diffraction limit by detecting and localizing individual intravenously injected microbubble (MB) contrast agents over thousands of ultrafast frames. Redfern and Brown (arXiv:2407.20407, 2024) proposed a ConvNeXt-style classification network (SRUS-ConvNeXt) that consumes 1, 3, or 5 successive frames and outputs, per pixel, an MB-presence Boolean plus x and z sub-pixel offsets on a 4-fold finer grid. It achieved its best F1 detection score (F1 being the harmonic mean of detection precision and recall; 72.02%) with 3-frame input but its lowest localization error (λ/22≈4.68\lambda/22 \approx 4.68 µm at 15 MHz) with a single frame, leaving an explicit detection-versus-localization trade-off; it argued, without experiment, that the architecture scales to 10-fold upsampling far more cheaply than U-Net decoders; and it framed real-time end-to-end SRUS as the overarching goal. The specific open core - jointly achieving single-frame-level localization error with multi-frame-level detection in such a network, experimentally validating a 10x-upscaled implementation with preserved precision, parameters, and per-frame inference time, and integrating the full chain into a real-time pipeline with quantified latency - remains unaddressed in the published literature to date, even though adjacent architectures have made substantial field-level progress.

Background

Ultrasound localization microscopy (ULM), also called super-resolution ultrasound imaging (SRUS), images microvasculature at depths unattainable by optical microscopy: an intravenously injected microbubble (MB) contrast agent (a few-micron gas sphere with an oscillating, strongly echogenic shell) is detected across thousands of ultrafast (hundreds of Hz to kHz frame rate) plane-wave ultrasound frames, the center of each MB is estimated on a grid several times finer than the image pixel grid, and the accumulated localizations form a vascular map whose resolution is set by localization precision rather than the acoustic diffraction limit (roughly one wavelength λ\lambda, e.g. about 103 µm at 15 MHz). Conventional pipelines require dilute MBs so that each point-spread function (PSF, the blurred image of a point scatterer) is spatially isolated, forcing acquisition times of minutes, and their processing (beamforming, tissue-clutter rejection by singular-value-decomposition filtering of the slow-time data matrix, detection, localization, tracking) is computationally heavy - both barriers to clinical adoption. Deep learning replaces parts of this chain: convolutional neural networks (CNNs, learned stacks of convolution filters), typically in the U-Net encoder-decoder form, predict MB locations directly - van Sloun et al. (IEEE Transactions on Medical Imaging 40(3):829-839, 2021, doi:10.1109/TMI.2020.3037790) demonstrated CNN-based SRUS localization, Youn et al. (IEEE TMI 2020) achieved λ/29\lambda/29 localization error with CNNs, and Liu et al. (IEEE TMI 2020) used sub-pixel convolutions (mSPCN) for dense MB localization. Feeding several successive frames (spatiotemporal input) lets a network exploit MB motion, increasing the number of MBs detected per frame and so shortening acquisition; Milecki et al. (IEEE TMI 2021) used a 3D V-Net (a volumetric U-Net variant with encoder-decoder skip connections) on spatiotemporal volumes to recover dense networks at high MB concentration, and Lok et al. (Phys. Med. Biol. 2021) a deep fully convolutional network on spatiotemporal data. In this landscape Redfern and Brown (2024) introduced SRUS-ConvNeXt: a U-Net-free architecture of repeated ConvNeXt blocks (modernized convolutions using depthwise-separable filters, i.e. spatial and channel mixing factorized, with 7x7 kernels matched to the static MB PSF), no up/downsampling, and a classification head emitting, for the central input frame, three per-pixel outputs - an MB-detection Boolean and two sub-pixel offsets on a 4x-finer grid - which sidesteps the transposed-convolution decoder whose cost grows steeply with upsampling factor. Trained on simulations produced with SIMUS (a MATLAB ultrafast-ultrasound simulation toolbox) modeling an L22-14v linear probe at 15 MHz (signal-to-noise ratio, SNR, 1.5-5.2 dB; 100-500 µm vessels) and validated on 36,000 in vivo mouse-brain frames, the network scored best on the F1 metric (the harmonic mean of detection precision and recall) with 3-frame input (72.02 +/- 0.08%) but localized best with 1-frame input (4.68 +/- 0.23 µm, about λ/22\lambda/22), ran at 1.78 ms per 228x228-pixel frame on an NVIDIA 2080-Ti, and claimed - without experiment - that its structure extends to 10-fold upsampling with far smaller parameter and inference-time growth than U-Net decoders. The unresolved agenda, stated in the paper, is therefore threefold and unified by one objective (real-time, scalable, accurate DL-based SRUS image formation): reconcile the spatiotemporal detection gain with single-frame localization accuracy within this architecture; realize and validate the 10x-upsampling extension; and demonstrate the full pipeline (beamforming, SVD clutter filtering, network inference, localization accumulation) operating in real time.

Problem Statement

Advance the deep-learning SRUS microbubble detection-and-localization architecture of Redfern and Brown (2024) toward its stated objective - accurate, scalable, real-time super-resolution image formation - by resolving three linked, experimentally unanswered questions. (1) Detection-localization reconciliation: modify the network so that spatiotemporal input of several successive frames attains localization error at the level of the best single-frame configuration (about λ/22≈4.7\lambda/22 \approx 4.7 µm at 15 MHz on the source's simulated benchmark) while retaining or improving the multi-frame detection performance (F1 score at the level of the 3-frame configuration, about 72%; 'at the level of' means parity within the source's reported uncertainties, i.e. localization error no worse than 4.68+0.234.68 + 0.23 µm and F1 no lower than 72.02−0.08%72.02 - 0.08\%), evaluated on held-out simulations and in vivo data under the source's protocol (SIMUS-based L22-14v simulations at 1.5-5.2 dB SNR, 100-500 µm vessels; in vivo mouse-brain ultrafast acquisitions). (2) Large-upsampling validation: implement the 10-fold-upsampling extension (per-pixel sub-pixel offsets on a 10x-finer output grid) that the paper argues for but does not build, and measure how localization precision, network parameter count, and per-frame inference time scale relative to the demonstrated 4-fold configuration, testing the claim of graceful scaling versus U-Net-style decoders. (3) Real-time end-to-end integration: combine beamforming, tissue-clutter filtering, network inference, and localization accumulation into a single pipeline that sustains the acquisition frame rate, and report total end-to-end latency and throughput. The objective is a single network that simultaneously detects microbubbles at the spatiotemporal rate, localizes them at single-frame precision, scales to 10x upsampling, and runs in real time, and the three questions above are jointly posed, quantified in simulation and on in vivo validation data of the kind the source itself used (ultrafast mouse-brain acquisitions).

References

  1. Arthur David Redfern, Katherine G. Brown, 'Deep Learning for Super-resolution Ultrasound Imaging with Spatiotemporal Data', arXiv:2407.20407 (2024), https://arxiv.org/abs/2407.20407
  2. Ruud J. G. van Sloun, Oren Solomon, Matthew Bruce, Zin Z. Khaing, Hessel Wijkstra, Yonina C. Eldar, Massimo Mischi, 'Super-Resolution Ultrasound Localization Microscopy Through Deep Learning', IEEE Transactions on Medical Imaging 40(3):829-839 (2021), doi:10.1109/TMI.2020.3037790 (preprint: arXiv:1804.07661), https://doi.org/10.1109/TMI.2020.3037790
  3. Jihwan Youn, Martin Lind Ommen, Matthias Bo Stuart, Erik Vilain Thomsen, Niels Bent Larsen, Jorgen Arendt Jensen, 'Detection and Localization of Ultrasound Scatterers Using Convolutional Neural Networks', IEEE Transactions on Medical Imaging 39(12):3855-3867 (2020), doi:10.1109/TMI.2020.3006445, https://doi.org/10.1109/TMI.2020.3006445
  4. Xin Liu, Tianyang Zhou, Mengyang Lu, Yi Yang, Qiong He, Jianwen Luo, 'Deep Learning for Ultrasound Localization Microscopy', IEEE Transactions on Medical Imaging 39(10):3064-3078 (2020), doi:10.1109/TMI.2020.2986781, https://doi.org/10.1109/TMI.2020.2986781
  5. Léo Milecki, Jonathan Porée, Hatim Belgharbi, Chloé Bourquin, Rafat Damseh, Patrick Delafontaine-Martel, Frédéric Lesage, Maxime Gasse, Jean Provost, 'A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy', IEEE Transactions on Medical Imaging 40(5):1428-1437 (2021), doi:10.1109/TMI.2021.3056951 (later version: arXiv:2310.08143), https://doi.org/10.1109/TMI.2021.3056951
  6. U-Wai Lok, Chengwu Huang, Ping Gong, Shanshan Tang, Lulu Yang, Wei Zhang, Yohan Kim, Panagiotis Korfiatis, Daniel J. Blezek, Fabrice Lucien, Rongqin Zheng, Joshua D. Trzasko, Shigao Chen, 'Fast super-resolution ultrasound microvessel imaging using spatiotemporal data with deep fully convolutional neural network', Physics in Medicine & Biology 66(7):075005 (2021), doi:10.1088/1361-6560/abeb31, https://doi.org/10.1088/1361-6560/abeb31
  7. Katherine G. Brown, Scott C. Waggener, Arthur D. Redfern, Kevin Hoyt, 'Faster super-resolution ultrasound imaging with a deep learning model for tissue decluttering and contrast agent localization', Biomedical Physics & Engineering Express 7(6):065035 (2021), doi:10.1088/2057-1976/ac2f71, https://doi.org/10.1088/2057-1976/ac2f71
  8. Katherine G. Brown, Arthur D. Redfern, 'Localization with deep learning networks for super-resolution ultrasound imaging', Journal of the Acoustical Society of America 153(3 Supplement):A242 (2023), ASA meeting abstract, doi:10.1121/10.0018773, https://doi.org/10.1121/10.0018773
  9. YiRang Shin, Matthew R. Lowerison, Yike Wang, Xi Chen, Qi You, Zhijie Dong, Mark A. Anastasio, Pengfei Song, 'Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy', Nature Communications 15:2932 (2024), doi:10.1038/s41467-024-47154-2, https://doi.org/10.1038/s41467-024-47154-2
  10. Sepideh K. Gharamaleki, Brandon Helfield, Hassan Rivaz, 'Ensemble Learning for Microbubble Localization in Super-Resolution Ultrasound', arXiv:2411.07376 (2024), https://arxiv.org/abs/2411.07376
  11. Sepideh K. Gharamaleki, Brandon Helfield, Hassan Rivaz, 'Evaluating Detection Thresholds: The Impact of False Positives and Negatives on Super-Resolution Ultrasound Localization Microscopy', arXiv:2411.07426 (2024), https://arxiv.org/abs/2411.07426
  12. Brice Rauby, Paul Xing, Jonathan Porée, Maxime Gasse, Jean Provost, 'Pruning Sparse Tensor Neural Networks Enables Deep Learning for 3D Ultrasound Localization Microscopy', IEEE Transactions on Image Processing 34:2367-2378 (2025), doi:10.1109/TIP.2025.3552198 (preprint: arXiv:2402.09359), https://doi.org/10.1109/TIP.2025.3552198
  13. Yuhang Zheng, Jianqiao Zhou, 'Deep Learning in Ultrasound Localization Microscopy', Advanced Ultrasound in Diagnosis and Therapy 8(3):86-92 (2024), doi:10.37015/AUDT.2024.240023, http://journal38.magtechjournal.com/auidat/EN/10.37015/AUDT.2024.240023
  14. Yang Cai, Shaoyuan Yan, Long Xu, Yanfeng Zhu, Bo Li, Kailiang Xu, 'Deep-Learning Based Super-Resolution Functional Ultrasound Imaging of Transient Brain-Wide Neurovascular Activity on a Microscopic Scale' (SR-fUS), arXiv:2511.14071 (2025), https://arxiv.org/abs/2511.14071
  15. Jaime Parra Raad, Beatriz Laureano, Leong Fan Fung, Daniel Lock, Kumar Ramnarine, Kirsten Christensen-Jeffries, 'Real-Time Processing of 2D and 3D Ultrasound Localisation Microscopy: from Radiofrequency to Super-Resolution', 2025 IEEE International Ultrasonics Symposium (IUS) (2025), doi:10.1109/IUS62464.2025.11201259, https://doi.org/10.1109/IUS62464.2025.11201259
  16. Sebastian Kazmarek Præsius, Lasse Thurmann Jørgensen, Jørgen Arendt Jensen, 'Real-Time Super-Resolution Ultrasound Imaging Using the Erythrocytes', IEEE Transactions on Ultrasonics 73(5):471-481 (2026), doi:10.1109/TUSON.2026.3683326, https://doi.org/10.1109/TUSON.2026.3683326
  17. Hyojin Seong, Jinhwan Jung, Dongkyu Jung, Nizar Guezzi, Sangwoo Nam, Sangheon Lee, Muhammad Noman, Taehoon Her, Eungyeong Cho, Heechul Yoon, Taeyoung Lee, Jung Ho Hyun, Jaesok Yu, 'Ultrasound localization microscopy lite (ULM lite): ultrasound localization microscopy with resource-efficient signal processing scheme', Ultrasonics 159:107849 (2026), doi:10.1016/j.ultras.2025.107849, https://doi.org/10.1016/j.ultras.2025.107849
  18. Su Yan, Clara Rodrigo Gonzalez, Vincent C. H. Leung, Herman Verinaz-Jadan, Jiakang Chen, Matthieu Toulemonde, Kai Riemer, Jipeng Yan, Clotilde Vié, Qingyuan Tan, Peter D. Weinberg, Pier Luigi Dragotti, Kevin G. Murphy, Meng-Xing Tang, 'CycleULM: A unified label-free deep learning framework for ultrasound localisation microscopy', arXiv:2603.09840 (2026), https://arxiv.org/abs/2603.09840