image processing | VALIANT /valiant Vanderbilt Advanced Lab for Immersive AI Translation (VALIANT) Thu, 21 Nov 2024 17:52:12 +0000 en-US hourly 1 Overcoming Labeled Data Barriers in Deep Ultrasound Imaging /valiant/2024/11/21/overcoming-labeled-data-barriers-in-deep-ultrasound-imaging/ Thu, 21 Nov 2024 17:52:12 +0000 /valiant/?p=3340  

Pan, Y.-C.; Vienneau, E.; Lefevre, R.; Eagle, S.; Byram, B. “” European Signal Processing Conference, 2024, pp. 770-774.

 

Deep networks have significantly advanced medical imaging. Initially, they were used for diagnosing conditions by interpreting images (classification). More recently, they are being applied to create the images themselves (estimation). This study focuses on ultrasound imaging and explores a method to address challenges with unlabeled data.


Fig. 1.

In (A), we show our method for resolving domain shift, providing us with the mapsGSTandGTSbetween simulated andin vivodata and the reverse. In (B), we then train a deep beamformer simultaneously regressing on sims andin vivoproxies. The two data types are both allowed to contribute to the beamformer, but augmented feature mapping is used, so that any aspects of the beamforming that are distinct to the simulated orin vivodata are preserved.

]]>
Frequency-independent dual-tuned cable traps for multi-nuclear MRI and MRS /valiant/2024/11/21/frequency-independent-dual-tuned-cable-traps-for-multi-nuclear-mri-and-mrs/ Thu, 21 Nov 2024 17:20:32 +0000 /valiant/?p=3317 Yang, Y.; Lu, M.; Yan, X. “” Journal of Magnetic Resonance, Volume 368, 2024, Article 107786,

MRI and MRS scans often require two systems: one for regular hydrogen imaging (used to map anatomy) and another for special scans of other nuclei (X-nuclei). To prevent interference between these systems, devices called cable traps block unwanted electrical currents.

This study introduces a new type of cable trap that works for both systems without needing separate traps. It combines two types of traps to block high and low frequencies independently but efficiently. Tests and simulations showed that the best setup uses one type of trap for the special X-nuclei system and another for the regular hydrogen system. This design proved effective across different conditions, improving safety, image quality, and versatility for advanced MRI scans.

Fig. 1.(A) Illustration of standard solenoid cable trap. (B) Illustration of float solenoid balun. (C) Illustration of frequency-independent dual-tuned cable trap.

]]>
Beyond MR Image Harmonization: Resolution Matters Too /valiant/2024/11/21/beyond-mr-image-harmonization-resolution-matters-too/ Thu, 21 Nov 2024 16:59:02 +0000 /valiant/?p=3307 Hays, S.P.; Remedios, S.W.; Zuo, L.; Mowry, E.M.; Newsome, S.D.; Calabresi, P.A.; Carass, A.; Dewey, B.E.; Prince, J.L. “), Volume 15187 LNCS, 2025, pp. 34-44,

Magnetic resonance (MR) imaging is widely used to monitor the body non-invasively, but the results can vary due to differences in scanner hardware, software, and imaging protocols. This variability can create challenges for processing algorithms, which may struggle to handle these differences consistently. To address this, image harmonization is used to reduce these variations and improve the accuracy of tasks like segmentation. However, most harmonization models focus on imaging parameters like inversion or repetition time and overlook the impact of image resolution.

This study evaluates how image resolution affects harmonization by using a pretrained harmonization algorithm. We simulated different 2D image resolutions by altering slice thickness and gaps in high-resolution 3D MR images and analyzed how the harmonization algorithm performs with these changes. Our findings show that low-resolution images cause issues for harmonization, as it doesn’t fully account for resolution and orientation differences. While super-resolution techniques can help address this, they are not always used in practice. This approach highlights the importance of understanding the limits of harmonization algorithms and how resolution affects their reliability, offering guidance for preprocessing steps and ensuring trust in imaging results.

Fig. 1.

Slice thickness occurrences for each MRI image contrast: T1w, T2w, and T2w-FLAIR

]]>
HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis /valiant/2024/11/21/hats-hierarchical-adaptive-taxonomy-segmentation-for-panoramic-pathology-image-analysis/ Thu, 21 Nov 2024 16:48:50 +0000 /valiant/?p=3301 Deng, R.; Liu, Q.; Cui, C.; Yao, T.; Xiong, J.; Bao, S.; Li, H.; Yin, M.; Wang, Y.; Zhao, S.; Tang, Y.; Yang, H.; Huo, Y. “.” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume 15004 LNCS, 2024, pp. 155-166, .

Segmenting large, detailed images of tissue samples in computational pathology is very challenging because tissues have complex structures at different scales. For example, in kidney pathology, you have larger regions like the cortex and medulla, as well as smaller structures like glomeruli, tubules, blood vessels, and various cell types.

To tackle this, we developed a new method called Hierarchical Adaptive Taxonomy Segmentation (HATs). HATs is designed to accurately identify and label these different kidney structures in large panoramic images by using a detailed understanding of kidney anatomy. The key features of HATs include:

  1. A special technique that uses the spatial relationships between 15 different tissue types to create a flexible model that can be applied to different scales, from large regions down to individual cells.
  1. A simplified way of representing these tissue structures in a matrix format, making it easier to process the whole panoramic image.
  1. Integration of a new AI model (EfficientSAM) to help extract features from the images without needing manual inputs, making it more adaptable and efficient.

Our tests showed that HATs effectively combines clinical knowledge and imaging techniques to accurately segment over 15 types of tissue structures. The code for HATs is available at .

Fig. 1.

Knowledge transformation from kidney anatomy to a hierarchical taxonomy tree. This figure demonstrates the transformation of intricate clinical anatomical relationships within the kidney into a hierarchical taxonomy tree. (a) Pathologists examine histopathology in accordance with kidney anatomy. (b) This study revisits kidney anatomy using a hierarchical semantic taxonomy for panoramic segmentation, covering 15 classes across regions, units, and cells. The tree incorporates spatial relationships into a semi-supervised learning paradigm and uses hierarchical scale information as prior knowledge to weigh the relationship between classes.

]]>
Interactive Segmentation Model for Placenta Segmentation from 3D Ultrasound Images /valiant/2024/11/21/interactive-segmentation-model-for-placenta-segmentation-from-3d-ultrasound-images/ Thu, 21 Nov 2024 16:43:50 +0000 /valiant/?p=3295 Li, H.; Oguz, B.; Arenas, G.; Yao, X.; Wang, J.; Pouch, A.; Byram, B.; Schwartz, N.; Oguz, I. “.” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume 15186 LNCS, 2025, pp. 132-142, .

Measuring placenta volume from 3D ultrasound images is important for predicting pregnancy outcomes, but manually outlining the placenta in these images is time-consuming and costly. While automated methods can segment the placenta, they often aren’t reliable enough for consistent use. Recently, interactive deep learning models, inspired by tools like the Segment Anything Model (SAM), have been applied to medical images. These models allow users to provide prompts, helping the model identify and segment the target area, which could make them useful in practice.

However, existing interactive models are not specifically designed for the unique challenges of 3D ultrasound images, which are often noisy. This study tested the performance of several state-of-the-art 3D interactive segmentation models against a “human-in-the-loop” approach, where a person helps guide the model during segmentation. The evaluation used several metrics, including the Dice score, which measures how closely the model’s output matches the manual annotation. A Dice score of 0.95 was considered a successful result.

The findings show that the human-in-the-loop model achieved this high level of accuracy and performed well even with a limited number of user prompts, making it both effective and efficient for segmenting the placenta. The code for this study is available online for further use and testing at .

Fig (1)

(a) The 3D interactive segmentation model (PRISM [10]), illustrated in 2D. (b) Prompt sampling for positive (left) and negative prompts (right). To mimic human behavior, we sample prompts from the FN and FP regions of the current segmentation at each iteration. The initial sampling only has positive prompts.

]]>
Semantic Segmentation and Classification of Active and Abandoned Agricultural Fields through Deep Learning in the Southern Peruvian Andes /valiant/2024/11/21/semantic-segmentation-and-classification-of-active-and-abandoned-agricultural-fields-through-deep-learning-in-the-southern-peruvian-andes/ Thu, 21 Nov 2024 16:39:13 +0000 /valiant/?p=3289 Zimmer-Dauphinee, J.; Wernke, S.A. “.” Remote Sensing, Volume 16, Issue 19, 2024, Article 3546, .

In pre-Hispanic times, the Andean people built massive agricultural systems to support intensive farming in the challenging, mountainous regions of the Southern Peruvian Andes. However, many of these complex systems were abandoned after the 16th-century Spanish invasion due to various demographic, economic, and political crises. This study aims to understand how agriculture in the Andes expanded and declined over time by analyzing both active and abandoned farmlands using satellite images across 77,000 square kilometers of the region.

Traditionally, identifying these fields in satellite images is a slow, manual process. To speed this up, researchers used a deep learning technique called semantic segmentation, which can automatically detect and classify agricultural infrastructure in the images. They first created a training dataset by manually identifying sample areas and then used it to train a neural network model. When comparing the results to manual surveys, the deep learning model proved to be more efficient and accurate at mapping large areas of Andean agricultural land. This approach provides a better way to study and document ancient farming systems on a large scale.

 

Figure 1. Map of the survey region, including a boundary outlining the Titicaca region as defined for this study.

]]>
Radiofrequency-transparent local B0 shimming coils using float traps /valiant/2024/11/21/radiofrequency-transparent-local-b0-shimming-coils-using-float-traps/ Thu, 21 Nov 2024 16:19:16 +0000 /valiant/?p=3262 Liu, C.; Liang, H.; Lu, M.; Gore, J.C.; Sengupta, S.; Yan, X. “.” Magnetic Resonance in Medicine, 2024,

In high-field MRI, uneven magnetic fields (B0 inhomogeneities) can lead to poor image quality. A technique called multicoil shimming uses small coils to correct this issue, but traditional coils can interfere with the MRI’s radiofrequency (RF) signals, further reducing image quality. To solve this, a new type of coil has been developed that fixes the magnetic field problem without disrupting the RF signals. The design includes special features that prevent interference, allowing the coil to be placed near the MRI’s main coils without causing issues. Tests showed that this new coil improves magnetic field uniformity, particularly near metal implants, and reduces image distortion while maintaining the quality of the MRI signal. This innovation could enhance MRI scans without requiring major changes to existing equipment.

 

F I G U R E 1

Design and construction of the transparent direct-current (DC) coil. (A) Schematic diagram of the terminated capacitor configuration, showing the shorted and terminated capacitors of the balun. This figure demonstrates a float balun with the terminated capacitors positioned at one end. Note that they could be terminated at both ends or in the middle. (B) Cross-sectional view of the balun illustrating the placement of copper foil and multi-turn wires. (C) Schematic of the complete coil design, including multiturn wires for DC current and float radiofrequency (RF) baluns. (D) Photograph of a single float RF balun. (E) Another view of the float RF balun demonstrating its capacitor soldering. (F) Top view of the normal DC coil in a square shape, showing the coil configuration and copper wire layout. (G) Top view of the transparent DC coil, highlighting the arrangement of the float RF baluns and multiturn wires. ID, inner diameter; OD, outer diameter.

]]>
Systematic Aerial Imaging and Documentation of the Archaeological Site of Huari: An Update and Perspectives for the Future /valiant/2024/11/21/systematic-aerial-imaging-and-documentation-of-the-archaeological-site-of-huari-an-update-and-perspectives-for-the-future/ Thu, 21 Nov 2024 16:10:32 +0000 /valiant/?p=3253 Jennings, Justin; Wernke, Steven; Berquist, Stephen et al. “IMÁGENES AÉREAS SISTEMÁTICAS Y DOCUMENTACIÓN DEL SITIO ARQUEOLÓGICO DE HUARI: UNA ACTUALIZACIÓN Y PERSPECTIVAS PARA EL FUTURO” (). ܲԲá (Arica), Volume 56, Issue 1, 2024, pp. 23-44. .

The Huari site, the largest city ever built in Pre-Columbian South America, was the center of a powerful political formation that lasted for approximately four centuries during the Middle Horizon period (600–1000 CE). Recognized as an early center of Andean civilization by Cieza de León in the 16th century and first extensively studied by Julio C. Tello in 1931, the significance of the site has been well-established through nearly a century of archaeological research. Despite this, the enormous scale of the city has made it difficult to fully understand its spatial organization. Currently, only basic sketch plans based on aerial imagery exist, with some sectors more thoroughly documented through on-the-ground surveys.

To gain a clearer understanding of the site, the Royal Ontario Museum and ý launched a joint documentation and modeling project in 2017. This project focused on a 2 km² area of the city that includes standing architecture. The research team employed drones (UAVs) to capture high-resolution imagery of the site, resulting in orthomosaics—detailed composite images created by stitching together aerial photos. These orthomosaics, with a resolution as fine as ~3 cm, are now available for public exploration and download through the Huari Mapping Project website (), and the platform will be updated as more imagery becomes available.

This article discusses the history of previous mapping and documentation efforts at the Huari site, as well as the methodology used in the 2017 project. The resulting imagery and 3D models will support ongoing research and heritage preservation initiatives, allowing for a more comprehensive understanding of Huari’s layout and its significance in Andean history.

Figura 2. Mapa de Huari de William Isbell, Patricia Knobloch, y Katharina Schreiber, 1970.

Map of Huari by William Isbell, Patricia Knobloch, y Katharina Schreiber, 1970.

 

 

]]>
Mitigating Over-Saturated Fluorescence Images Through a Semi-Supervised Generative Adversarial Network /valiant/2024/09/22/mitigating-over-saturated-fluorescence-images-through-a-semi-supervised-generative-adversarial-network/ Sun, 22 Sep 2024 03:58:25 +0000 /valiant/?p=2994 Bao, Shunxing, Guo, Junlin, Lee, Ho Hin, Deng, Ruining, Cui, Can, Remedios, Lucas W., Liu, Quan, Yang, Qi, Xu, Kaiwen, Yu, Xin, Li, Jia, & Li, Yike. (2024). Mitigating over-saturated fluorescence images through a semi-supervised generative adversarial network. In Proceedings of the 21st IEEE International Symposium on Biomedical Imaging (ISBI 2024), Athens, Greece, May 27-30, 2024. https://doi.org/10.1109/ISBI56570.2024.10635687

This study addresses a key challenge in multiplex immunofluorescence (MxIF) imaging, a technique used in biomedical research to provide detailed insights into cell structures and spatial organization. While MxIF imaging, such as using DAPI staining to identify cell nuclei and CD20 staining for cell membranes, is invaluable for understanding cell composition, it suffers from saturation artifacts. These artifacts occur when certain areas of the image become overly bright, making it difficult to analyze individual cells accurately. Existing methods for correcting these saturation issues, like gamma correction, often fall short because they assume uniform saturation, which is rarely the case in practice.

The authors propose a novel solution using a hybrid generative adversarial network (GAN) called HD-mixGAN, which combines two different types of neural networks (CycleGAN and Pix2pixHD) to correct saturation artifacts. This approach takes advantage of both small datasets where paired (before and after) images are available and larger datasets that only have unpaired images of over-saturated regions. By generating synthetic data from the unpaired datasets using a CycleGAN and combining it with real data, the model effectively learns to correct saturation artifacts, improving the overall image quality.

The method was tested in a task to detect cell nuclei, where it significantly outperformed traditional methods, improving the accuracy (F1 score) by 6%. This approach represents the first focused effort to address saturation issues in multi-round MxIF imaging, providing a data-driven solution that enhances the accuracy of single-cell analysis. The study also makes its code and implementation freely available, facilitating further research and applications in this area.

This study addresses a key challenge in multiplex immunofluorescence (MxIF) imaging, a technique used in biomedical research to provide detailed insights into cell structures and spatial organization. While MxIF imaging, such as using DAPI staining to identify cell nuclei and CD20 staining for cell membranes, is invaluable for understanding cell composition, it suffers from saturation artifacts. These artifacts occur when certain areas of the image become overly bright, making it difficult to analyze individual cells accurately. Existing methods for correcting these saturation issues, like gamma correction, often fall short because they assume uniform saturation, which is rarely the case in practice.
The authors propose a novel solution using a hybrid generative adversarial network (GAN) called HD-mixGAN, which combines two different types of neural networks (CycleGAN and Pix2pixHD) to correct saturation artifacts. This approach takes advantage of both small datasets where paired (before and after) images are available and larger datasets that only have unpaired images of over-saturated regions. By generating synthetic data from the unpaired datasets using a CycleGAN and combining it with real data, the model effectively learns to correct saturation artifacts, improving the overall image quality.
The method was tested in a task to detect cell nuclei, where it significantly outperformed traditional methods, improving the accuracy (F1 score) by 6%. This approach represents the first focused effort to address saturation issues in multi-round MxIF imaging, providing a data-driven solution that enhances the accuracy of single-cell analysis. The study also makes its code and implementation freely available, facilitating further research and applications in this area.
]]>
SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI /valiant/2024/07/21/se3-equivariant-and-noise-invariant-3d-rigid-motion-tracking-in-brain-mri/ Sun, 21 Jul 2024 21:36:32 +0000 /valiant/?p=2794 Benjamin Billot, Neel Dey, Daniel Moyer, Malte Hoffmann, Esra Abaci Turk, Borjan Gagoski, P. Ellen Grant, & Polina Golland. (2024). . IEEE Transactions on Medical Imaging, 1-12. https://doi.org/10.1109/TMI.2024.3411989

Tracking movement accurately in medical imaging, like MRI scans of the brain, is crucial to get clear and useful images. Traditional methods use a type of artificial intelligence called convolutional neural networks (CNNs) to detect and correct these movements, but CNNs aren’t good at handling rotations of the images.

The researchers have developed a new method called EquiTrack, which uses advanced CNNs that can handle both shifts and rotations in the images. However, these advanced CNNs struggle with noisy images, which are common in medical imaging. To fix this, the researchers combined their CNNs with a noise-reduction tool to separate irrelevant noise from important image features.

EquiTrack was tested on brain MRI images from both adults and fetuses and was found to perform better than current leading methods. This new technique promises more accurate tracking of movement, leading to clearer and more reliable MRI scans. The code for EquiTrack is freely available online for other researchers to use and build upon. Source:

Overview of EquiTrack. The fixed and moving volumes are first processed with a denoising CNN that removes anatomically irrelevant intensity features (noise, histogram shifts, etc.), so that its outputs only differ by the unknown rigid transform. Crucially, we then use a steerable SE(3)-equivariant E-CNN to extract K matching anatomical features across images. A rigid transform Tˆ is estimated by computing summary statistics (centres of mass), providing us with two corresponding point clouds that are registered with a differentiable closed-form algorithm [28].
]]>