TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Xueqi Guo; Luyao Shi; Xiongchao Chen; Qiong Liu; Bo Zhou; Huidong Xie; Yi Hwa Liu; Richard Palyo; Edward J. Miller; Albert J. Sinusas; Lawrence Staib; Bruce Spottiswoode; Chi Liu; Nicha C. Dvornek

doi:10.1016/j.media.2024.103190

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Xueqi Guo^*, Luyao Shi, Xiongchao Chen, Qiong Liu, Bo Zhou, Huidong Xie, Yi Hwa Liu, Richard Palyo, Edward J. Miller, Albert J. Sinusas, Lawrence Staib, Bruce Spottiswoode, Chi Liu, Nicha C. Dvornek

^*Corresponding author for this work

Radiology

Research output: Contribution to journal › Article › peer-review

Abstract

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (⁸²Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical ⁸²Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

Original language	English (US)
Article number	103190
Journal	Medical Image Analysis
Volume	96
DOIs	https://doi.org/10.1016/j.media.2024.103190
State	Published - Aug 2024

Funding

This work was supported by the National Institutes of Health (NIH), United States under grant R01CA224140. An early version of this work was accepted for publication by Springer Nature at SASHIMI: Simulation and Synthesis in Medical Imaging, a workshop of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2023 (Guo et al. 2023b). Here, we included a more comprehensive analysis of the complete evaluation using simulated motion and ablation studies, a comparison of the conventional non-rigid and deep learning-based motion correction methods, an evaluation of an independent cohort of real-patient motion cases, additional background and implementation details, and in-depth discussions considering the viability of clinical use and prospective future directions.

Keywords

Dynamic cardiac PET
Early-to-late frame conversion
Inter-frame motion correction

ASJC Scopus subject areas

Radiological and Ultrasound Technology
Radiology Nuclear Medicine and imaging
Computer Vision and Pattern Recognition
Health Informatics
Computer Graphics and Computer-Aided Design

Access to Document

10.1016/j.media.2024.103190

Cite this

Guo, X., Shi, L., Chen, X., Liu, Q., Zhou, B., Xie, H., Liu, Y. H., Palyo, R., Miller, E. J., Sinusas, A. J., Staib, L., Spottiswoode, B., Liu, C., & Dvornek, N. C. (2024). TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction. Medical Image Analysis, 96, Article 103190. https://doi.org/10.1016/j.media.2024.103190

@article{9881f8263f3d4c06b82a4bc9d82f2a50,

title = "TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction",

abstract = "Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.",

keywords = "Dynamic cardiac PET, Early-to-late frame conversion, Inter-frame motion correction",

author = "Xueqi Guo and Luyao Shi and Xiongchao Chen and Qiong Liu and Bo Zhou and Huidong Xie and Liu, {Yi Hwa} and Richard Palyo and Miller, {Edward J.} and Sinusas, {Albert J.} and Lawrence Staib and Bruce Spottiswoode and Chi Liu and Dvornek, {Nicha C.}",

note = "Publisher Copyright: {\textcopyright} 2024 Elsevier B.V.",

year = "2024",

month = aug,

doi = "10.1016/j.media.2024.103190",

language = "English (US)",

volume = "96",

journal = "Medical Image Analysis",

issn = "1361-8415",

publisher = "Elsevier",

}

Guo, X, Shi, L, Chen, X, Liu, Q, Zhou, B, Xie, H, Liu, YH, Palyo, R, Miller, EJ, Sinusas, AJ, Staib, L, Spottiswoode, B, Liu, C & Dvornek, NC 2024, 'TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction', Medical Image Analysis, vol. 96, 103190. https://doi.org/10.1016/j.media.2024.103190

TY - JOUR

T1 - TAI-GAN

T2 - A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

AU - Guo, Xueqi

AU - Shi, Luyao

AU - Chen, Xiongchao

AU - Liu, Qiong

AU - Zhou, Bo

AU - Xie, Huidong

AU - Liu, Yi Hwa

AU - Palyo, Richard

AU - Miller, Edward J.

AU - Sinusas, Albert J.

AU - Staib, Lawrence

AU - Spottiswoode, Bruce

AU - Liu, Chi

AU - Dvornek, Nicha C.

PY - 2024/8

Y1 - 2024/8

N2 - Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

AB - Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

KW - Dynamic cardiac PET

KW - Early-to-late frame conversion

KW - Inter-frame motion correction

UR - http://www.scopus.com/inward/record.url?scp=85194456534&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85194456534&partnerID=8YFLogxK

U2 - 10.1016/j.media.2024.103190

DO - 10.1016/j.media.2024.103190

M3 - Article

C2 - 38820677

AN - SCOPUS:85194456534

SN - 1361-8415

VL - 96

JO - Medical Image Analysis

JF - Medical Image Analysis

M1 - 103190

ER -

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this