Poster + Paper
15 February 2021 Deep learning-based 3D image generation using a single 2D projection image
Yang Lei, Zhen Tian, Tonghe Wang, Justin Roper, Kristin Higgins, Jeffrey D. Bradley, Walter J. Curran, Tian Liu, Xiaofeng Yang
Author Affiliations +
Conference Poster
Abstract
Due to the inter-fraction and intra-fraction variation of respiratory motion, it is highly desired to provide real-time volumetric images during the treatment delivery of lung stereotactic body radiation therapy (SBRT) for accurate and active motion management. Motivated by this need, in this study we propose a novel generative adversarial network integrated with perceptual supervision to derive instantaneous 3D image from a single 2D kV projection. Our proposed network, named TransNet, consists of three modules, i.e., encoding, transformation, and decoding modules. Rather than only using image distance loss between the generated 3D image and the ground truth 3D CT image to supervise the network, perceptual loss in feature space is integrated into loss function to force the TransNet to yield accurate lung boundary. Adversarial loss is also used to improve the realism of the generated 3D image. We conducted a simulation study on 20 patient cases, who had undergone 4D-CT scan and received lung SBRT treatments in our institution, and evaluated the efficacy and consistency of our method at four different projection angles, i.e., 0°, 30°, 60° and 90°. For each 3D CT image of a breathing phase in the 4D CT image set, we simulated its 2D projections at these two angles. Then for each projection angle, a patient’s 3D CT images of 9 phases and the corresponding 2D projection data were used for training, with the remaining phase used for testing. The mean absolute error (MAE), peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) achieved by our method are 99.5±13.7HU, 23.4±2.3dB and 0.949±0.010, respectively. These results demonstrate the feasibility and efficacy of our method on generating a 3D image from single 2D projection, which provides a potential solution for in-treatment real-time on-board volumetric imaging to guide treatment delivery and ensure the effectiveness of lung SBRT treatment.
© (2021) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Yang Lei, Zhen Tian, Tonghe Wang, Justin Roper, Kristin Higgins, Jeffrey D. Bradley, Walter J. Curran, Tian Liu, and Xiaofeng Yang "Deep learning-based 3D image generation using a single 2D projection image", Proc. SPIE 11596, Medical Imaging 2021: Image Processing, 115961V (15 February 2021); https://doi.org/10.1117/12.2580796
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
3D image processing

Lung

Computed tomography

4D CT imaging

Computer programming

Integration

Radiotherapy

Back to Top